Building Secure Services in the Cloud

Many security professionals are skeptical about cloud-based services and infrastructure. But it’s a skepticism we’ve seen before, when a new computing paradigm encounters a suspicious—if not downright hostile—mindset (data-center-centric) and installed base. In this paper we will discuss some of the general philosophies and perspectives that will assist anyone who wants to securely leverage the benefits the cloud by using its strengths to overcome issues that have traditionally been labeled as weaknesses.

As with any paradigm shift, cloud computing requires different rules and different logic. Operationally, security organizations need to change their thinking and processes from traditional data-center-centric models to new, more statistical models. For example, we exchange hands-on control over physical hardware with odds over a population of hardware. From a systems administration perspective, we exchange scripts and manual capacity planning and scaling with API calls, triggers, feedback loops and the automatic provisioning and de-provisioning of spot-bid compute resources. And, of course, from a security perspective, we face the challenge of a world that is not under our complete physical control.

While many veteran security professionals react to the cloud with suspicion and outright hostility, there are two realities: 1) the cloud is here to stay; and 2) it gives us an extremely powerful new set of tools for securing this environment. When properly leveraged, cloud-based IaaS offers availability benefits beyond what many enterprises can easily achieve on their own. By employing a combination of automation, integration with IaaS provider’s APIs and thoughtful system design you can achieve a level of security that is actually higher than most legacy in-house services currently provide.

“There are two realities: 1) the cloud is here to stay; and 2) it gives us an extremely powerful new set of tools for securing this environment.”

With a few well-placed API calls you can have a scalable army of hardened, patched, encrypted, scalable and disposable virtual machine resources at your disposal, ready to do your bidding, securely deliver your results and then self-destruct before returning from whence they came. Of course, doing this and leveraging it properly requires a different thought process than we may be accustomed to, but the reward can be well worth the shift in thinking.

Design Design Design

Defense in depth is traditionally a matter of strict design principles and security policies distributed across a number of departments and areas of expertise. In a system designed for the cloud you have the tools to design, implement and refine your policies, controls and enforcement in a streamlined and centralized fashion. The tools exist to add security at the network layer (with security groups, access management, host-level firewalls and VPNs), at the O/S layer, (with encrypted storage, strict privilege separation, and ruthlessly hardened systems), at the application layer (with the latest updates and thoroughly enforced policies) as part of your design and development cycle, rather than as part of ongoing operational maintenance.

“Your paper becomes reality, with no need to move cables, rename hosts, or worry about maximizing the ROI on a piece of equipment that is no longer relevant.”

One major advantage to deploying services in the cloud is the freedom to design your network and security measures from the ground up and implement your secure designs in code, which is not subject to the same concerns you have in a physical data center or hosting facility. Legacy compromises, rogue cross-over cables and obsolete equipment and software can all be things of the past. APIs such as those from Amazon Web Services allow you to design an entire network exactly the way you would like it to be implemented and to then recreate that network, complete with firewall rules, the latest security updates, and value-added IaaS tools such as identity and access management. The ability to re-size your storage, memory, bandwidth and compute dynamically or through your release-cycle to suit new designs and business needs removes the final layer of hardware management and multi-factor capacity planning inherent in large home-grown or hosted virtual machine deployments. When you need to re-deploy or re-scale your infrastructure, there is no need to worry about legacy issues that would prevent you from making the types of sweeping changes that look so good on paper. Your paper becomes reality, with no need to move cables, rename hosts, or worry about maximizing the ROI on a piece of equipment that is no longer relevant.

Cloud tools allow you to take security management to a new level by enabling you to fully automate your controls and tests. By moving the systems administration away from distributed scripts and systems administrators and into the hands of production-ready code–which can be rigorously reviewed, tested and updated along with the rest of your service–you can achieve a scale and ease of management unthinkable in traditional paradigms.

In this new paradigm, you are free to design your system with all of the security controls you could ever want but were probably never able to achieve in a brick-and-mortar data-center or hosting facility. Since your entire infrastructure is ephemeral, the best approach is to automate your deployments leveraging the cloud-based tools that allow you to make the installation, baselining and management of things like file-integrity-checkers trivial, so that all of your virtual machines can have file-integrity software and baselines built in from the ground up. By using APIs to programmatically assign virtual machines to role-based security groups that are well-designed in advance you can scale your network to massive sizes without ever having to worry about firewall rule ordering, optimization or audit as part of your operational cycle.

Some IaaS providers allow you to build your own virtual private network of virtual machines according to your own network topology. This affords some advantages in terms of leveraging predictable host-names and allows you to employ an network-layer protections such as Intrusion Prevention Sensors (IPS) or Web-Application Firewalls (WAF) that are available as virtual machine appliances or that can run as software on your platform. These additional layers of protection and convenience allow you to leverage some of the successful technologies which were designed within the data-center paradigm and still incorporate these controls into your SDLC and minimize operational cost.

With the kind of programmatic flexibility brought to bear by cloud APIs you have the ability to engineer a system with security built in at every level, and the scaling and management of those controls has never been easier.

As a result of this transformational new paradigm we have to focus on the design of our security systems and leverage the reliability and automation that cloud providers afford us to operate securely in this new environment.

This paper does not attempt to cover every best practice one should employ in order to build secure and scalable systems in the cloud, but we will discuss some of the foundational design principals which will help guide you in your thinking as you design such systems.

Defense in Depth. Everything. All the Time.

A cloud-based service needs to be thought about holistically, as an integrated system. This system has layers, components, interfaces and interactions, which are all under your control and programmatically scaled to wherever you set the dial. Each of these factors needs to be carefully considered from a security perspective that flows from a central design.

Data needs to be considered in its three elemental forms: at rest, in motion, and in use. You also need to be able implement and monitor access control across all of the various virtual machines and applications (monitoring applications, third-party applications, development resources). Interfaces and APIs need to be scrubbed as clean as possible and limited to authenticated users and systems. Keys must be stored away from locks… All of the traditional rules need to still apply, but can be executed in new and efficient ways.

There are a lot of details, but at the heart of it is the system, with inputs, outputs, storage, memory, and transport. Each of those must be thought of on its own and in combination with the other components it interacts with. It is both that simple and that complicated.

Cloud IaaS providers and their partners offer a multitude of mature tools to assist with things like key-management, user management, Access/Authentication/Audit, load-balancing, caching, messaging, volume management, and much more. Leveraging these tools and interfaces can drastically reduce the engineering effort required to implement and manage your security controls.

If you properly leverage your design and automation tools, your security will become fractal, and embedded in every layer of your system as it scales and evolves.

All Things Are Possible With Automation

Thinking of your entire infrastructure as part of your code-base changes the game completely in terms of what you are able to achieve. There is no longer a gap or disconnect between the operational physical layer and the software that runs on top of it. By simply changing your thinking, machine and network failures are now simply exceptions to be caught and handled by your system. New boxes can brought online in real-time to replace or reinforce your existing fleet, new capacity can be bid on and purchased, and your infrastructure can now evolve and support your system because it is part of the system.

From a security perspective this coupling enables you to use your infrastructure to adapt automatically. For instance, a service registry can be kept which keeps the IP addresses and ports of all of the registered services in your system and your code can use your cloud provider’s APIs to restrict network communication to only the ip-addresses and ports which are required by the system to function. SSL/TLS services can generate new key-pairs and shared secrets and securely store them on encrypted volumes every time you deploy your service. Host firewalls, host-IDS and integrity-checkers can be configured based on the tags you assign to each virtual instance. All of these measures can be unit-tested and QA’d as part of your Secure Development Life-Cycle (SDLC), allowing you to rapidly develop enhancements in pace with your product.

Less is More

Simplicity leads to security. Simplicity of design, of interfaces, and of data-flow all help lead to a secure and scalable system.

Reduce, Reuse, Recycle

Keeping APIs and other interfaces simple, clean and minimal, designing-in code-re-use and centralizing configuration information will help keep your attack-surface to a minimum as well as allow for easier troubleshooting and easier, faster turn-around on any security-related fixes that need to be made.

Do the Right Thing

Your system has I/O, storage, memory and network underneath it, as well as your software components. Think about every place that information is exchanged, transferred or transformed in your system and make sure you are doing the right thing there. If this is input, sanitize it. If it is storage, network or memory (where possible) encrypt it. If it is output you are feeding back to your customer or another component, sanitize that too. Don’t trust client-side verification, instead enforce everything at every layer.

Leave No Trace

You can safely use these giant blocks of anonymous pay-by-the-hour compute by leaving a minimal footprint, minimizing your attack surfaces and eliminating the chances that you’ll accidentally leave sensitive data exposed.

Default Deny/Whitelist

Allow only expected connections. If your front-end web-application needs to accept connections from anyone in the world so be it (but it’s more likely your load balancer does, in which case your web-app instances should only talk to the load-balancers). As part of your infrastructure as software design, you know what needs to talk to what and on what port and under what circumstances, so you should only allow that, everything else is bit-bucketed and alerted on. In software-driven cloud-based deployments, there is no longer any excuse for any other way of doing it.

Encrypt It All

At rest, in motion, and in use; any data that is ephemeral can be kept on encrypted ephemeral storage and the keys can simply be kept in memory. When the instance dies, the key dies with it.

Longer-lived data should be stored away from the keys that secure it. If the data is particularly sensitive, in addition to keeping it encrypted, you can program your system to securely wipe the data using any of the various best-practice protocols for secure data deletion before spinning down the disk and giving it back to the pool. Tools are readily available for installation on your virtual machines, which make this secure data deletion simple and easily automated.

Everything Else is Disposable

Ephemeral is a powerful concept once you embrace it. Secure and encrypt everything you care about, the rest of your virtual machine fleet can be re-booted, terminated, or sacrificed.

It’s Easier Than You Think

Abandoning your legacy infrastructure and centralizing your security design and enforcement within your code-base will ultimately allow for greater security at lower human and capital cost.

Joan Pepin is an Information Security veteran with 15 years experience in healthcare, manufacturing, defense, ISPs and MSSPs. A recognized expert in security policy and lifecycle management, she is the inventor of SecureWorks’ Anomaly Detection Engine and Event Linking technologies.. Joan has achieved noteworthy success in her nine years with the Guardent/VeriSign/Secureworks organization and is now the Director of Security for Sumo Logic. Her specialties include policy management, security metrics, incident response and security thought-leadership: she was a keynote speaker at 2008 Forrester Security Summit and presenter at 2008 Gartner Security Summit CSO Series.