CloudPassage Cloud Security Product Review

Updated:I came across a white paper published by CloudPassage, most likely in the attempt to drive interest in their cloud security product.  It worked, since it resulted in my doing a little digging into their product.  They did bring up a security threat I hadn’t thought of and it was just enticing enough to get me to investigate their offering a little further.  Their paper was focused on Infrastructure as a Service (IaaS) environments like Amazon Web Services (AWS) (reviewed here), or Rackspace (reviewed here).  The CloudPassage paper focused on the security threats within the context of the products that they offer in order to mitigate these threats.

I will review some of the key points made by CloudPassage here, and then I’ll review their solution offering.   Hang on though -- this article assumes you understand cloud IaaS, and some general Linux and security topics.

Typically, cloud hosting providers like AWS don’t take any responsibility for the security of a client’s host environment for IaaS.  How can they?  Clients have full access to do whatever they desire because they have full administrator access to their compute environments.  Providers do take responsibility for things that they can control, like physical security and security of systems that provide provisioning and edge network services.  In addition, I would expect hosting providers take security responsibility for Platform as a Service (PaaS) like Amazon’s S3 storage.

Customers of IaaS providers are expected to use the services offered by the provider, and implement local software solutions and tactics to implement secure compute environments to protect against various threats.  This is where companies like CloudPassage come in.  They offer the ability to centrally manage entire clusters of systems, lowering administrative costs and presumably improving security via consistency.

The Magnetism of the Cloud

The white paper’s author asserts that “Cloud Servers attract e-Criminals”.  There was no data given to support this statement other than relying on intuition that big targets attract more attention.  I don’t think the cloud is much different than locally hosted public-facing solutions.  Although it might be a huge deal if Rackspace provisioning servers were penetrated.  It’s more likely that your systems will be part of another hunting scan.  If your public-facing systems are found to be vulnerable, the next phase of penetration of your systems will likely begin (think of the insidious drilling machines from the movie “Matrix Revolutions”).  This doesn’t really change if you move to the cloud.

The Clouds lack of DMZ

CloudPassage suggests that “Servers have more exposure in the cloud” because of the lack of demarcation zones (DMZ), thus more servers are exposed.  Most companies implement DMZs as a way to firewall data and nonessential computing from public access to reduce risk.  Many times this is implemented via the use of hardware firewall equipment.  Regarding lack of a DMZ configuration, AWS implements DMZ configuration via “Security Groups” which is easy to setup (they are limited to inbound rules only) and they offer virtual private nets with additional network configurations.  Furthermore, with the ability to configure “private clouds”, administrators have a little more control.  DMZs can be implemented at Rackspace as well but require more work and are expensive, as customers need to implement  a private cloud on dedicated hosts.  Many hosting providers do not make setting up a DMZ for a compute environment easy and some don’t support it at all leaving their customers to implement local firewall rules to achieve the same thing.

The lack of being able to have control over high speed firewalls that are tuned specifically for customer’s environment is a downside.  Many companies use Linux iptables to further define what ports are open and to whom.  Cloud customer can use iptables or other methods, but there is where CloudPassage makes things easy.  More on that later.

The Elasticity of Clouds Opens a New Attack Vector

CloudPassage submits that if you have an infected machine image that is used to quickly scale a compute environment, the infection blooms when that environment starts to scale (manual or auto-scaling).  This is a perfect environment for a botnet infestation.  In addition, images that are used for replication slowly become out of date due to missing configuration in your security plan or software patches.

This is one area risk that I agree with and they make a very good point.  One misconfigured system could be the “Typhoid Mary” of a cluster.  CloudPassage offers a solution (no surprise) that allows customers to manage security firewall settings, scan and monitor from one place.  They also offer some other centralized services for authorization.  Although there are several solutions on the market for scanning and configuration, CloudPassage’s differentiation may be in their very thin (CPU wise) approach and the use of a centralized console to manage and monitor all of this.

Using The Cloud for Development Isn’t Without Risk

The CloudPassage white paper’s final point focused on the use of public clouds for development.  The threat focused on the assumption that development environments are loosely managed and are therefore more vulnerable (probably true).  If that company were to just move an infected virtual machine in the cloud to locally hosted system, the infection moves with it.  This was a good point and it’s an infection vector cloud customers should be aware of.  I think that CloudPassage’s hidden point here is that if it were easy to secure, it’s more likely to be protected.  Not mentioned was that penetrated development systems not only release code into the wild, they hold the potential for a hacker to insert backdoors and other nasties into your production application.

The CloudPassage Solution

CloudPassage approach to the threats mentioned above is a product called HALO.  The product is a software as a service (SaaS) approach  to security.  The solution is a combination of a small thin daemon that runs on Linux or Windows servers, and a SaaS console for command, control and reporting.  It provides network firewall, vulnerability assessment via auditing and monitoring.   The advantage may be in having a very thin daemon that is installed on each of the VMs.  The other selling point is the centralized management of iptables on Linux or firewall settings on Windows brought together.  The HALO daemon is installed on each virtual machine and then controlled from a web based console hosted by CloudPassage.  The console, or portal as they refer to it,  is also used to report status, threats, and other audit reporting from the compute cluster.

Centralized Firewall Configuration Management

Having a single place to manage firewall security settings for Linux and Windows will go a long ways toward reducing the security holes due to mistakes in system configuration management.  Centralizing this also lowers costs of managing large farms, alleviating the need to touch 100s of systems when the admin makes a security change.  Assuming template machine images used for “cloud bursting” are configured with the HALO client, the firewall configuration tool reduces the probability that a system cloned from that template will be running with outdated configuration settings.  This doesn’t apply just to the public cloud, this could be very useful to companies that host their own systems (I’ve seen firsthand how difficult configuration management can be for a few servers in a cluster).

Firewall configuration allows for “server groups”, making the task of managing servers with different roles easier.   CloudPassage firewall may enable a virtual acceptable DMZ design for application deployments in the cloud.  AWS has “security groups”, firewall rules that are applied to systems, but AWS security groups only support the use of inbound rules.

Because of the value that CloudPassage brings to Rackspace customers, it’s not surprising to find that CloudPassage is listed in the Rackspace marketplace.

I quickly tested the firewall management console after installing HALO client on my Rackspace Linux server and was surprised at how easy it was to configure different rules for classes of machines.  The console also has the ability to choose from “templates” of recommended firewall rules to get admins started as well.

Multi-Factor Authentication For Network Port Access

GhostPorts multi-factor authentication is a solution that supports SMS or a USB dongle on a client for multi-factor authentication (or more often referred to as two-factor authentication defined in this article).  Ghostports opens network ports only after the one-time password has been entered into their portal.  Multi-factor (or two-factor) authentication methods are common for application layers, this solution works at the network layer and is perfect for SSH and other administrative port access to cloud servers as a secondary protection measure.  GhostPorts could be used for application access as well but it’s meant for locking down ports used to access machines directly. I like the CloudPassage solution agnostic approach.  It has several other features, like allowing temporary access.

I tried this feature and it’s powerful and easy to use.  When I provisioned my Rackspace server for my review of Rackspace, access to the root account was via password.  Although I could set up ssh keys, rotating keys would be difficult in large clusters.  (Note: AWS is one vendor that offers ssh key rotation tools)  I imagine there are a lot of hosting providers that use standard passwords for authentication.  The risk to penetration via ssh is effectively eliminated with GhostPorts.  Many would argue that use of strong passwords (we are professionals after all) would suffice.  But with sophisticated hacking techniques being found all the time, this argument erodes quickly.   I’d prefer the extra layer of security.

I signed up for CloudPassage and quickly installed the HALO daemon on my server.  I then configured an inbound firewall rule that would apply to all of my servers that requires GhostPort access to the ssh port.  Although the rule was set for all GhostPorts users, admins can also specify a single user.  Once the configuration propagated to my server (it took less than a minute), the ssh port was effectively blocked for all users.  To open it just for my IP address, I logged into CloudPassage portal and clicked on “Open GhostPorts”.   A one time key was sent to my cell phone a few seconds later, and I entered that in.  I was able to open a connection to the server within a minute.

Not only would this be good for system level access (like ssh) but also when accessing other administrative tools.  Many applications and middleware tools run administrative web front-ends (like Oracle’s Weblogic Java App engine).  The Weblogic admin server, for example, runs at port 7001 by default. Adding a GhostPorts rule for these ports adds an extra level of protection to other administrative entry points, which I’ve never seen use two-factor authentication.

Under the hood, all GhostPorts is doing is modifying firewall settings on the server (iptables in this case) and adding the user’s particular IP to the list of hosts that can access that port.  After 4 hours, or if the user clicks on “Close GhostPorts”, the rule is removed from the firewall.  Giving authorized user the ability to open a port for her system is handy given that workstations change IP addresses often it enables the use of mobile devices.  There are other ways to handle this (VPN to private network interfaces is one example).

One downside that I could see is that GhostPorts is per user, per IP.  That is, if I had two workstations that I wanted to open a ssh session with, I would have to open it for one, establish a connection, click on “GhostPorts” and then reopen it on the other machine. This wouldn’t work for stateless connections like web access to admin tools.  You could always add a manual rule if desired but it breaks the model.  If GhostPorts were smart, it would recognize that you are using a different machine IP.

Another downside is that if GhostPorts SaaS were unavailable, or the management console couldn’t contact your machine, you could be locked out.  If you use something like GhostPorts, think about these scenarios in your design.

If you are behind a proxy server or NATed subnet, opening a port for that IP opens it for everyone on that server, so be careful. GhostPorts isn’t a replacement for strong authentication on your systems and administrative applications, but it takes your security defenses to the next level with very little effort.  I like it.

A BLOG posting at CloudPassage makes a good point, even if you use “virtual” firewall protection features on your VM provider (like VMware), you may be better off utilizing endpoint firewalls.  Because the configuration of your system’s firewall settings are on that system, a failure by HALO won’t result in your firewall coming down.  Worst case you make changes to your firewall until you can resolve the issue.  This does warrant more research into the various failure modes when designing your cloud deployment.

Security Monitoring / Auditing

CloudPassage has a couple of tools for continuous security auditing in their SaaS offering. The first is a tool for monitoring security configuration.  Not only can customers monitor for simple configuration errors on each system in a cluster, the tool can also monitor files and other configuration mismatches to policy.

The second tool monitors software vulnerability on each of the servers in a cluster.  Examples are misconfigured ssh and other software packages customers may have installed on their systems.  This isn’t vulnerability scanning from the outside but from the inside via their HALO daemon.

CloudPassage file integrity monitoring looks for changes to important system files and binaries.  Being able to constantly monitor critical files could be an early warning system for security teams, allowing them to detect penetration early and take action.

There is also a Vulnerability Scanning report that shows what exploits you may have installed on your environment.  By sending an inventory up to the portal (encrypted), known vulnerabilities can be seen from the inside out.  I'd like to see some patch management added to the tool to compliment this tool.

Account Management

The last two tools allow for centralized system level account management and event logging.  Managing access to a large cloud deployment could be daunting without this.  Although AWS does support ssh keys, that’s more of a group level access.  If you need anything more than that, it could be a real problem.  

I used the CloudPassage account manager to add a system account to my Linux box and it was ready in just a few seconds.  Disabling the account was effective on the system 6 seconds after I pressed the button in the management interface.

I wasn’t able to add an account to all servers in a group, nor was there any sort of UID and group management available so that you could keep track of user’s UIDs.  An admin can add an account to all servers in a server group by using the restful APIs. CloudPassage provides good documentation and examples on how to do this.  This is lame in my opinion.  (More in the evaluation section below).

Event Logging

The problem of event log management is an old one.  In addition, this problem grows with the number of servers that need to be monitored and now with virtual machines, we setup 8x+ the number of server we use to (because it’s so easy).  As your cloud grows, so do the magnitude of logs.  The real issue is deciding what to take action on and what to ignore.  Being able to monitor from a central location, the ability to create rules, and the fact that CloudPassage brings together Windows and Linux platforms makes their HALO Event Monitoring & Alerting worth looking at.

I’m guessing that event logging and alerting may not be as mature as say ArcSight  or TippingPoint (both now owned by HP) but this may be the lowest cost solution bundle out there.

Pricing

Pricing for CloudPassage is similar to many software as a service models, you pay for what you use.  The good news is that they have a “free tier” that allows you to try out some of the CloudPassage tools.  Currently, new accounts receive all of the features of a pro account for 30 days to enable customers to evaluate the service.

After the 30 day pro experience expires, the free tier allows you to secure up to 5 servers with firewall controls (but doesn’t include all of the services). One thing that I noticed is that the pricing page indicated that you needed to sign-up for the “NetSec” tier (the lowest cost after free) in order to use “GhostPorts” two factor authentication.  When I went to sign-up, I found this not to be the case on the signup page.  Cloudpassage informed me that the free tier does NOT include “GhostPorts”.   In addition to not receiving server security, integrity and intrusion detection, the free tier does not include retention of logs, alerts, and other data used for compliance or historical purposes.  The other two tiers retain 2 years of data.

I did find the pricing a bit confusing.  CloudPassage needs to call out exactly what features are in each of the pricing tiers.  Also, pricing is a bit unpredictable because it’s based on $/server hour although they do state a quick estimate when you purchase.

Auditing the Auditors

Because CloudPassage is a software as a service (SaaS) vendor, potential customers should evaluate CloudPassage like they would any other SaaS vendor.  In this case however, keep in mind that if your account or the CloudPassage system were compromised, security of all of your servers is at risk.  (It’s like captain Kirk lowering the shields of his opponent's ship in the movie “Wrath of Khan”).  Given the risk, I suggest evaluating CloudPassage solution using the most stringent security review standards for services in this class (say TOP SECRET).

As of this writing, CloudPassage didn’t have any web pages that described where their servers were located, auditing certifications, or other information for me to evaluate the security of their daemon, or SaaS.  I contacted their support and received some very good information about how they secure the daemon, secure the communications between the grid and the halo daemon, and hw how they protect the CloudPassage grid and the portal.  You can see their detailed response here.  CloudPassage support is currently working on the web page that will describe all of this so I’ll link to that when it’s published.   I spoke with CloudPassage and, as I expected, they are hosted in a major cloud data center in the US and have deployed site-to-site for high availability.

I would like to see CloudPassage portal use two-factor authentication due to the criticality of their service on my environment.  CloudPassage support responded that they would be rolling this feature out in a future release of their portal.  This was reiterated in a second email with my request about how they security their components.

CloudPassage is a relatively new company and I’m guessing pretty small.  Even though they have been around since 2010, I don’t think they had much of a commercial product until 2011, and real traction until 2012 when some of their core products were released.  They are privately held so any financial evaluation can’t be completed at this time.

CloudPassage is staffed with security professionals from the likes of Cisco and judging by the work they have put into the client, there is a pretty good brain trust at the engineering helm.

Support

All of my questions made to the CloudPassage support community were answered within one hour and my two support ticket questions were answered in under two hours (I flagged them as low priority) so I never had the need to use the phone.  Fantastic service.

My Thoughts

I liked CloudPassage’s solution offering and approach.  They’ve built a platform that they can continue to grow using the HALO daemon and centralized management.  Given the level of activity, I expect to see additional features released on a quarterly (or sooner) basis.

The account management features fell way short of what I would expect to see for a cloud account management solution.  There is no way to add an account to more than one server at a time (you can use a simple script API).  Most applications are not going to require adding a lot of system accounts but if I had a new admin join my team, I’m going to add that person to all or a group of systems.  In addition, there is no management of UIDs or groups.  I may be unfair here, because the features I want are more in the realm of an identity management and single sign-on solution... Given how close CloudPassage is, it wouldn’t take much more to enable some of this of capability.

The event monitoring works but I’d like to see more intelligence built in.  Nothing's worse than getting a mailbox full of alerts that nobody looks at.

I am concerned about the size of the company, and lack of documentation about how their service is protected but they are working on this and more than willing to talk to potential customers like they did with me about anything you may have in this area.  They face stiff headwinds from competition and being a private company means I have no access to financials so I’m concerned about future viability.  CloudPassage did mention that they have customers in the fortune 500.

Because the HALO platform brings together Windows and Linux systems, the price, and the raw potential, there certainly seems to be some value here.  I would recommend implementing this on a cloud development environment and look at the fit to the total cloud system design.

Looking at the product, it solves one of the most important pieces of configuration management and that’s consistent security configuration for all servers.  They are poised to extend this capability into patch management and even ensuring application config files are consistent or at a minimum, tell administrators when they aren’t.  

I look forward to seeing where CloudPassage goes and will update this review as they continue to evolve.

Security In The Cloud is gaining More Focus

In my opinion, security is the number one topic that comes up in any discussion that involves moving to the cloud so it should be the #1 focus area for security vendors and hosting providers.  As I was researching for this article, I found that Amazon updated their AWS homepage, and now include a link to a set of pages focused on AWS security.  These pages show their audit certifications, physical security and privacy.  I found it interesting, Rackspace, Microsoft Azure, Google App Engine, and Flexiscale  did not.  Providers like AWS and HP know this.  In fact, HP uses “security” as a differentiator when discussing their offering and has the word “secure” on their cloud homepage six times (when it loads). CloudPassage is sitting in this sweet spot of security needs of many cloud hosting providers.

I expect to see other competitors emerge in the cloud security market and hosting providers starting to focus more on explaining how they secure their customers cloud infrastructure, and use those features as a differentiator.  As the public cloud hosting market matures, I expect the hosting providers to buy or create their own security suites and integrate them into their standard offerings and consoles.

In general, when I look at the number of products and services that are offered in the cloud security market, two things stand out.  First, security risk as a barrier to moving to the cloud may be coming down.  Second, no matter if you are deploying a public facing application in your own data center or a public cloud (using IaaS) you need someone on your team that really understand security or can come up to speed fast.  Without a good understanding of security it doesn’t make any difference if you deploy internally or on a public cloud.  Any cloud deployment has to consider all of the security requirements and tactics to be used before you start building out your cloud deployment.

References & Related

• Challenges & Risks of Implementing Cloud Computing  (Cloudrant)
• Cloud Servers: New Risk Considerations  (Cloudrant)
• Comparing Local to Cloud Security (Cloudrant)
• Evaluating Cloud Software as a Service Vendors - Cloudrant
• Do Govt. Laws present undue Risk to Cloud Computing? - Cloudrant
• Nuking Linux Servers, one script at a time.  By Paul Venezia (InfoWorld)
• Evaluating Cloud Security (LinkedIn Group Discussion)
• ArcSight Security

Full Disclosure:  I previously worked for HP but I don’t know anything about their cloud offering or security products outside of what I can find on their public facing websites.

Updates:
9/17 - Per comments from Cloudpassage's Rebecca Scanlan, clarified that the ghostports feature only comes with Netsec and Pro packages, it's not free.  Two-factor authentication will be in an upcoming release, not necessarily the next release.

-- Chris Claborne