Deploying enterprise cloud apps to public or remote cloud infrastructure provides a lot of useful benefits, but can be problematic for many organizations due to concerns over security and control. Once you are ready for the move from your warm and windowless data center to someone else’s warm and windowless data center, here are 5 keys to being successful.

1) Protect your data (part 1)

Cloud security is the number one issue with using public clouds. Many organizations are building applications that include personal information, confidential information, or intellectual property that needs to be protected. Help all the paranoids sleep better by building in security features from the start, even going to the point of overkill if necessary. Database encryption is a common way to protect data at rest, but also consider treating your cloud instances as single use servers.  Remove all the keys and passwords from disk once your application is running and disable all unnecessary connections including ssh. Adopt a devops strategy that allows quick launches and centralized logging and management. If a server does stop responding or gets rebooted for some reason, just throw it away and get a new one.  Taking a few extreme security measures will help you get more opportunities to utilize the true power of the cloud.

2)  Protect your data (part 2)

Cloud technology does pose some new challenges. At this point, server failure rates can vary and the faster options include storage that does not persist across restarts or failures. Data loss is a real risk unless a high availability or replication strategy is adopted. There are also plenty of cloud based data solutions popping up, including Cloudant, Amazon RDS, and Rackspace MySQL Cloud, that can put the HA burden on someone else’s shoulders. Bottom line, losing data will kill a project fast.  Keep it safe.

3) Architect for failure (and recovery)

As I mentioned before, there can be more variables affecting the reliability of cloud servers. Not only can servers fail, but the underlying hardware can have issues, and whole cloud regions can be affected by outages. This is compounded by the fact that you are no longer in direct control of when the hardware may be restarted. Several times recently we have received notices that hardware our servers were running on were going to be rebooted. If you are using some of the cloud security measures mentioned above that make your application not able to restart, these outages may require some intervention to keep your cloud application running as smooth as possible.

Creating an architecture for these realities of cloud solutions doesn’t have to be overwhelming. The important part is to consider the effects of failures on your users and decide what your tolerance for failure is. That tolerance can be a guide to the technologies to put in place to handle the failure scenarios. Perhaps you are fine with periodically restarting the application or going to a hot spare, or maybe your application must be up at all times and you need to consider deploying across regions or even across clouds to ensure the greatest resiliency possible.

4) Work in the cloud

Just as there can be a big difference between a developer’s machine and a traditional production environment, there are differences with cloud environments as well. It’s important to get a feel for the specifics of how cloud servers will affect your application by doing significant work and testing in the same cloud environments you expect to deploy to. By having your developers working with the cloud on a daily basis, you’ll start to develop an understanding of where the bottlenecks will occur and how they will affect your application.

5) Keep an eye on costs

One way to really put a damper on your cloud party is to surprise your boss with an unexpected cloud bill for much more than you anticipated.  With automation, functional testing, and continuous integration always validating your code, it can be very difficult to track the number of instances used (of course you are doing at least some testing against production-like environments, right?) Cents per hour may seem very cheap, but it’s easy to implement a testing strategy that can use tens or hundreds of servers per run, depending on the project (note that cloud instance billing rounds up: 1 minute = 1 hour). Without some form of visualization of the costs, nickels and dimes can add up unexpectedly. Fortunately, awareness can go a long way to alleviate some of the hidden costs. Updating the testing strategies to reuse servers when possible or considering utilizing an internal development cloud for most test runs can prevent cloud costs from escalating out of control.

Of course, these are not the only things that will make you successful in the cloud, but these are some of the things we experienced as we started building the CloudSwing platform. What other key issues have you experienced when building your cloud based enterprise applications?

Subscribe to The Enterprise Open Source Blog via email

Follow @openlogic
Follow @cloudswing
Follow @esweid01

This work is licensed under a Creative Commons Attribution 3.0 Unported License .

Cloud technology is the next step in the evolution of predictable software engineering processes. With the cloud, servers become instant-on commodities. They are built the same way every time, providing consistency from the ground up. Servers instances are cheap and easy allowing each component of an application to be isolated to minimize conflicts. Instances are disposable, so there is no longer a need to worry about server drift. Just get a new one every time.

We can follow several recent advancements in software engineering processes that have been building on each other to see how this all fits together:

• Open source software provides quality, freely available building blocks to build applications around.
• Agile processes turn features into bite size stories that can deliver value rapidly.
• Repeatable builds, functional tests, and continuous integration turn quality control into a predictable, automatable procedure.
• Virtual machines give the ability to better utilize hardware and provide better isolation.
• Devops engages developers in how to build for a production environment and test that environment early and often.

Combining all of these things together can help create a streamlined software delivery process that gives confidence to release quickly.

Successfully utilizing cloud technologies definitely takes a shift in mindset. Engineers tend to be control freaks. I used to be the one cherishing every new piece of hardware. Carefully selecting every component that went into the box. Coddling those servers and tweaking every last setting to get it absolutely perfect. Shuddering at the thought of anyone else putting their hands on them and knowing each touch point meant an opportunity for the server to deviate from the plan. I’ve also looked at cloud options in the past thinking “That’s not the OS I’d use” or “that version is not new enough”.

But I’ve also been the one cursing dependency issues when running multiple apps on a single server at the same time. (What do you mean I can’t have glibc 2.4 and 2.5 at the same time. I need both!) Ferreting out slight version differences in libraries between (Why does this work on my box but not yours?). Managing IPs, ports, and reverse proxy configurations to get the most out of each box.

Creating a cloud based development process, either for a public cloud (like Amazon or Rackspace) or for an internal enterprise cloud, has been like a breath of fresh air. I quickly get as many environments as I need, I get a fresh server exactly how I want it, every time.

Like clockwork. Predictable.

This is a central concept to what we are trying to accomplish with CloudSwing, Creating a system that enables users to deploy to the cloud with predictable, repeatable results that can help people realize the potential of the cloud.

Subscribe to The Enterprise Open Source Blog via email

Follow @openlogic
View Eric Weidner’s profile

This work is licensed under a Creative Commons Attribution 3.0 Unported License
.

I burned an Ubuntu Hardy live cd with Brasero (the GNOME default burning utility) the other day and it had problems booting.  Kept getting some error about SQUASHFS over and over again and never go to the login screen.

After seeing some hints about a bad burn and some unconfirmed discussion about TAO versus DAO, I burned a new copy with K3B (the software I normally use).  This disc worked perfectly.  Not sure what the difference may be in the default burn settings, but I now have very low confidence in Brasero.

Anyone aware of issues in Brasero?

Anyone looking to relieve the pain of no non-Windows support from their hardware VPN vendor or that has issues wading through the IPSec options to get vpn working from their Linux or Mac machines should take a look at OpenVPN.  This firewall project is based on the SSL standard that has been battle tested by all the ecommerce applications out there and is fairly easy to set up.  I had a test server up and running in about 30 minutes on Ubuntu and that included reading a bunch of documentation before actually doing the install.  Ubuntu repositories have a recent version that is a good place to start.  Here's a decent tutorial to follow.

Last year, there was a bit of discussion about how the launch of Vista was going to open a lot of doors for Linux on the desktop.  I’m not going to go into a discussion about the pros and cons of Windows Vista and why the door is open to alternatives, but I have been very interested in seeing how this would turn out.

Linux has been making great strides on the desktop for quite a while.  I remember that the first time I thought that Linux was really getting there was in 2002 with SUSE 8.  In the last 12 months, Ubuntu has finally made Linux on (my) laptop feasible by finally getting over the last hurdles that had been plaguing me for a long time (projector support with nvidia-settings, and easy WPA Wi-Fi support from the network manager).  My wife recently switched to Linux (after her Windows install corrupted) and her first comment was “This isn’t as bad as I thought it would be”.  Linux still has some very frustrating issues: Multiple monitor support for all video cards (without knowing how to edit your xorg.conf file and without restarting X), VPN configuration (check out OpenVPN), and the number one issue for corporate desktop use, Microsoft Exchange integration (specifically full group calendaring support).

Mac has long been known for creating a nice desktop experience, but has often been left out of the corporate world for business interop issues.  IBM apparently has a pilot to test switching to Mac.  New statistics also show that Apple is picking up a lot of steam in the US.  At OpenLogic, we have a pretty typical split.  The vast majority of non-engineering users are on Windows.  The vast majority of engineering users are on Linux (typically Ubuntu is the Linux of choice on the desktop).  What both sides are starting to agree on is that Mac’s are an attractive option and we’ve started refreshing older machines with Mac’s.  Engineering will not replace all Linux machines, but Macs will have an increased presence in our development environments, particularly for laptops.

I recently participated in a survey about Windows Vista adoption and results indicated that more companies are planning or considering switching from desktop Windows to an alternative and Apple was the top choice for switchers with nearly 1/3 considering Mac.  But if you look a bit deeper and combine all the Linux distro’s together, 2/3 of switchers are looking at some distribution of Linux.  The report did not take into account server vs. desktop use though and that is an important distinction here.

Linux really needs to keep making strides at creating an environment that I can hand to my grandparents and feel comfortable that they will be up and running without constant support from a “techie”.  This is the model that Apple has been so successful with.  A coordinated push with a big supplier such as Dell would go a long way.  Otherwise, Linux may miss this boat to the desktop and have to wait for the next one.

 

(OpenLogic Engineer) Brad's 9 yo daughter uses a program called Hackety Hack to learn and write Ruby. She wrote a program to download an mp3 and play it the other day. When she asked Brad what he was working on, he replied "basically the same thing". 

Cheesy movies are so quotable. The "landmark" news that Microsoft and Novell are going to be collaborating reminds me of a scene from Ghostbusters.

Dr. Peter Venkman: This city is headed for a disaster of biblical proportions.
Mayor: What do you mean, "biblical"?
Dr Ray Stantz: What he means is Old Testament, Mr. Mayor, real wrath-of-God type stuff.
Dr. Peter Venkman: Exactly.
Dr Ray Stantz: Fire and brimstone coming down from the skies. Rivers and seas boiling.
Dr. Egon Spengler: Forty years of darkness. Earthquakes, volcanoes…
Winston Zeddemore: The dead rising from the grave.
Dr. Peter Venkman: Human sacrifice, dogs and cats living together – mass hysteria.

After the initial shock wore off and I started thinking about what this announcement really meant, one thing really came to the forefront. Virtualization. There are a couple of things that make me think that virtualization is a driving force here.

1) Many customers and companies I talk to, install at, etc. are diving deep into virtualization technologies. Most of them are using VMware (use it, love it, wish the support pages didn't give me the run around) but I know Novell has been putting a lot of weight behind Xen (haven't used it, want to).

2) For many years I've seen large companies reluctancant to deploy Windows servers in their data centers. I'm sure info from Microsoft would state the contrary and I have no stats to back that up. I just have my experiences with both ISV's and large companies. One ISV I was working with was switching their entire product from .Net to Java specifically because their target audience wouldn't use a Windows server in their production environment.

Given these two observations, I think Microsoft is evaluating all Virtualization offerings and figuring out how to make sure that they maximize their potential, especially if companies are more inclined to run their virtualization hosts on *nix OS's (VMware Infrastructure 3 is Linux based). If Windows works great with Xen, then there is still a chance that they can keep virtualized user desktops running MS software. A virtualized Windows 2003 server is probably easier to manage as well (if it dies, bring up the snapshot) so they may be able to open up new opportunities there as well.

From Novell's perspective, they need ways to catch up to VMware and hopping in bed with the juggernaut that is Microsoft gives them extra clout in pursuing their own objectives. I think the legal benefits that are a part of the agreement are really just a nice sales/marketing tool given the current legal landscape. Stormy has a very insightful post on that topic.

Only time will tell what really comes from this relationship. Now if only we could get more native Linux support for PC gaming.

Travel tip: If you find yourself checking into a hotel at 1 am (after being up for 23 hours) and the clerk offers you a lesser room because the one you booked is near an elevator, take the other room. Near an elevator doesn't mean there may be people walking outside your room. It really means that they located the elevator motor in your room and it will run intermittently all night long.

You've probably seen the big news in the Open Source world already. Oracle is hijacking Red Hat's Linux distribution and will ship a version named Unbreakable Linux, undercutting Red Hat's support pricing. Thanks for doing all the work Red Hat, we'll take it from here. To me, it appears that Oracle cares nothing about the open source community. The open source community thus far has thrived using a culture of collaboration and friendly competition (competition of alternative options). Oracle is getting into open source with the mentality of cut-throat competition. They previously bought the company behind the transaction engine that MySQL used, forcing many to reevaluate if they could rely on MySQL features owned by another proprietary database company. And the problem I see is not that they are going to offer a competitive Linux distribution at a lower price point. It's that they don't even have the decency to build their own competing distribution like Novell/SUSE.

I've never really been a big fan of Oracle software myself, mostly because of the big monolithic lump that they ship. It's not exactly a light solution. I've also always found their claims of having the most used application server a bit dubious considering they seem to count every cd they ship whether someone uses the app server or not.

I'm sure they'll get some traction in the corporate world. There is still a great divide between the corporate open source circles and the technical open source community. Companies will be willing to lump their Linux support in with that giant Oracle contract that they already have. It will be interesting to see how the technical community reacts to this news though. I don't think the community traditionally supports this kind of behavior. I think we'd all be better off if we could foster more cooperation between corporate and community groups.

One thing I find very ironic is that Red Hat just purchased the JBoss Group, which made waves by proclaiming they owned open source by buying up projects such as Hibernate and listing (Apache) Tomcat as one of their projects on their website. I've heard that Oracle has been making similar proclamations that they wanted to own open source as well. No wonder they were trying to buy JBoss before.

Perhaps some good will come of this. There are many companies that are dumping their products into the OSS space. Some are trying to give new life to an uncompetitive product. Others are "donating" code they don't want to maintain into groups such as the Eclipse Foundation to try to get publicity. Then there are companies open sourcing projects that truly add value to the whole community. This creates so much noise out there that you sometimes have to look deep to find the gems that are available. Hopefully this event will make other companies think long and hard about why they are considering open sourcing their projects so that both the company and the community benefit from the contributions.

And in the end, a little disruption to force companies to improve can be good thing. Just watch out for those sharks in the pool.

BTW, CentOS has been producing a non-Red Hat branded version of RHEL for a long time making the core features of RHEL available to everyone (This is great for product testing). I wonder if Oracle has poached some of the CentOS developers or if they are launching this initiative with an internal team.

Interesting developments keep happening in the Open Source world. "Oracle's Open-Source Shopping Spree" says that Oracle is negotiating to purchase the JBoss Group (JBoss Application Server), Zend (PHP), and Sleepy Cat (database). The acquisition is already announced on the Sleepy Cat page.

This follows news that Oracle purchased InnoDB, the company behind a key piece of technology for the MySQL database (InnoDB technology adds transaction support to MySQL, a feature that my friends that know more than I do about databases say is what elevates MySQL above toy status). Could this be Oracle's way of forcing MySQL's technology hand? Realistically, MySQL should now focus on adding the InnoDB functionality natively.

2 database technologies, both from the C world, a Java-based middleware suite, and a non-Java web framework technology. Somewhat strange bedfellows. Both PHP and JBoss can be made to work with the "C world" databases so that's not a real stretch (other, but the PHP world and Java aren't common bedfellows. There are attempts going on to bring them closer together, but as far as I can tell, there isn't anything exciting happening in that realm just yet. Maybe they picked based on name recognition in those 3 different areas. I guess we'll wait and see how Oracle plans to tie all of this together or if they are just spreading the wealth.

Speaking of strange bedfellows, this also presents some strange business relationships as well. Novell has picked JBoss as a core component of their stack support in SUSE Linux (Red Hat went with Jonas. Even more interesting is the recent partnership announcement between JBoss and Microsoft.

How will the community handle big companies gobbling up smaller OSS project parents? With JBoss havin has kept a stable of the most important developers so losing some external community support is probably not a big deal. I'm sure Oracle can throw a few hundred people on it anyway. Will communities rebel against the leadership (e.g. Mambo)? Will we start seeing project forks?

What remains to be seen is what would happen when Oracle's priorities shift. Would they drop providing resources to these projects? Would they then whither and die or have a new champion take up the cause?

The important moral of this story is… OSS is a major factor now and isn't going away. The big companies are starting to pay attention. If you can't beat them, join them.

UPDATE: This News.com article says Oracle tried to buy MySQL but Marten Mickos was having none of it. Very interesting. If you can't beat them, buy them?