There are now so many options when deploying to the cloud, enterprises are being faced with a Tyranny of Choice.  

They can get an IaaS (Infrastructure-as-a-Service) solution from:

• Amazon
• GoGrid
• Joyent
• Rackspace
• Savvis
• Slicehost
• SoftLayer
• Terremark
• VMware vCloud
• many others

Or they can roll their own IaaS by starting with:

• Cloud.com
• Eucalyptus
• OpenStack
• VMware
• many others

If they require services over and above basic IaaS, they can get a PaaS (Platform-as-a-Service) solution from:

• Amazon
• Azure
• EngineYard
• Google App Engine
• GridGain
• Heroku
• VMware/SalesForce
• many others 

And now Red Hat with their JBoss-based PaaS solution for cloud computing.  Underlying this solution is an Apache-licensed open source package called Deltacloud, which intends to abstract cloud provider API's to increase solution portability.  (The cloud is built on open source, you know.)

But Deltacloud is not the only cross-cloud enabler out there.  Not by a long shot.  There's also:

• fog
• jclouds
• libcloud
• libvirt
• lots more

That's a lot of options to get the basics up and running.  What if I want monitoring, security, disaster recovering, or other sprinkles on top?  Don't worry, there are lots of choices:

• AppDynamics – cloud monitoring console, performance stats
• BitNami – software packages, multi-cloud deployment
• Cloudkick – tagging, monitoring, multiple clouds, web terminal
• Cloudscaling – services around building private clouds
• CloudStatus – tracks Amazon EC2 and Google App Engine for outages, performance
• CloudSwitch – migrating enterprise apps to the cloud, VMware-based
• enStratus – management, governance
• InMage – cloud-based disaster recovery (DR) solutions
• New Relic – cloud monitoring console, performance stats 
• RightScale – auto-scale, management, deployment
• rPath – packaging, deployment, updates
• Scalr – auto-scale Amazon EC2, database backup & replication, monitoring, stats
• Standing Cloud – software packages, multi-cloud deployment
• Ylastic – Amazon and Eucalyptus monitoring

Wow.  With all those implementations, it would be great if there were industry standards to rely on so interoperability and portability were realistic goals.  Or maybe at least some cross-vendor groups focused on working together, even though some people think it's way too early for cloud standards.  No problem:

• Cloud Commons – Trip Advisor for the cloud
• Open Cloud Manifesto – let's all play nice
• Open Cloud Consortium – interoperability, testing
• Open Cloud Initiative – OCP (Open Cloud Principles) – definitions
• Open Cloud Computing Interface Working Group – IaaS interface API's

Ouch.

There are now so many choices in all these areas that paralysis may set in.  Inevitably, consolidation and fall out will take place over the next few years, but anybody who wants to jump in today will need to navigate some fairly murky (cloudy?) waters.

Paula Bach over on Port 25 talked about how to go hybrid yesterday, but she's not talking about cars.

She's talking about how proprietary companies are borrowing techniques from the open source community and how certain successful communities are going commercial.

For me, the interesting part is that she specifically calls out usability as an area where open source is typically lacking.  I couldn't agree more. 

I think one of the key reasons people stay with proprietary software vendors is that they put a lot of work into the user experience whereas it's usually an afterthought (or worse) for the average open source project.

Apple's success in designing and selling beautiful hardware and software clearly shows that people are willing to pay a premium for an enjoyable experience.  That enjoyment comes in no small part from the efforts of talented user experience experts.  These people focus solely on the user and how he or she will perceive the feel of a product, the consistency across features, the use of empty space, the judicious application of color, the use of intelligent defaults, and the like.

At OpenLogic, user experience professionals help design new features for our OLEX site to make sure we convey a lot of information in a manner that doesn't overwhelm our users.  We also continue to hone our layout, style, and content over time to incorporate end user feedback. 

All this takes work takes time, effort, and a skill set that's not found in the average programmer.  I think this goes a long way to explain why open source projects have trouble in this area.

Perhaps up-and-coming user experience professionals will realize that, just like programmers, doing a good job on a very public open source project is a phenomenal way to show off their talents to the world.  It's a great way to get real-world experience and an incredible resume booster.

So, come on all you creative designers and usability people, pick an open source project and show us your stuff!

BTW, I'm not surprised to see Microsoft talking about usability in open source.  It's what I expected from them as it plays into their strengths.

As Abhijit Nadgouda says in "Benefits of Online Repositories", it only takes a simple command or two in Linux to download and install or upgrade a package.  If necessary, even dependencies will be downloaded and installed and you don't have to know anything about these new packages to make it all work in a matter of seconds.  And you don't have to use a browser or any kind of GUI to make this all happen.

Isn't this all just a little too easy? 

Well, it's not if you're a developer trying to get a job done quickly or you're working at home and don't really think about licensing terms or other obligations. 

But what if you're working in an enterprise or at an ISV where it really does matter what the license is for not just the top level project but all dependencies and their dependencies and so forth?  Are you violating any of your company's policies around software acquisition, in particular the policies related to open source licensing and distribution?  And what if one of those packages in the tree has an obligation requirement that your company can't accept or is not willing to meet?  How do you go about getting production support and/or indemnification for all the packages in your new hierarchy?

I think it's pretty easy to see that in certain settings there can be real issues related to the ease of software acquisition in our world of open source ubiquity.  This is a problem OpenLogic helps to solve through our library of over 400 certified packages backed by indemnification and SLA support.  We don't want to take away the ability for a developer to quickly find and install open source packages, but we do want to provide a thin layer of enterprise control and management around the process.  We want to make it easy for developers to research and find open source that not only does what they want, but is also compliant with existing policies so they don't have to waste time ripping and replacing "illegal" components later.

So although it's great that the logistics of acquiring open source packages has gotten far easier in recent years, it's important for enterprise decision makers to realize that too easy can lead to even more headaches than a slightly slower process that enforces reasonable constraints.

This year at JavaOne was the first time since 1999 that I actually got to attend a bunch of sessions, thanks to not having to man an OpenLogic booth.  Overall, it was quite good.  There were quite a few parallel tracks and always something interesting in at least one of them.

Perhaps unsurprising, but the major themes were:

• SOA
• AJAX
• Dueling component frameworks (JBI, SCA, OSGi)
• Scripting languages for the JVM (JRuby, Groovy, Scala)

SOA 

I expected to see SOA take off this year, but not quite to the extent I found at the show.  It seems that 2008 is truly the year that SOA goes from paper to reality.  There were a number of real users presenting case studies around their migration to SOA, performance implications, and lessons learned.  There was a separate sub-pavilion of vendors, both open source and proprietary, showing off their SOA-related businesses, tools, and services.  I think it's finally getting real.

AJAX

The growth of AJAX (aka Ajax) continues to surprise me.  The number of AJAX-related open source projects seems to be increasing at a faster and faster pace lately.  Along these lines, I saw sessions pitching Javascript for both client and server, sites that allow pseudo-technical users to create their own mashups visually, a number of widget libraries that sit on common toolkits such as Prototype and Script.aculo.us, and sessions on improving the user experience.  I also attended a few sessions on advanced web security that should make AJAX implementation a lot more interesting for the average developer. 

One related presentation recommended that developers turn their application into a platform so ordinary users can create their own content and even deploy mini-applications a la the Facebook Platform.  This clearly doesn't fit every application, but it certainly helps increase your Network Effect when it does.

Dueling Component Frameworks

In the "too many ways to do this" department, I saw sessions on JBI, SCA, and OSGi.  Yes, they're all about integration, but they each have their own niche.  JBI seems to be about creating business components that can be integrated across vendors, SCA seems more about integrating your own components within an application, and OSGi seems to be about integrating components from multiple vendors in your application.  If that isn't crystal clear, it's not you.

In the OSGi arena, SpringSource has recently announced their Application Platform which is built on OSGi.  This has rightly set off a firestorm of responses as it means the company is placing a big bet against the traditional application servers.  It could win big, lose big, or simply be ignored.  Only time will tell.

Scripting Languages for the JVM

I've spoken on Groovy for years and on JRuby for a while now, but this year at JavaOne they both had a much larger following than I expected.  Each of them has improved so much in the last 12 months that it's truly hard to imagine how they did it.  Their feature sets, stability, and performance have gone from "interesting – keep an eye on it" to "wow, this is the real deal".  We now have JRuby on Rails versus Groovy on Grails, both of which are fast and easy to use.  Just like Java with its incredible JVM, your JRuby/Rails and Groovy/Grails code will continue to get faster, more scalable, and use less memory over time without any code changes on your part.  I can't wait to see how far they'll go by next year!

Parting Thoughts 

One last comment on the show.  It seems to me that Sun is really starting to get serious about making itself known in the open source space lately.  Of course, there's the MySQL acquisition and OpenSolaris, but there's also the fact that they have an open source tool that competes with VMware (VirtualBox).  They're also going to release a site soon that provides open source software project hosting similar to SourceForge.net or java.net.  Their IDE, NetBeans, is now arguably the best platform for development Ruby on Rails apps (or at least JRuby on Rails apps).  They've released a GUI that makes it easy to visualize MySQL performance on OpenSolaris through DTrace (there's also a NetBeans plug-in for DTrace).  The list goes on.

I think they've realized that they've never really been able to capitalize on software, so why not open source it and gain some attention via the halo effect?  They seem to be doing a good job of that lately, but we'll have to give the MySQL move some time before making the final judgment.

Microsoft just held their 2008 "MVP Global Summit" in Redmond last week.  This is an internal conference where they recognize awardees in a number of divisions and have over 400 technical sessions on a variety of topics.

The interesting bit for me is that Jamie Cannon reports that there's an informal meeting of Microsoft Rubyists going on at the event.  They're planning to discuss IronRuby, open source, and other topics.

Wow, Ruby is definitely getting some love lately.  I'm excited about that because OpenLogic has been a heavy user of Ruby, Rails, JRuby, and related projects for almost a year now.  The more people use it, the better it gets.  Once an open source project hits critical mass, it really takes off.

Speaking of which, I think JRuby will start to hit its stride this summer if they really do concentrate on Rails performance next as Ola Bini suggests.  I'm very much looking forward to having the option to deploy our OpenLogic Exchange (OLEX) on JRuby.  Mongrel is great, don't get me wrong, but I've always been a little suspicious of server processes that need to be monitored and restarted so frequently.  Call me crazy.

As we're on the topic of alternative Ruby implementations, I'm anxiously awaiting the first releases of IronRuby.  Will it really be the JRuby of the .NET world?  Will it be fast, reliable, scalable, and fully-compliant?  I sure hope so.  It would be great to have yet another deployment option even though I'd like to standardize on a JVM if I have a choice.

With Microsoft's growing interest in the world of open source and some internal Ruby programmers, I tend to think they'll get there.  Perhaps JRuby will have some real competition on Windows next year.

Abhijit Nadgouda says it's been a decade since the term open source was chosen to represent the concept.  This is true, but the concept has been around far longer.  I remember reading through freely available source code to BBS systems back in the mid-late 80's to discover their secrets.  Long before that, software was freely given away with source code by hobbyists.

With such a long and rich history, it seems somewhat strange that "open source" is just now enjoying its 15 minutes of fame.  Sure, it has forever changed the way software is created and consumed, but why now all of a sudden?

I believe it's just the right time and place to happen.  The ubiquitous tools of the Information Age finally make it trivial for large numbers of diverse and geographically separated people to collaborate towards a shared goal about which they are passionate.  In the olden days, it was a real pain to dial-up point-to-point with each computer you wanted to explore.  It took a lot of manual labor just to find interesting computers, let alone exchange information with them.  It was also tedious to hold a simple on-line conversation with somebody at 110 baud where you could see the characters being displayed one or two at a time.  The concept of open source still flourished, but there was just too much overhead involved to break into the mainstream.

Nowadays, with fast Internet connections, great search engines, and excellent collaboration tools (written by the open source community), it's almost effortless to find open source software, join a community, contribute something back, or even create your own project. 

In fact, it's almost too easy to create your own project.  It's so easy that there are approximately 200,000 open source projects in the wild today.  Most of them are defunct, abandoned, or just plain not very good.

In the olden days, it took so much effort to go through the mechanics of setting up and maintaining a project that you had to have something really important to say (in the form of code) before you'd spend the energy.  This meant that there were far fewer projects, but they were of generally higher quality, on average, than we see today.

In other words, the downside of open source is that anybody can create a project and a pretty web site in a matter of hours.  The hard part is coming up with something truly useful that will be well-managed, well-maintained, and will foster a dynamic and active community over time.

To stretch an old saying perhaps a bit too much, open source projects are like opinions, everybody's got one (and most of them stink?).  There's no "editor" in the world of open source, which is part of the reason I created OpenLogic.  We filter down those 200,000 open source projects into a more manageable list of a few hundred that make sense for large enterprises. 

So, everybody has an open source project and everybody has a blog.  I can only hope this one makes it into the "truly useful" category some day.

Over at Microsoft, they've just celebrated their first Open Source Day.

Wow.

I think Jamie Cannon was right when he (she?) joked that "Hell has frozen over".

I know Microsoft is full of smart, hard working people, but this is still a significant milestone for them, and one that occurred before it's too late for them to change.

All along, I've been expecting the standard response and outcome: first they ignore you, then they ridicule you, then they fight you, then you win.  It looks like we're on the right track.  However, we've all seen Microsoft's strategy of "Embrace, Extend, Extinguish" that only begins in the "then you win" phase.  That means the open source world still needs to be cautious, but Open Source will be a tougher nut to crack than office suites, browsers, and programming languages.  After all, it's pretty hard to extinguish open source, isn't it?

Well, if you were interested in embracing, extending, and extinguishing individual open source projects, there are two simple strategies:

1. Buy the owners and/or committers.  Hire them, contract them, or whatever else it takes to get their attention.  Then slowly change the roadmap over time, morph the project into something else, ignore the core user base, and gently drift into obscurity.
2. Same as above, but dedicate your own people to the project and do it from the inside out.  This will be tricky, especially once the community at large catches onto the theme.

Neither of these options is easy and it will take much skill to avoid being totally transparent, but they seem doable.  Given a large enough bank account, option 1 seems particularly appealing when applied as a surgical strike to key trouble-making projects.

So, could anybody really kill "Open Source"?  Not likely.  It does, however, seem possible that with enough determination and resources that an entity could dismantle a few handfuls of key projects that would be dramatic enough in the aggregate to undermine confidence in the "system".  But after the headlines are made and the dust settles, I think new project owners, committers, and contributors would then appear and fork the original projects under new names and "Open Source" would resume it's regularly scheduled releases.

Okay, so what does Open Source Day at Microsoft mean then?  I think it means that Microsoft has begun the Embrace process.  Extend will certainly come next, but perhaps Extinguish is off the plate this time.  I think it means we're about to see yet another explosion in the growth of Open Source, the number of enterprise-ready projects, and the diversity of platforms and languages.  We're going to have more competition among projects across the board which will force vast improvements in the areas of user interface and aesthetics (think "Extend").

It's Open Source Day everywhere.

I've been a big proponent of Agile Development for quite a while now, mainly since I read all about eXtreme Programming in 2000-2001.  I always liked the idea of pushing those tenets in software development, but there were always some sticking points.  Mind you, they're not exactly the same list Abhijit Nadgouda mentions in his recent post on Extreme Programming Getting Stagnant, but they're similar.

In particular, it's always been hard to convince developers to try Pair Programming.  It might actually be easier to convince managers than it is to convince developers.  They really like autonomy, creativity, and freedom.  The thought of sharing a keyboard all day just rubs them the wrong way, even though they're extremely collaborative for brief periods of time when working through a tough problem.  Maybe it's just the expectation that they should pair that causes the problem and not the actual pairing itself.

In any case, at OpenLogic we use an Agile method based on SCRUM, which, as our product manager points out, is quite nice.  Contrary to Abhijit, I think it's easier to estimate when you're doing it in small chunks, like as in a 2 week iteration.  It does take some practice, but we've gotten quite good at estimating and hitting those estimates, thanks to the feedback cycles built into SCRUM.

As with any Agile development method, SCRUM is meant to tailored to your organization.  The key is to try it "out of the box" then make changes that fit your company, team, and environment.  Expect to keep changing it over time as team members grow, move on to other projects, you create new products, etc. 

In other words, the software development method itself needs to be just as agile as the software you're building.

Karri Dunn recently announced DreamSpark on Port25.  DreamSpark is a program that provides free Microsoft development tools to students for non-commercial purposes.

As Microsoft has wisely done for a very long time, it wants to make sure that college students who will be writing the next generation of software are used to Microsoft tools, API's, platforms, and plans.  This makes the Microsoft route the path of least resistance for all those new graduates when they enter the workforce.  As they need more software over time, it will be natural for them to upgrade their tools and expand the Microsoft footprint inside their new companies.

But I wonder… won't those new developers find out about open source development tools while still in school?  I mean, there are hundreds of enterprise-grade open source packages out there.  Won't they want the same ability to customize, improve, and integrate the Microsoft tools as developers who choose the open source route?  Won't they be at a disadvantage if they don't have those abilities?  I think it won't be long before they look around and discover another world of tools and start to vote with their feet.

Microsoft is moving in the right direction with CodePlex, their "Open Source Project Community", so I think they see where things are heading.  Sure, they would be giving up a huge revenue stream if they open source their development tools, but with the end game in mind it seems like a small gambit in order to not just retain their current developers but to attract others to their platform.  If done well, they could even attract a number of current open source devotees. 

If there's one thing Microsoft excels at, it's marketing.  They have the money, motivation, and marketing ability to make the transition successful.  The only question is will they lead the way or will they be dragged kicking and screaming by their own development community.

One well-known issue with Open Source is the so-called "Dependency Hell" problem.

Background

This situation arises when one package, let's call it "A", depends on a number of other packages, "B" and "C", that have their own dependencies in turn (B depends on D and E).  When trying to get A installed correctly, you first have to find and install all of it's dependencies.  In this case, you need to install B first.  But wait, that means you have to install D and E before B can work. 

The tree looks like this:

        A
       /  \
     B     C
    / \
  D    E

The Problem 

Here's where the real fun begins.

It turns out, according to your research, that D requires a special tool, compiler, or operating system library called "X" before it can do its thing.  And C requires another thing "Y".  That's all well and good, but lo and behold, X and Y are found to be incompatible.  For whatever technical reason, X and Y either can't be installed simultaneously, are different versions of the same tool, or simply cause another chain of problems when installed together.

This means it's time to start jumping through some hoops.  One option is that you can install X, build D, remove X, and finally put on Y to build C.  Whew, that was hard!  But wait, there's more.  What if both X and Y are required at runtime to work properly?  Now you're really in trouble.

Unfortunately, the scenario I just described is not simply fiction, it's an every day occurrence in the open source world.  Many projects, especially when they're based on the C programming language, get into these other circular dependency situations that make programmers pull their hair out and punch their neighbors.

The Solution

It might be hard to hear for some die-hard C fans, but I think the solution is either A) don't depend on shared libraries and other operating system resources, or B) use Java or another language that makes it easy to include all necessary run-time dependencies with your code.  Yes, both of these solutions can lead to dramatically increased disk space usage, but hey, when's the last time your new computer came with a 30 megabyte hard drive?  Laptops now ship with fast 160GB+ drives; it's time to move on.

How Operating System Vendors Help (Or Don't Help)

Okay, here's where I'm going with all of this.  Linux and most of the thousands of open source projects in its ecosystem are written in C.  Vast numbers of these projects suffer from the problems I've described above, so what are the Linux vendors doing about all of this?  Well, it looks like they're saying to developers and users, "Look at all these cool lego blocks we've given you! Have fun building things with them!"  In other words, they're doing nothing.  Actually, that's not quite right.  They're doing worse than nothing.  They're continuing to ship new versions of their operating systems with updates to all these shared libraries and other tools that quickly break fragile dependency trees.

But perhaps I'm not being fair to the OS distributors.  It's not like they're the ones creating the fragile dependency trees in the first place, right?  They're not the ones forcing people to use the outmoded tools and libraries they package up along with the core OS.

Hmm… maybe we're onto something here.  Why is it that the OS vendor ships this stuff in the first place?  I mean, they're just a bunch of unrelated add-on packages, applications, tools, and utilities that go uncertified, untested, and unsupported.  Why do they come with the operating system any way?  Shouldn't the operating system vendor spend all its time trying to make the operating system (and the accompanying desktop environment) as robust, stable, and security-hole-free as possible?  Or maybe they like this situation because large software vendors like Oracle can pay them to certify a particular application on each new release of the operating system?

Conclusion

I think what's needed to solve these nasty problems is for the open source community in general to move away from low-level operating system dependencies and other C-based tools that expect pre-existing shared resources in their run-time environment.  It's 2007; let's concentrate on virtual machine technology for both hardware and software purposes.  That is to say, let's continue the migration to Java, Python, Ruby, and other languages that shield us from these painful problems of yesteryear.  Let's move away from depending on operating system vendors to provide our above-the-os world.