Several disparate activities have conspired to produce this post.

1.) I've been listening to Phil McKinney's Killer Innovations podcast a lot lately on my 140 mile round trip commute to OpenLogic. 2.) I've been reading the book The Evolution of Useful Things by Henry Petroski. (not while driving :-) 3.) I've been frustrated by the unnecessary complexity and bloat of numerous mainstream open source projects 4.) Commercial software is every bit as bloated except there's no opportunity to do anything about it. (While these are all related motivations for me, #3 and #4 are a topics of future posts...this post I'll devote to Petroski's book and how it's relevant to open source innovation.)

Engineer As Critic

Petroski contends it is frustration with existing products or processes, rather than necessity, that is the real mother of invention. He studies the characteristics of historic inventors and concludes it's the "engineer as critic" that drives invention. It's not a vacuum or lack of a product which can address a particular problem, but how imperfect current solutions are at satisfying the need. Petroski's example of the evolution of the zipper, from straight pins and buttons to the modern zipper in the early 1900's, is a great lesson from which open source developers can learn. From the time work began on the zipper, it took about 30 years and numerous failed companies before it was finally patented and in common use. In the meantime, people didn't walk around naked, but their garments were definitely more drafty. The draftiest of all might have been the early adopters of the zipper who found their newfangled gadget was wedged to such a degree it had to be cut out out of the fabric in order to extract the occupant from the pant or skirt. At least the occupant could get out - I can remember more than a few wedged Windows '98 users who weren't so lucky. There are huge insights for the open source community in Petroski's book. First, the impetus for most open source projects is exactly his "engineer as critic." This is a good thing - the open source community really started this way. It's axiomatic in open source culture that if you don't like something, you roll up your sleeves and fix it or create something to fix it. You literally and figuratively have the license to do that. It's this cultural DNA of open source which gives me unrestrained confidence that most of the "useful everyday things" we'll see in technology over the next 30 years will be derived in some way from open source efforts.

Open Source as Innovative Process

With that insight, it's not a stretch to see that open source is an entire system and process for innovation. Open source is more an engine for innovation than it is a specific software package or money saving device. Open source by definition is structured to incorporate the Petroski "engineer as critic" as the fundamental driver of change. As the whole process develops, the actor(s) driving the change will evolve, but for right now, engineer-as-critic is where most innovations begin because engineers are typically on the front line of doing something about it. With those glowing remarks behind us, I think there are a lot of open source projects, even some mainstream open source packages, that look a lot like the early zipper when it got wedged at inopportune moments. Without putting too fine a point on it, the open source analog can leave early adopters with their zipper down and no way to zip it up. The growing recognition of that fact has created the impetus for certification, indemnification, and emphasizes the critical need for community support around an open source project. Again, imagine if you were an early adopter of the zipper and couldn't get zipped up one night in the wash room during intermission at the opera. You get that sinking feeling that you're more than a few degrees away from choking the inventor through your LinkedIn network. You can't really roll up your sleeves and fix the problem as would be prescribed in open source culture. There's no community of zipper technicians available to help resolve your embarrassing problem. The obvious analog with commercial software is that unless you're a huge company, and possibly unless you're a government entity, you have no leverage with Oracle, Microsoft, IBM or any of the other large tech company. Even with purchased, commercial support, you could still have some embarrassing software "draft." The obvious analog in open source is if the OSS project isn't mature, or actively developed, and if there's not a vibrant, helpful community behind it, you can have the open source equivalent of commercial software "draft." In that respect there's not much difference between commercial and open source software - it still comes down to how well it's supported and how responsive the people behind it are. Despite the technical similarities between commercial software and open source software, I submit that your chances of affecting a positive outcome are greater with open source than most commercial software.

Is Source Code Access Really the Issue?

Consider these questions: (I've got to pick on open source a little) The last time your Linux box failed to behave correctly after a distro's kernel update, was your first thought: a) "I better roll up my sleeves and dust off my kernel debugger and figure this thing out...I'm going to get to the bottom of this, I have the tools and source code" b) "I'll bet I can find a Bugzilla entry somewhere and patch it up" c) none of the above Now it's Windows turn in the barrel: The last time you had a problem with your Windows operating system, was your first thought: a) "I should let Microsoft know about this, I'm sure they'll want to improve their product and my feedback will be important to them" b) "I can fix this, but I have no source code." c) none of the above No doubt the answer for 99.9% of those reading this post is c) in both cases.

The Impetus for Open Source is Changing

This leads to an inescapable conclusion that while the engineer-as-critic is the impetus for open source and technical innovation in general, it's not the end of the story. As the questions above attempt to demonstrate, whether you have source code access or not, the likelihood that Joe Average computer user will dive into the internals of any project of moderate complexity and start banging away on source code like one of the million monkeys trying to write Shakespeare, is next to zero. Even if you have the raw skills and the desire, you might not have time to learn everything that needs to be learned in order to fix it and submit the fix. Again, the value of the support and community begins to shine, doesn't it? That is the essential rub for any software product to take its next leap in subscribers. Any product, whether it's a zipper or a web application server or database, has to work well and be well supported by technically savvy people for it to be a success. There have to be other innovations in the delivery that go with the necessary, but not sufficient, Engineer-as-critic innovation system. We need Accountant-as-critic, Tech-writer-as-critic, Marketer-as-critic, Architect-as-critic, Graphic-artist-as-critic, User-experience-critic. We need the whole lot of them to take open source beyond the early adopter zipper phase it's in now. It's time to innovate in all areas, not just engineering, to accelerate the adoption rate of open source.