How SaaS and Cloud Technologies like Hadoop Make Big Data, Big Processing Applications Faster and Easier

OpenLogic offers several solutions to help companies scan their software and systems to find any open source software that is being used.  At OpenLogic, we’ve proven that a (software-as-a-service) SaaS solution coupled with cloud technologies creates a faster and easier solution for our customers.

When scanning for open source software, there are two critical factors: lots of data and lots of processing requirements. The “lots of data” requirement comes from large volumes of source code, indexes and fingerprints for hundreds of thousands of open source projects. The “lots of processing” requirements comes from the need to quickly find all the potential matches where your source code includes open source code, and more importantly to narrow down those potential matches to the RIGHT match using sophisticated noise reduction techniques. 

Just to give you an idea of the technologies involved, we are using Hadoop, HBase and Solr to manage and search the big data. The analysis and processing is done in a Ruby on Rails SaaS application that is distributed with dozens and dozens of workers across an array of servers. There is also a small client-side Ruby application that does fingerprinting of local code so that your code never leaves your firewall.

We’ve seen three key benefits for customers in using these SaaS & cloud technologies:

1. Instant on, zero hardware requirements

Customers don’t require any special hardware or any special data storage. They get an “instant on” and can be scanning within seconds. This is a well-known benefit of a SaaS solution, but it is even more critical when you are dealing with applications that rely on big data.

2. Better results, better performance

There are always new and better techniques to find and pinpoint matches between your code and open source code. Because the heavy-lifting of processing those matches is done in OpenLogic’s data center, we can add new and better algorithms without slowing down customer scanning or requiring them to add new hardware.

3. Latest and greatest data, no maintenance required

When we add new fingerprints to expand our database of packages or add new ways to fingerprint code, the customer instantly gets access to this new data. They aren’t forced to update a local fingerprint repository or add hardware to accommodate the expansion.

We’ve seen how big data, big processing public search sites from Yahoo and Google can benefit from SaaS and cloud approaches. Our application, which scans for open source code, is a real-life proof point of how more specialized applications can benefit as well and deliver significant benefits to customers.

Earlier this week, the latest round of BusyBox GPL lawsuits were announced by the SFLC. The current offenders are Best Buy, JVC, Samsung, Westinghouse and 10 others. It seems that every few months, there are a new round of violations or lawsuits announced, so it behooves every company that distributes software or products containing software to ensure they aren't ever on the list. We announced our Open Source Fulfillment Center service earlier this week that helps companies ensure they comply with the GPL.

Jason Perlow of ZDnet gave a pretty good explanation of the challenges in complying with the GPL. Dana Blankenhorn, his colleague, talks about why those explanations are not an excuse. I want to focus on some concrete steps companies can take to avoid being the next defendants.

Assume that you may be using GPL code.

Realize that if you have software in your product, there is a high probability that you have open source software in it. There is also a HIGH probability you have GPL or LGPL code in your software, even if you think you don't.

At OpenLogic, we offer an Application Audit service for customers where our experts scan your source code to identify any open source and help you comply with the licenses. When we do an Application Auidt, we always find open source, we always find more open source than the company thinks they have, and we almost always find GPL code that the company is unaware of.

Scan all of your code – including code from outsourcers.

Even when companies have good source code control practices, it's almost impossible to get a 100% accurate list of the open source and licenses used. The main challenge is that open source often has other open source components that are bundled inside it, and that use different licenses than the primary license specified by the project. The only way to find a complete list of open source and the relevant licenses is to scan your code.

It's also critical to scan all code from outsourcers. At OpenLogic, we find that many cases of "hidden GPL" come from this source.

OpenLogic provides both services and products for scanning code to identify open source.

Understand how to fulfill the source code requirements of the GPL/LGPL.

Once you determine that you have GPL or LGPL code in your software, make sure you understand and follow all of the appropriate requirements. Some of those requirements may not be readily apparent, but they can include:

• Including source code with your product OR including an offer to get source code with your product
• Providing the exact "corresponding source" that goes with the binaries for each product and model
• Providing instructions on how to create binaries from the source – including scripts, information on the tools needed, and details on how to replace the standard code in your product with the modified code
• Providing the source code in electronic AND physical media
• Maintaining the source code for a period of time after the latest distribution of your product

OpenLogic provides an Open Source Fulfillment Center service that helps you comply with the GPL.

I also recommend A Practical Guide to GPL Compliance from the SFLC as a great primer for do-it-yourselfers.

We announced last week that we were offering support for CentOS — a community distribution of Linux that is created from the publicly available source code that is used to create RHEL.  CentOS strives to be binary compatible with RHEL and is built from the source code that is used to make RHEL — minus all of the Red Hat names, logos and trademarks.

Here's what's included in a CentOS support subscription from OpenLogic.

Technical Support with multiple service level options: OpenLogic provides business hour and 24×7 SLA options with guaranteed response times.

Access to Updates:  OpenLogic provides a service (called OpenUpdate) which notifies customers of new releases and patches for not only CentOS but 500 open source packages in the OpenLogic Certified Library.

Indemnification: OpenLogic offers limited indemnification to customers who buy CentOS support for IP actions against CentOS.

Companies that distribute software containing open source often struggle to comply with open source licenses.  (See latest lawsuit by Artifex against Palm).

Most companies that violate open source licenses do so because they haven't properly tracked the open source software they used or what open source licenses apply.  The use of code from outsourcers and the nesting of open source projects creates additional complexities for companies' compliance efforts.

Today at OpenLogic we announced a new source code scanning and license compliance solution to help companies accurately identify the open source in their code.  Scanning solutions to identify open source code have been around for several years.  However, as we used a variety of scanners to audit customer's applications, we found that these existing solutions have significant limitations.  We developed OSS Deep Discovery and the OLEX License Compliance module to address these issues.  The OpenLogic solutions provides:

• High accuracy, low noise:  Using a range of advanced matching and noise reduction techniques, OSS Deep Discovery provides pinpoint identification of the correct origin of open source code.  OSS Deep Discovery eliminates the extraneous matches that are common with previous solutions.
• Lightweight, instant-on architecture:  The OLEX License Compliance module is a SaaS-based solution that eliminates the need for additional hardware or infrastructure and OSS Deep Discovery is a lightweight downloadable client that enables you to start scanning your code within minutes.
• Value-priced:  OpenLogic's solution provides a low cost way to ensure open source compliance.  Subscriptions provide a low-entry price for small companies as well as a cost-effective "unlimited scanning" option for companies scanning large volumes of code.

Companies now have new options for scanning source code to find open source and ensure compliance with open source licenses.

After four years supporting a wide variety of open source projects for over 120 enterprises, OpenLogic is now expanding our offering to include commercial support for community distributions of Linux.  Today we announced that we are offering commercial support for CentOS as our first community Linux distro.

Here's a Q&A explaining the why's and how's of our announcement:

Why are you now offering support for Linux?

Customers come to OpenLogic to get one-stop support across all of the open source that they use.  Since applications using open source often run on Linux, our customers have asked us to offer support for community distros of Linux.  

Why are you starting with support for CentOS?

We have gotten requests for CentOS support from our customers.  In addition, CentOS is a great choice for enterprises looking for a lower cost option for Linux on their servers.

Why would a company get CentOS support from you versus other support vendors?

There are three main reasons customers get support from us — aggregation, quality and value. 

• OpenLogic is a one-stop shop for companies looking for open source support covering hundreds of open source projects – including many projects for which there is no other commercial support option.
• OpenLogic provides high-quality, expert support that doesn't stop at the boundary of a single project.  When issues span multiple open source projects and even involve the operating system, OpenLogic can still help when other vendors begin to point fingers.
• OpenLogic is value-priced compared with offerings from other vendors.  OpenLogic offers low priced support options for unlimited servers and unlimited incidents.

How is your CentOS support priced?

Our CentOS support can be bought in two ways — based on the number of servers or an unlimited server option.  The low priced unlimited server option can provide significant cost advantages for as few as 15 servers.

How do you handle bug fixes?

The CentOS project stays up-to-date with fixes from the upstream vendor.  OpenLogic will fix bugs and contribute fixes back to the community on behalf of its customers.

What is CentOS?

From the CentOS Wiki & website:

CentOS is an Enterprise Linux distribution based on the freely available sources from Red Hat(R) Enterprise Linux.

CentOS conforms fully with the upstream vendors redistribution policies and aims to be 100% binary compatible. (CentOS mainly changes packages to remove upstream vendor branding and artwork).  CentOS is free.

 

CentOS is a trademark of CentOS Ltd.

Red Hat is a registered trademark of Red Hat, Inc.

I know everyone's been saying it: The economic downturn will drive more people to open source. Because OpenLogic sells support on 500 projects as well as open source governance services and solutions, we get a very broad view of enterprise use of open source. After analyzing our Q3 and 2009 results, I've seen the data, and I'm here to tell you that open source is looking good.

Our press release hits some of the highlights, but I wanted to share some more details here.

Companies are choosing to pay for open source support

Our business has been growing strongly since initial funding, but the current economic downturn has caused an inflection point. In Q3 we've blown away our previous highs for number of new customers and number of transactions. In the first 3 quarters of 2009, we've seen growth in revenue from new customers up over 60% from 2008.

This growth shows that not only are more people using open source, but more people are choosing to pay for open source support or governance solutions. We tend to gain customers as open source usage becomes more accepted, broad and pervasive in an organization. Some have said that companies may choose to self-support open source as they become more familiar with it, but we've seen the opposite in our business. Our results indicate that more and more companies are interested in stepping up to pay for support when it's at the right price and delivered in the right way.

Many more people are proactively looking for open source support & governance

We generate our new customers through a combination of inbound leads (pay per click, organic search), and outbound marketing (webinars, telemarketing). The number of inbound leads is always an interesting number, since it represents prospects that come to us of their own volition and is a good indicator of how many people are looking to buy open source support & governance.

The number of inbound leads has been steadily growing, but starting in Jan 2009 we saw a jump and starting in Jun 2009 we saw an even larger increase. We are now getting 77% of our new customers from inbound leads — up significantly in both numbers and percentages from a year ago. This indicates that more companies are proactively looking for solutions unprompted by our outbound marketing efforts.

Open source is spreading beyond early-adopter industries

We've always had a lot of strength in early adopter industries — financial services and technology being the top two. In 2009, we've seen a striking increase in the breadth of industries in our customer base. We are seeing strong growth in a variety of industries, including some (retail, healthcare, manufacturing, transportation, energy) that are not always known for being on the bleeding edge. This indicates that open source usage is spreading in industries that don't operate on the leading edge of technology.

The data from our results is reinforcing the idea that open source has grown as the economy has struggled — it's use is spreading to new industries and spreading within organizations. Even more importantly for open source vendors, we have not seen a flight toward self-support, but rather an increased willingness to pay for commercial support of open source.

Join the conversation and let us know what you've been seeing in your open source business.

A recent survey we did of enterprises using open source, showed that they are not big fans of the open-core business model.  Recent blogs by Matthew Aslett, Jay Lyman, John Mark Walker center around this issue of the future of the open core business model.

At OpenLogic, we are a VC-backed company that has developed a business using a "services-based" model around open source.  Our definition of services-based matches with Matthew Aslett's.

By “services” we are not simply referring to support services, training, implementation services and consulting. The term also includes software delivered as a service, and features delivered as a managed service – such as via Red Hat Network. It includes anything you can think of, in fact, that can be considered a service, rather than a product.

I wanted to share with you a few of the lessons we've learned as we've developed this services-based model in the hopes that it may help others in that are considering a services model for their open source businesses.

1. Expand your picture of "product"

In the old, proprietary software world, your product was a piece of software. In the world of "open-core", most vendors just split the software into two pieces — the free open source part and the proprietary part.  They then sell the proprietary part just like any proprietary software.  That may be comfortable, but it can also force you to act just like a proprietary software vendor so that you can control and own the IP.

If you consider a model where the software is completely open source, you need to look beyond the software to monetize your business.  If you expand your view to the "whole product", you now have a variety of other options that customers value that you can use to monetize your business  (channels, add ons, partnerships, services, etc). At OpenLogic, we've built our business around several of these "whole product" elements:

• Certified library of open source software
• Notification and updates on open source changes
• Indemnification on open source software
• SLA Support on 500 projects
• Consulting services
• Training
• SaaS Open Source Governance platform

Our value is in being a one-stop shop with expertise across many open source projects.

Other companies have also figured out ways to monetize other "whole product" elements:

• Certification of open source with other products (RedHat, SLES)
• Hosting of open source software (SugarCRM)
• Add-on management tools (SpringSource, MuleSource)
• Commercial licenses (MySQL)
• Backport of patches (RedHat)

When you consider your business model — look broadly at your whole product to find the best way to generate revenue.

2. Learn to live without lock-in

At OpenLogic, we offer support on hundreds of open source projects — some of which have competing support vendors.  At any given point, a customer may decide that one of those vendors offers a better support offering.  When the contract comes up for renewal, if we haven't delivered value or provided outstanding support, we likely won't get a renewal.

Instead of fearing this, we embrace it.  It means that we need to stay focused every day on delivering value, offering competitive pricing, solving customer issues quickly and accurately and ensuring we meet customer needs. 

Instead of agonizing over the fact that customers could switch, we just get on with it.  Lots of other businesses across many industries are in that same situation.  If they can build successful businesses, so can we.

3. Build your business so you can price for value

Open source is typically priced lower than competing proprietary products.  However, when you build a business around completely open source software (not open-core), you likely will have vendors who compete with you on  support, services and training for the product. When you are operating in a competitive landscape, you better make sure your pricing is competitive.

To do that, you need to examine all of the costs associated with traditional, proprietary software and determine how you can lower the costs of your business.  Open community development models (as opposed to a vendor-controlled community) can save on development costs.  Open source can also reduce sales and marketing costs.  However, you need to go beyond these obvious areas to find other ways to operate differently.

At OpenLogic, we developed a unique federated support model that aligns our costs to actual customer issues.  We also have developed lead generation and sales processes that are much lower cost than expensive enterprise sales processes.  The specifics of your business will vary, but there are many opportunities to structure your business in a way that you spend less and can price for value.

4. Re-think your misconceptions about margins

Many people from the proprietary software world, including VCs, have the misconception that "services" businesses have lower margins.  The misconception comes from equating "services" with "professional services".  Once you realize that a "services" business goes above and beyond traditional professional services, the picture on margin changes drastically.

Margins on support business are very good.  Look at RedHat's margin (over 80%) as an example of this.  If you look at proprietary vendors like Oracle, their business is built around making money from their support and maintenance offerings.  

Once you let go of the idea that services always means lower margins, you'll open up many more opportunities for your business model.

I recently wrote a blog post about what what happened when Projity (backers of OpenProj) got acquired by Serena Software.

After months of trying to get attention from the OpenProj community about some missing source code (including forum posts, emails, calls to Serena) I finally resorted to public embarassment in hopes of getting a result.  This is definitely not my preferred means of getting a response, but we had a customer issue that we needed the source code for.  I blogged about it and also twittered (with an @serenasoft) in a last ditch effort to get a response.

The good news is, it worked.  The very next day, the missing source code was posted and our Expert Community member has created a patch that we are testing for the customer.

As I step back and look at this incident, I see a couple of positives and a couple of interesting things to think about.

Positives:

•  Because OpenProj was open source meant that we could resolve an issue for the customer, even when it wasn't a high priority for the OpenProj committers.
• We were able to attempt to contact the committers (even though we didn't get a response).
• Twitter in combination with a blog provided us a way to escalate the issue when we didn't get a response through traditional channels.

Thoughts:

• It's still not clear to me if Serena understands fully the implications of "owning" an open source project.  It's not just about adding code, but also about fostering the community.  I think this could be an issue for other proprietary companies that buy open source vendors as well.
• This also highlights some of the problems with a "vendor controlled community".  When the vendor is no longer focused on the open source project, it may mean that project languishes.
• By opening and developing the OpenProj community, they could build support for OpenProj and those benefits could help their "for pay business".

Although the OpenProj team has now made the missing source code available and continues to add patches,  I'm interested to see if they start participating more actively in the mailing lists and forums.  If the community as continues to languish, we will have to reconsider whether it belongs on our list of certified open source packages.

Update 10/5/09: The day after I wrote this blog and twittered it, the OpenProj committers provided the missing source code.

There has been lots of discussion about what will happen to MySQL once it is acquired by Oracle.  In a generic sense, the question really comes down to this:

What happens to an open source project, when the open source company that acquires it is bought by a proprietary software company.

I thought I would share an example, OpenProj, where an acquisition by Serena Software, a proprietary software company, has not boded well for the open source project.

OpenProj is an open source project management tool that was originally backed by a company called Projity.  Projity offered OpenProj under an open source license, and then sold an "On Demand" hosted version for a subscription fee.

About a year ago, Projity was bought by Serena Software.  Serena said that they would continue to offer OpenProj under an open source license, but would roll the On Demand functionality into another proprietary product — Serena Mariner.

So far so good.

However the committers on OpenProj seem to all be employees of Serena.  Although they continue to add patches to OpenProj, they no longer respond to any questions or issues on the forums or mailing lists.  They have posted nothing to the developer list in the last year. 

We recently were helping a customer to resolve an issue in OpenProj.  To change the code, we needed a particular file.  Unfortunately, the source code for that particular file was not available on the project site.  Only the binary for that file was available.  Source code for other files was available, but this one was missing.  We were apparently not the first ones to notice this, because someone had already posted a question to the mailing list with no response.  Our posts to find the missing source file as well as many attempts to reach out to project committers have been unsuccessful to date.

We're assuming that the lack of source code for this particular file was an oversight.  And, I'm guessing that the project committers are too busy on their Serena work to pay attention to the mailing lists.  However, by not responding to inquiries on this issue or resolving, OpenProj as it stands right now is not fully open source — since it doesn't meet the basic test of making source code available.  We're still working to try and get a response from the OpenProj committers (if anyone can help with this, let us know).

This issue raises a larger question though for open source users — what will happen to other open source projects when the sole company that backs them is bought by a proprietary company whose interests lie elsewhere?  This risk is especially high with projects that have a "closed" list of committers that work for a single vendor backing the project.  I'd be interested in hearing about other examples of similar acquisitions and what happened to the open source project.

We've had some people asking about the sample for our recent survey that asked users about open source business models and license models.  So here's some data on our sample and the respondents.

We sent the survey to about 10,000 people in our database. Some of these are customers, but most are not.  These represent companies in a variety of industries that may use open source at varying levels. We also posted the link on Twitter, but less than 10% of the response came from Twitter.

We asked respondents for their company size, industry and job function.

Company size 

• 66 % of respondents were from companies over 200 employees
• 33% of respondents were from companies under 200 employees.

Industry

Respondents came from a wide range of industries, but here are the top 6 industries.

• 25% Technology
• 10% Telecommunications
• 9%   Banking/Finance
• 8%   Consulting
• 6%   Government
• 5%   Aerospace & Defense

Job function

Respondents tend to be technical.  Here are the top job functions of respondents.

• 13% Developers
• 12% CIO/CTO
• 12% Architects
• 9%   Technical Lead
• 9%   Legal/Compliance
• 8%   IT/Development Manager

Overall we got a good sampling of different job functions and industries.  Check out the press release with survey results.