If you are a web application developer and have never heard of HtmlUnit , it is time to give it a look.

It is a simple API that allows you to write a Java program that can simulate a user traveling through a website. It doesn’t just read the URL into an input stream; it can parse the HTML and allows you to simulate clicking on buttons, clicking on links, firing JavaScript functions, and more.

HtmlUnit for testing

HtmlUnit was intended to be used for UI testing. For example, let’s say you host an eCommerce Java EE application. In order to perform some testing, you would want to simulate the entire life cycle of a user creating an account, verifying the account, logging in, filling up their shopping cart, and paying for the items.

Normal Java testing methods use JUnit and only test the the back-end Java code that runs on the server. This is fine and dandy, but that is not how the user uses the website. You need to use a UI test in order to truly test. While testing the back-end is extremely valuable and should never be forgotten, you are potentially not testing several key things that can only be tested using the actual browser.

For example, here are some issues that cannot be tested on the back end:

• When trying to view the shopping cart, a JavaScript Error occurred and prevented the cart from loading.

• When attempting to make a payment, the user’s session was invalidated and they were brought to the log in screen.

• The "Check Out" button was disabled when it was not supposed to be.

JUnit and HtmlUnit can be used in combination extremely effectively. You can create assertions on each Html element returned from each page within the web session.

And it also lets you pick the browser you want to simulate. You can tell it to act like IE8, Firefox, Opera, etc. So if the web page you are testing has a browser dependent issue, you should be able to test for this using HtmlUnit.

HtmlUnit for Automation of Tasks

I was working on a team who heavily used Salesforce.com. There were several daily audit procedures (like TPS reports from Office Space) that required manual entry of the audit data into the Salesforce website. It was a tedious step and I hated doing it, so I decided to see if I could write a Java program that could automate those annoying manual entries.

Well, what API do we know of that allows you to simulate a user web browser session through a web application? HtmlUnit of course!

In under an hour, I had created a Swing application that took the Audit inputs and punched the values into Salesforce automatically. HtmlUnit logged into Salesforce, traveled to the page that lists the audits, clicks the “Create audit” button, fills in the fields, and clicks submit.

And then I started to realize the possibilities are endless with this tool!

Some Limitations

While HtmlUnit is very cool and extremely powerful, it isn’t perfect. As you can imagine, web browsers are extremely complex and have invested 1000’s of hours of development to create them. The open source developers who made HtmlUnit do not have those sorts of resources and therefore are not going to be able to perfectly simulate each web browser.

So apart from the HTML results of web page actions being quirky and a bit different than you’d expect at times, the other common issue is that the JavaScript engine fails to parse the JavaScript on one or more of the pages and throws exceptions. Once the JavaScript fails to parse, you will have to disable JavaScript completely.

So if you find yourself performing mundane website data entry and you get sick of it, or if you need to test the full user experience through your website instead of just the backend Java code… give HtmlUnit a whirl.

Open source application audits using source code scanning tools are a critical part of a corporate open source software policy management and governance process; there literally is no way around it these days.  Without the use of a scanning tool, organizations may rely on homegrown tools, manual inspection and inventory of source code repositories, and developer interviews to implement the governance process.  In our experience, even with full disclosure of open source usage from very honest and open development teams, things slip through the cracks.  And, lets face it, manual inspection of source code is painfully slow.  Homegrown tools might be a realistic approach for larger companies, but they require the allocation of internal resources, not only to use the tools but also to also maintain and update them regularly.

Most open source auditing engagements are completed in the context of scanning a code base of a product line to confirm that a company has appropriately separated their intellectual property from the third party components.  When third party components are used and distributed all licenses for these components need to be identified and there needs to be confirmation that appropriate license compliance steps have been taken.  OpenLogic’s Application Audit and Certification of Compliance services are one solution to consider when outsourcing to a team of experts as these are a full report of all materials, licenses, and a re-verification of compliance steps being completed.

Dependency Scanning Use Case

Depending on the industry and level of maturity of the open source policy management process, a more granular level of scanning may be needed.   Open source packages often bundle other open source software within the larger or parent project.  Some companies want to know not just which open source projects are included in their code, but also identify and then associate the sub-components or dependencies to a parent project.  Open source communities come in all shapes and sizes with varying degrees of attention to the issue of documenting dependencies.  In fact not all open source communities that build and maintain projects accurately disclose and update the dependent libraries that the project uses.  There may have been significant changes from version to version resulting in an old and previously accurate list of dependencies being partially incorrect in the newest versions. Consequently, what was once a pre-approved version of an open source project to use in a distributed code base, could easily be a policy violation and potential license violation in that next version.

If you are familiar with OSS development and license types a single file can make a very big difference.  For example, in one of our scans the OpenLogic audit and IP analysis team actually found a license conflict between source code components in an open source project.  We contacted the community to inform them of the conflict as they were not even aware this conflict existed.  The community acknowledged someone had in fact contributed code that created this conflict and the community did the right thing for their end users by removing the conflicting code and replacing it.

If you scan and analyze the open source software project code directly, you can then determine all the dependencies that are used by the specific version.  For example, if an organization's states that the most recent version of Zlib must be used, then this organization would complete a scan to find out if anything has changed from version to version.  As a result, the organization can then confidently make the statement to customers, investors, acquiring companies, etc. “Yes we ship the Zlib library with our product, we always ship the most recent version of Zlib, and we can tell you exactly what Zlib is using in the newest version.  Would you like to see it?”  Then obviously the company would introduce the most recent Zlib Bill of Materials and Licenses to the audience.

The OSS Deep Discovery scanning tool has a customizable setting for this exact situation thus reducing the number of false positives found in the initial results.  In other words, by adjusting the settings accordingly, the scanner will identify all the components inside of Zlib instead of simply reporting that you have matches to Zlib.

The real world example for this level of diligence goes back to having and managing an actionable open source policy.  Open source review boards that have monthly, bi -monthly, weekly, or even impromptu daily meetings about what can and cannot be used and under what conditions need the ability to quickly identify and document these occurrences, make decisions, implement critical policy rule changes and communicate all of this easily to the organization.  One new or changed file can make a big difference in protecting millions of dollars of development and intellectual property.

What an exciting time it is for the open source software community! 

Cloud technology is evolving faster than we can read about it. Marketplaces are now part of our common vernacular.  The ever changing world that we live in and these advancements in technology are providing the open source software community with infinite opportunities to showcase creativity to the world.

I have highlighted three of the more recognizable, and recently launched marketplaces, that speak to this opportunistic time for open source software contributors and users.

Amazon Web Services (AWS) Marketplace

On April 19th, Amazon announced the launch of their Amazon Web Services (AWS) Marketplace.  The announcement was heralded by many in the cloud technology and open source software arenas, as a giant step for "cloud-kind."  ZDNet, an extremely popular tech-blog, referenced the recent AWS Marketplace launch in an article titled, "Amazon Launches AWS Marketplace as One-Stop Cloud Shop."  Hitting the nail on the head with that reference, the marketplace provides a unique opportunity where customers can "check-out" right from the AWS Marketplace platform with pre-configured stacks and software as well as a variety of services, right in their own personal shopping cart.  The marketplace also integrates with the AWS cloud, which allows for purchase and deployment of your technology without ever leaving the platform.

Salesforce.com AppExchange

Another pioneer in the growth of marketplaces, SalesForce.com, announced the launch of their marketplace, AppExchange, last November, and then almost immediately announced that their AppExchange had gone mobile.  The mobile announcement provided Salesforce.com users with a single location to enhance their experience by incorporating the availability of mobile apps to eliminate most plausible obstacles to accessing their data.

The AppExchange provides partners with the opportunity to develop apps with the "social, mobile, open and trusted capabilities" of Force.com, Salesforce.com’s social enterprise platform.  Salesforce.com's AppExchange provides users with easy mobile access to critical data and contributors with the opportunity to distribute apps to over one million users on AppExchange. This in turn fosters creativity and forward thinking by providing developers with an opportunity to decrease the time spent on building customized apps for their company's needs.

The AppExchange marketplace offers several free open source applications such as the HTML5 Contact Viewer and Salesforce Viewer for iPad.

With so much discussion around marketplaces and the value they're providing to the open source community, I thought it would make sense to hear first hand from one that just celebrated its first birthday!

HubSpot App Marketplace

HubSpot, a Cambridge, Massachusetts based marketing software company, launched their own HubSpot App Marketplace in April of 2011.  The marketplace currently has 54 apps available and serves over 6500 customers.  After being founded in 2006 by two former MIT students, (Brian Halligan and Dharmesh Shah) and receiving over $65 million in funding from sources like Google Ventures and Salesforce.com over the last 5+ years, HubSpot has become arguably the leading SaaS marketing solution available at this time.

Arjun Moorthy, Vice President Business Development and Partner Products at HubSpot, and closely involved in the success of the HubSpot App Marketplace, was kind enough to spend some time sharing his thoughts on marketplaces, success, open source software, and beyond.

Please feel free to add your own answers in the comments section below!

Q:  What do you think has been the biggest software related factor contributing to the growth and success of your marketplace?

A:  Growth has come largely from a very low barrier to develop apps on the platform and also provides customers an easy way to install and use those apps.

Q:  With the announcement of Amazon AWS Marketplace and marketplaces popping up with great frequency, what do you think has been the biggest reason for this expansion?

A:  The iPhone has made the metaphor for app purchases become mainstream.  It no longer seems technical but actually consumer-friendly.  So now other software vendors have followed suit and are doing their own app marketplaces.

Q:  In your mind, what makes for a successful marketplace?

A:
1.Easy to create apps
2.Easy to adopt and use apps (one-click install, integrated billing)
3.Clear support
4.High quality apps to seed the market
5.Quality controls to maintain sanity of marketplace

Q:  Can you speculate as to where you see marketplaces evolving to?

A:  In the business world, marketplaces may coalesce around verticals and/or functional lines.  So, marketers will have a marketplace for their needs, finance professionals may have theirs, people who work in healthcare may have theirs etc.

Q:  In your opinion, what has the evolution of marketplaces done for open source software?

A:  Many marketplaces (iOS App Store, Google Play, Facebook Application Directory) thrive because of the perception that a single developer or small team can quickly develop an app that has immediate access to a high-interest user base.  That quick development relies almost completely on open source development frameworks, tools and technologies to come together.

Q: How do you think open source software has contributed to the growth and evolution of marketplaces?

A:  The impact of the large availability of high quality tools for development can't be overstated.  The total cost of development to put together an "app" is in most cases a trivial amount beyond the time of those involved.

Arjun provided some great insight and details that appear to highlight the similar common denominators existing in each of these three successful marketplaces.  There seems to be one common implied value that these successful marketplaces offer to us: they make our lives easier.  Somehow, someway, to whichever audience they have chosen to serve, the ease of use, the low barrier to entry, or the tasks they simplify, it all makes our lives easier and to me, seems to be the fundamental formula of a successful marketplace.

How would you answer these same questions?

OpenLogic is very proud to announce that we were included in the launch of the Amazon AWS Marketplace.  You can view all of our open source software products and solutions here, and be sure to check out our fully flexible PaaS solution, CloudSwing, to deploy your apps to the cloud in minutes and also choose a support option that fits your open source software needs.



Subscribe to The Enterprise Open Source Blog via email

Follow @openlogic
Follow @cloudswing
Follow @AaronMandelbaum

View Aaron Mandelbaum's profile

This work is licensed under a Creative Commons Attribution 3.0 Unported License

.

JBoss, Tomcat & Apache are 3 of the most popular open source packages being used in the enterprise and have been for quite a while.  Enterprises tend to have a high level of comfort using them in their development and other non-production environments.  But as soon as you bring up the idea of open source software in production a funny thing happens.  You can hear brakes squealing, sirens going off, and see panic set in with business leaders in your company.  Then the questions start.  What if it breaks?  Who is on the hook at 2 am?  Is it secure?  Is it as good as commercial software?  What about indemnification?

Now, your head is spinning too.  Take a deep breath and let me explain the answers to those questions.

What if it breaks?  Who is on the hook at 2am?
Purchasing SLA technical support from a third party vendor means that if you have problems or something breaks you can call them.  Most production support provides a 1 hour response time and a 72 hour work around time, even at 2am.

Is it secure?
Open Source Software isn’t any less secure than commercial software.  In fact because it is open and has more eyes on it at any given time, security vulnerabilities can be caught much sooner.  Our support team also routinely looks at each package we support and cross references it with the National Security Vulnerabilities Database.  We know the same day that a vulnerability is found and notify our customers.

Is it as good as commercial software?
If you mean is the technology as advanced?  The answer is often that it is more advanced.  Open Source Software communities don’t rest on their laurels.  They are constantly writing code to enhance their project.  They track and fix bugs.  With the increased number of eyes on a package, development happens quickly.  And since OSS communities are a meritocracy you can rest assured that individuals who are contributing and committing code are top notch developers!

What about indemnification?
This will vary by vendor.  Some vendors don’t offer indemnification and many others do.

Now what are you waiting for?  Tell your manager you want to use OSS in production and tuck a copy of this blog in your back pocket to answer all of the questions before they're even asked.



Subscribe to The Enterprise Open Source Blog via email

Follow @openlogic
Follow @CloudSwing

This work is licensed under a Creative Commons Attribution 3.0 Unported License
.