The Open Source Test Tool Paradigm

Testing is often seen as an effort to determine the quality of the product at the end of a project, so it needs to be executed when development has finished instead of being a means to deal with risks at the earliest stage possible. Therefore, project budget, is in most cases spent on the processes that actually produce tangible products, at the expense of the testing budget. Whatever budget is left for testing will be spent on people rather than on test tools, especially since most of the mainstream tools are often perceived to be too expensive. A solution to this may be found in the use of open source test tools. With no license fees, the use of open source tools can provide a customer some of the benefits of test automation, without the costs.

At this moment, Open Source test tools should be on the verge of penetrating the market as stated by Gartner in 2008. Next to this, actions of the Dutch government, have also contributed to the promotion of Open Source Software.

fig 1

You would then expect that some years later, adoption of Open Source test tools would be common practice. But, even today, Open Source tools are still not used very often within organizations, at least not for test purposes. There are some professionals who use Open Source test tools because of lack of any tool at all. But even then the company they are testing for, does not support the tools and sometimes even disapproves of the use.

Why is this then? In my opinion there are several reasons for this, which I found during my work with Open Source test tools and test departments. Some reasons are the usual “open doors” and some will be myths fed by ignorance.

fig 3

The “nerd-in-the-attic” syndrome

Open Source is often related to the idea that it was created by someone still living with his parents on their attic doing nothing all day but playing games and writing lines of code. To be honest, this may be the case in some of the Open Source projects, but it does not have to degrade the functionality and the quality of the software. If the project was started this way, there should not be a problem as long as it is adapted by a large community. Linux is just one of the greatest examples in this respect, created by a lonely Fin and adapted by some hackers, maybe not on the attic, but nonetheless. For that matter, Microsoft, Apple and Facebook were also created by the “nerd-in-the-attic.”

Most projects nowadays are not the work of a lone ranger, but in some cases, the work of large multinationals that started the project and then released it to the Open Source community.

Support issue
Easy access does not always mean ease of use. As some companies might say; “We downloaded it, now what?” Companies tend to forget that with Open Source the licensing model and therefore support is different from Closed Source software. The projects are sometimes supported and sponsored by large vendors, but most of the time, it concerns communities that work on a “when there is time” basis.

In most cases, the majority of test tools are not (yet) supported by any vendor. This can become a problem when issues in the tool itself occur or changes should be made in order to use the test tool within the environment of a customer. However, there are companies specialized in the support of Open Source software who have the knowledge and the volume to provide this support. It is therefore not a support issue as such, but a shift in mindset, that the supplying party will not be the one to provide the support.

Security uncertainty
Security is always an issue no matter what software is used. With Closed Source software, the security issues are mostly covered by the vendor. With Open Source there is no vendor to guarantee this. The general idea with Open Source is that, it can be hacked very easily because of the Open character. Of course, the source code can be viewed by anyone who wants to do harm. On the other hand, because of the Open character, Open Source is better secured. This seems a contradiction in terms, but can be supported by the fact that anyone can contribute to a better security of the software, whereas with Closed Source, it is the sole responsibility of the vendor. The advantage of Open Source is that everyone can see how the software works and contribute to it.

Technical knowledge
Downloading Open Source is easy enough, but it is not always just a simple “click” and “play” executable. For some projects, it is even necessary to download additional software (freeware or Open Source) in order to be able to use it. Most of the time, people tend to stop right after unzipping the downloaded file, not having found an executable with it. This is true for a lot of Open Source Software, but it is improving. Installations have been made easier and user manuals made more accessible.

But even without the improvements to the software itself. it is good practice to have some technical knowledge in house, This may only be for the purpose of having the ability to implement the software on your own network and for support.

It is free so it must be rubbish
Free software cannot be good. The comparison to a shop is often made where free products almost always have an implication or some fine print, which makes sure the customer buys more, excluding the “free” product. There seems to be a hesitance that free software somehow is related to malware and therefore not installed on the network of the company. “You can and may use it, just not on my network!” With this comes the big load of Open Source licenses, which seems to have been overlooked. Free is not always as free, as people seem to think. Every Open Source Software product has a license of its own and there are a lot of differences between them. There are about 1300 Open Source or comparable licenses from which 72 are OSI approved.

All in all, the distrust of Open Source software can easily be disproved, but it should be taken into account that Open Source has its own advantages and disadvantages. Just like Closed Source, there is a solution to every problem, but with Open Source these solutions may be of a different kind. Support can be found with third parties and the guarantee, provided by large companies, of the security of the software. Technical knowledge is only needed when you want to use the software yourself and not involve a third party. In other words, the use of Open Source Test Tools can be a solution for a lot of testers who do not have any tools available to them because of the huge costs of the Closed Source solutions. This should not be feared by the IT departments, but should be embraced as an opportunity to gain better quality through the use of supporting software no matter what the origin is.

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.