Open source development projects typically include a testing phase, just like their commercial counterparts. But how do open source projects manage the testing process? What tools and resources do these projects use? This article discusses the ways that open source projects accomplish testing goals, and how open source testing compares to testing in a commercial project environment.
Imagine that you've become the test lead for Project X; however, your testing team has no official members. You have to rely on the contributions of volunteers from around the globe to accomplish testing tasks. You'll have almost no face-to-face contact with your testers. You have to provide all of your volunteer testers with access to Project X's software. You have to create a bug tracking system that can scale to 1,000+ users. You and your team have to accomplish testing tasks and meet the goals of the testing effort.
What would you do? How would you manage testing for Project X?
Open source projects have to tackle many of these issues when managing the testing process. There are challenges unique to the open source arena, so innovative, resourceful tools and methodologies must be called upon to accomplish testing tasks.
What Is Open Source Software?
There are many opinions regarding the definition of "open source" software. In fact, a large sector of what I'm calling the open source community rejects the open source label and prefers the term "free software" (as in free speech). This article is not a forum for such a debate. For the purposes of this discussion, open source software is considered any software whose source is freely available, with no limitations on source modification and redistribution. There are exceptions to this rule, and many different software licenses are considered open source (e.g., the GNU General Public License, the BSD license, and the Mozilla Public License), but such a broad definition will suffice for this discussion.
The GNU/Linux operating system is perhaps one of the most recognized examples of open source software. For an excellent introduction to the concepts behind open source development, read Eric S. Raymond's essay collection The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary (published by O'Reilly).
For open source projects, development occurs in the global community and proceeds in an evolutionary manner. Many people use the software while it's still in development. Users file bug reports. Bugs are fixed. Community debate and discussion steers project development. The open source development model resembles a community bazaar. As Raymond writes, "The Linux community seemed to resemble a great babbling bazaar of differing agendas and approaches…out of which a coherent and stable system could seemingly emerge only by a succession of miracles."
What Challenges Do Open Source Projects Face when Testing?
Open source projects introduce unique challenges when testing a software application, especially when compared to commercial development practices. Some of the major challenges are as follows:
No full-time testers. When working on a commercial project, you expect to have other test team members to work with. If you're a test lead, you expect to have a test team whose efforts you direct. However, in the open source community there are no full-time, paid testing teams. Volunteers, typically working in their spare time, contribute most of an open source project's testing work. Open source endeavors must constantly promote awareness and recruit volunteers from the community. Testing productivity is directly proportional to the number of active volunteer testers.
Global team coordination and communication challenges. Volunteer testing teams contain members scattered across the globe. Testing team efforts must be coordinated on a global scale.