Caution: Independent Test Group Ahead

For software companies that develop complex products, independent test groups that don't have strong product expertise can actually decrease the quality of the delivered programs because separating test from development sets up a situation where groups blame one another instead of working together. Let me explain.

Ideally the best person to test a program is its creator. They know the code, they know the design, and they want their programs to work because it's a reflection of their reputation as software developers. Also, bugs are fixed much quicker by the original programmer. But there are problems with this arrangement. First, human nature. Few mothers believe their baby is ugly, just as few programmers believe their creation is flawed. It's not easy to be critical of your own work. Also, humans are goal oriented creatures. If you approach a task with the goal of proving it works, the goal steers you toward test cases that will help you achieve it. As a result, many quality assurance experts believe the best way to find bugs is to approach the testing task with the goal of finding problems, which can be a difficult mind-set for programmers who test their own code.

The other big advantage of having programmers test their own code is to avoid "the blame game." By this I mean that the existence of a separate test group leaves no one ultimately responsible for bugs. If a bug slips through, the programmer can say it's the tester's fault. On the other hand, the tester can say “garbage in, garbage out”, in other words, the program is in a lot better shape than when it first arrived, and given the time and resource constraints of the project, it works better now than before. So who do you believe? Actually, it doesn’t matter who you believe because it’s the customer who uses the buggy program whose opinion really matters.

Having subject matter experts in the Independent Test Group helps alleviate the problems inherent in "the blame game" because it accomplishes one very important thing: the tester has something on which to base his value to the overall project. The programmer already knows his worth—special knowledge of computer systems. The tester has product knowledge to back claims of program faults while adding value to the end product. James Bach, a software testing expert and co-author of the book, Lessons Learned in Software Testing, says to testers, "Your most important relationship is with the developers. This is the one that makes your job or breaks it." A relationship between self-respecting (and hopefully mutual-respecting) individuals at least has a chance of not breaking down into finger-pointing. Another testing expert, Bret Pettichord, in an article entitled Testers and Developers Think Differently," that appeared in the January/February 2000 issue of Software Testing and Quality Engineering Magazine also describes the importance of this symbiotic relationship: "As system designs become more complicated, developers have to spend more and more time and effort making sure that they understand the designs. At those times, having testers take responsibility for understanding user needs and behavior can be very helpful to making sure that the product is tested in realistic scenarios."

I recently finished a UC Davis Extension course entitled, Software Testing Methodologies, taught by Gary Mulcahy, CQA, CSTE, and President and CEO of Ranch River Group, Inc., a software quality consulting company in the Sacramento area. One thing he said that really stuck with me was selecting test cases is the most important thing a software tester does. It occurred to me that if he's right, then subject matter expertise is necessary to select the best test cases. Otherwise how do you even know where to start looking? Then after finding a bug, selecting the right test cases is important to further isolate it by discovering under what specific conditions and minimal steps it occurs. This enhanced bug report has positive effects that go well beyond testing. It can help project managers determine how important it is to fix a particular bug, and it can help programmers implement the best fix in a much more timely manner. Next, and often overlooked, selecting the right test cases is vital when testing the fixed program to insure no other serious problems are accidentally introduced. Finally, knowing the product as good as an expert user helps speed up test-setup, often the lengthiest part of executing a test.

Of course, an independent test group with product experts is no silver bullet. The process must still be managed effectively so “the blame game” doesn’t become an issue. Programmers must still write and test their own code, while product experts help programmers identify and set up test scenarios only experienced users of the system can think of. I still think the ideal situation is one where the programmer tests his own code. It's kind of sad that we don’t live in an ideal world. But we can't let that stop us from finding ways to be successful in this imperfect one.