Not only does testing the right things affect the return on the testing investment, but testing them in the right way makes a big difference, too. Various techniques exist for test design and execution. Some tests don't involve running the system at all, others do. Some tests require inside knowledge of how the system works, while others require an understanding of what the system does. Some tests are run step by step by hand, while in other cases a computer does the test execution, though skilled test engineers must perform the test design and result interpretation. The skilled test professional achieves the optimum return on the testing investment by selecting test techniques wisely.
A static test is one that evaluates the quality of the system without the system actually running. This may seem very strange, even to some experienced testers. How can we run a test without running the system?
A static test looks at the system—or portions of or artifacts related to the system—to see if we can find problems early. When programmers read their code after writing it, this is called desk-checking, which is one kind of static test. Another kind of static test, in this case a social one, is a review meeting that evaluates requirements, design, or code. The benefit is clear once you think about it: If you can find a problem in the requirements before it turns into a problem in the system, that will save time and money. (In Software Requirements , Karl Wiegers reports a return on investment for requirements reviews of as high as 800%.) Even if we find the problem by reviewing the code, that saves us the overhead of building, installing, and running the system to find the bug.
Not all static testing involves people sitting at a table looking at a document. Sometimes automated tools can help. For C programmers, the lint program can help find potential bugs in programs. Java programmers can use tools like the JTest product to check their programs against a coding standard, which can be customized.
To get value from static testing, we have to start at the right time. For example, reviewing the requirements after the programmers have finished coding the entire system may help testers design test cases. However, the significant return on the static testing investment is no longer available, as testers can't prevent bugs in code that's already written. For optimal returns, a static testing should happen as soon as possible after the item to be tested has been created, while the assumptions and inspirations remain fresh in the creator’s mind and none of the errors in the item have caused negative consequences in downstream processes.
Effective reviews involve the right people. Business domain experts must attend requirements reviews, system architects must attend design reviews, and expert programmers must attend code reviews. As testers, we can also be valuable participants, because we're good at spotting inconsistencies, vagueness, missing details, and the like. However, testers who attend review meetings do need to bring sufficient knowledge of the business domain, system architecture, and programming to each review. And everyone who attends a review, walkthrough, inspection, or whatever it’s called in your organization should understand the basic ground rules of such events. A good reference on this topic is Daniel Freedman and Gerald Weinberg's Handbook of Walkthroughs, Inspections, and Technical Reviews .
Structural (“White Box”) Testing
Static testing is a powerful—but not a sufficient—test technique. At some point we must actually run the system and look for bugs in it. Tests that involve running the system under test are