Review By: Andrew L. Pollner
07/09/2010

This book offers sound concepts in software testing and defines the TestFrame Methodology. The 235-page book comprises six chapters and an Appendix. It discusses the following topics: Introduction, Preparation, Analysis, Navigation, Execution, and Test Management.

The book begins by reminding us that testing is more than just “ensuring a system does what it is supposed to do”; rather, it also needs to be about finding faults, because through these faults we can gauge the “quality of an information system.”

After briefly explaining the benefits of starting the test process early, the authors turn the lens of quality measurement from the information system back to the testing process itself. Six characteristics of a test process are described, each of which can aid in ensuring that a test has an established level of quality, thereby making the testing of an information system highly effective. From here we move to the TestFrame methodology—a structured approach which is conducive to rapid and thorough testing. The TestFrame model is depicted as a Greek temple whose roof symbolizes an information system’s quality and efficiency objectives. The temple’s base represents reusable test components. The key then is to create supporting “columns” or properties to allow the base to properly support the roof. Clever!

Four TestFrame phases—preparation, analysis, navigation, and execution—describe each part of a test process and are introduced in the first chapter. Then, chapters 2, 3, 4, and 5 are each devoted to one of these phases.

In the Preparation phase, we begin by exploring questions about the structure of a system. These help us determine our desired results, effort required, organizational planning, and test environment conditions.

The Analysis phase follows the Preparation phase, although the case is made that phases do not always need to follow a rigid prescribed order. In this phase we seek to ensure that the content of a test set is both correct and complete. This is done through the definition of data required to execute a test, clusters (a testable unit), test conditions, and test cases. We define the scope of testing by assessing depth and breadth of the testing effort and include available documentation to make this determination.

Navigation begins with an explanation of the advantages automation has over manual testing. Automated testing is an important part of the TestFrame method. The advantages of TestFrame automation are then compared to basic test automation, including making the test components more accessible; scripts that are easier to read by nonprogrammers; trace-back of scripts to test cases; and customized reports. Ensuring that a testing tool is compatible with a given application is stressed along with key issues to examine.

Next the authors describe how one would go about creating an action-word function using a form of pseudo-code called TestFrame Virtual Language (TVL). TVL is used to illustrate the programming logic required without resorting to favoring the programming language of any of the available test automation tools. This section is not for the nonprogrammer, and will quickly lose a tester who either a) has never programmed a testing tool; or, b) has no prior programming experience. This section is best suited for those testers who are already comfortable programming scripts and creating functions but want a more structured way of doing so. The chapter discusses good programming practices and naming conventions.

The final phase of the TestFrame model is execution. Here’s where the rubber meets road. Where the tests need to work properly, check accurately and report thoroughly. Strategies for running tests are described, including whether to continue or restart testing once errors surface. A discussion of how outcomes can be interpreted and what to measure and report follows.

The last chapter covers test management—the pitfalls and roadblocks that frequently get in the way of progress when new and progressive ideas are introduced to an organization. The single appendix covers additional details of TVL and serves as a programming primer.

The authors provide valuable concepts and information on how to structure a test process. Much of this information can be applied to an existing test process and make it better, or as a template for a new test process. A major aspect of the book is dedicated to discussing actions and action words and explaining how these can be created and implemented with functions that essentially become navigation scripts.

The benefits of using action words is that test cases appear less cluttered and that each action word can be reused often. This is because each action word comprises several individual actions that are documented, through manual or automated means, elsewhere. However, the true benefit of action words must come when automated tools are used to process them. It appears that the TestFrame method—at least that part of it which covers development of test cases and execution—is really created with automation in mind. It is in this area that this book may fall short for testers wanting to learn how to implement action words to their existing automation tools. The skills necessary to codify this technique require detailed knowledge of how test tools work along with strong programming skills. Comparisons are made to the traditional method of automation—record-and-playback. However, a more apt comparison would be with data-driven approaches, which most tool vendors now support.

Although the TestFrame method also supports manual testing, this was not described as clearly. Specifically, as action words help to abstract the details of navigation, I came away wondering how a tester was supposed to execute a test manually without having the detailed reference to the specific navigation steps involved. The test case tells you what to do, but not to the details of how to do it. That detail is presumably found elsewhere. This is in contrast to a traditional verbose test case which spells out every detail of the “what” and the “how.” To summarize, the TestFrame method is one of several alternative approaches which are gaining momentum in the software testing community and which can really help an organization better structure and better automate its testing.