As delivery cycles get shorter, rapid test techniques are gaining in popularity. In this article, Sridhar Kasibhatla and Andrew Robins explore the concept of using coverage maps and test notes to support exploratory testing and concurrent test design. These maps and test notes also are used to review and track test coverage and can help document dynamically generated test cases for future re-use.
In this article, we explain some of the tools and techniques that we have developed and used that have helped provide test support for faster delivery cycles for a range of highly configurable products. Some of the ideas and tools contained in this article come from training we received from James Bach as part of his Rapid Software Testing course, which has influenced many of the recent changes in our company's testing process.
We always have implemented exploratory testing at our company, but after meeting James Bach our approach techniques changed. We now view exploratory testing as a key part of our test strategy for most releases. It is a planned, structured, and managed activity that supports, and in some cases replaces, other test approaches.
Two key concepts we adopted from James are:
- Session-based testing and regular session reviews
- Concurrent test execution and test design as a core competency for testers
Use of Test Notes to Support Session-based Testing and Concurrent Test Design
One of the issues we faced with exploratory testing was how to track and report our test coverage.
Now we promote that our testers should consider the notes they keep during an exploratory test session a key test deliverable. We encouraged this in a number of ways, but most effectively by regular review of test sessions and the test notes produced during these sessions.
Of course, we are interested not only in recording what we tested, but also in helping our testers make good decisions about what to test and what not to test. One way in which we have tried to meet the need to capture test outcomes and promote good design practices during exploratory testing is through the use of a test notes template. The test note template prompts the tester to consider risks and associated tasks, capture new test missions, and identify test ideas that he wishes to explore during the current test session. The test notes template is not and should not be applied universally to each test session, but it always should be considered for application, as the template establishes common expectations about the kind of information that should be delivered. It also is a useful guide for good test design.
However, one of the first decisions a tester may make is not to use the template and to capture his notes in another form.
If the concept of concurrent test execution and test design is taken on board, then test managers need to be open to receiving and reviewing test notes that are captured in a variety of different forms. They then need to be able to ensure that the various customers of testing are able to use the information provided. This isn't so hard to do if you've already built a relationship of trust with your development team. Exploratory testing as a whole works much better if the developers are plugged in to your process. This is best achieved by having regular discussions with the developers and getting them to explain the code structure, while highlighting any areas that they are concerned about. Doing this establishes a good rapport with the developer.
Figure one shows a fairly typical example of a set of test notes for a relatively tightly constrained piece of testing.
Figure1. Typical Test Notes at Tait Electronics
Use of Coverage Maps to Support Concurrent Test Design
A "coverage map" representing various test scenarios also can be used to help capture test notes and encourage good test design.
In our vocabulary, coverage maps are diagrams that can capture a lot of information and generate many test ideas. Most