Software Testing has become a self-governing and a very important profession over a period. As the software development process is becoming a complex activity day by day, the demand to continuously evolve the software testing practices and keeping them aligned to the needs of software engineering is becoming important as well.
One of the most challenging aspects of software testing is designing good test cases. Since test cases lay a foundation for effective test management, and further for sustainance engineering, it should be treated as a product itself and test professionals should take pride in the quality of the test cases because it is their creation.
The objective of this paper is to introduce and discuss why test cases should be treated as a product. The paper further introduces the practices that have evolved to design good test cases that consistently helped testers to design good test cases, deliver them as a product and meet the demands of test projects that testers undertake.
Software Testing & Test Cases
There are many definitions of software testing, but following definition goes very close to the standpoint of this paper:
“A technical investigation of the product under test conducted to provide stakeholders with quality-related information.”- Cem Kaner and James Bach
As Cem and James explain, technical investigation is an organized and thorough search for information. The information objectives could be many. Such as finding important defects and to get them fixed, helping managers make release decisions, checking interoperability with other products, assessing conformance to specifications, etc. The information objectives depend upon stakeholders’ requirements.
As depicted in the figure below, various information objectives will require specific testing strategies and will yield different tests, different test documentation and different test results.
Figure 1: Representation of information objectives, test types and test cases
If we extend the thought of “information objectives (which could be many)” and “finding important defects (as a result of our thorough research)”, and attempt to find its relationship with test design activity, we may find Robert Binder’s definition of software testing helpful. When I look at Cem and James’s definition of testing, it shows me “what is” and when I look at Robert Binder’s definition of testing, it shows me “why should we”.
Robert Binder mentions in his book, “Testing Object-Oriented Systems”, software testing is an effective process to find defects that could be result of Poor user interface design, Elicitation error, Specification omission, Programming error, Configuration/Integration error, Inefficient algorithm, Inefficient programming, Infeasible problem, etc. These errors may induce defects that may fall in one of these types- Side effects and unanticipated, undesirable feature interaction, inadequate performance, real-time deadline failure, synchronization, deadlock, livelock and even a system crash.
Therefore, the test approach (or some may call it a test strategy) essentially guides us to understand what tests should we design to reveal the defects and the technical information.
Cem and James Bach teach in their BBST course that to get the technical information from a piece of software, one must ask a question to the program and this question is nothing but a test case.
Some other definitions of test case are-
“A set of test inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement.” (IEEE)
“A test idea is a brief statement of something that should be tested. For example, if you're testing a square root function, one idea for a test would be ‘test a number less than zero’. The idea is to check if the code