The problems due to unstructured, decentralized test management can be solved by reengineering the test management process. A testing project starts by building a test plan and proceeds to creating test cases, implementing test scripts, executing tests, and evaluating and reporting on results. This article explains the goals of reengineering test management and how to achieve them.
The Importance of Software Testing
In March 1992, a man received a bill for his unused credit card stating that he owed $0.00. He ignored it and threw it away. In April, he received another and threw that one away too.
The following month, the credit card company sent him a very nasty note saying they were going to cancel his card if he didn't send them $0.00 by return mail. He called them and they said it was a computer error and told him they'd take care of it.
This is a rather old but amusing story found on the Internet, but current reality is not very different. According to a recent study on the economic impact of an inadequate infrastructure for software testing in the U.S., the total national annual cost resulting from the inadequate infrastructure is estimated to be $59.5 billion.
It is generally believed that software can never be made perfect.
Testing is necessary because the existence of faults in software is inevitable.
Consequently, we test software to find as many faults as we can to ensure that a high-quality product with a minimum of faults is delivered. Identifying defects is the testing team's primary responsibility. However, there is another important responsibility-verifying that the application's functionality meets the user requirements.
Testing and Test Processes
Testing is the process of exercising software to verify that it satisfies specified requirements and to detect errors. The main objectives of testing are to identify defects, nonconformance, and associated risks in a work product; to communicate all known issues to the project team and ensure that all issues are addressed in an appropriate manner before release; and to ensure that the system meets the requirements of the customer.
A generic functional testing process is illustrated in the diagram below:
Testing is not the last step; it is an ongoing part of the iterative development process.
Testing is not a one-time activity-applications need to be tested throughout their lifecycle. Every version upgrade, module addition, or enhancement, as well as every implementation at a new site or increase in user load, needs to be put through comprehensive testing.
Testing Efforts in the Development Lifecycle
A program written in one hour can have thousands of possible logical branches to test. That makes testing the most time-consuming, labor-intensive, and costly part of the development cycle.
Depending on the risk and complexity of the application under test, the average percentage of the software development lifecycle that is normally devoted to the testing effort is between 30 percent and 50 percent. The cost of testing in the majority of commercial system development can be between 40 percent and 60 percent of the total cost. The percentage may be more or less in different environments, but the important issue is that the cost of testing is significant.
The Need to Reengineer Test Management
If testing is such an important and significant part of the project lifecycle, how is it managed in most projects?
Most organizations don't have a standard process for defining, organizing, managing, and documenting their testing efforts. Often testing is conducted as an ad hoc activity, and it changes with every new project. Without a standard foundation for test planning, development, execution, and defect tracking, testing efforts are nonrepeatable, nonreusable, and difficult to measure.
Generating a test status report is very time consuming and many times not reliable. It is difficult to procure testing information such as
- How much testing needs to be done?
- How much testing has been completed to date?
- What are the results of these tests?
- Who tested what, and when was it