During the lifetime of a test tool it will be necessary to upgrade, maintain, or customize the test tool. The upgrade and maintenance of test tools needs to be carefully managed and planned in order to mitigate the risk of affecting the end users. This article offers a framework for helping organizations cope with the changes that a test tool experiences during its lifetime.
Test tools fall under various categories. Test Management tools handle version control, defect management capabilities, storage for test scripts, test planning artifacts, test execution, and test results. Record/Playback tools capture and playback recorded processes. Capacity Testing tools generate system load and traffic. These test tools have different user populations.
Test Management tools typically have a large end user population. Developers, business analysts, and testers have access to the tool. In contrast to Test Management tools, Record/Playback tools and Capacity Testing tools have a much smaller end user population. The test manager should notify end users when a test tool will be brought down for either maintenance support or an upgrade. There are many reasons for upgrades, maintenance, and customization of test tools. A few of those reasons are:
- Tool needs to have some patches installed.
- Tool needs to be customized to support a new software release or testing effort.
- Vendor stops supporting an older test tool version or might phase out a test tool.
- Tool's back-end requires database maintenance (i.e., archive older records, database upgrade, etc).
- Vendor folds up due to financial problems, and the test tool no longer has technical support from vendor.
- Vendor has completely upgraded and revamped the test tool to include new functionality or to fix previous bugs.
The test manager should document the risks and rewards associated with making changes to a test tool and only make changes to a test tool when the risk is minimal to the end users and the testing efforts. An organization should not upgrade a test tool merely because a vendor has come up with a newer version. An organization should initiate a tool change only when there are compelling business needs driving the change in the test tool. A reason for upgrading a test tool would be that a previous version of the test tool does not recognize the project's custom controls during recording, and the new tool version will.
Test Script Maintenance
The test manager should review with the vendor what would occur to previously recorded test scripts if a test tool is upgraded to a newer version or if patches are installed. If a test tool is upgraded from version 1 to version 2, will the previously automated test scripts also need to be upgraded or re-recorded? Are the automated test scripts capable of playing back on the newly upgraded test tool? What if the automated test scripts require immediate execution and there is no work around, other than re-recording the test scripts because the test tool has been upgraded?
These are issues that require much cogitation, especially for companies that have libraries of recorded test scripts from various testing releases. The test manager needs to consider the existing testing phases and assess whether upgrading the test tools is feasible. If all the test scripts need to be reconstructed or modified for the newly upgraded test tool, it might be wise to wait until the project has more resources and no imminent deadlines to upgrade the test tool. If the test tool upgrade does not affect previously recorded test scripts but offers new enhancements and features, it might be practical to work with the latest version of the test tool.
A test tool upgrade might require a transitioning phase for the end users and the reconstruction of customizations to the test tool. As an example, I once upgraded a test management tool from a fat client installation to a new thin client installation running as a Web based solution and requiring the use of a browser to launch the test management tool.
Although the newly