Strategic Testing in the Web World


How does your company measure the value of testing their Web applications? Do they only notice when an application fails or when a business partner or end user calls with a complaint? What if tests could provide management and marketing with information to directly affect the bottom line? QA would then become a different kind of hero. This article offers ways you can promote the value of testing and QA in your organization.

Most companies measure the value of Web application testing in terms of fewer tech-support calls and emails to the Webmaster. In fact, management probably doesn't consider the role that testing plays until an application slows, fails, or exhibits a glitch. Solving problems during these moments proves your worth, but what if you could provide your boss with return-on-investment (ROI) numbers from testing and provide strategic information to the marketing department? Companies are constantly looking at possible actions and purchases with business value. Test and QA engineers can participate in strategic business goals by providing ROI data. This article presents ideas for strategically evaluating software test vendors, examples of ROI from testing scenarios, and some of the benefits realized by automated testing.

Evaluating Software Test Vendors for ROI
Space Requirements
Carefully consider the resource footprint of the test software as well as the ability of the software to scale. Will you have to buy additional machines or memory to run the test software? What's the memory usage per virtual client? One company was able to cut hardware and memory needs in half by getting more virtual users out of each PC. This is because the test software vendor they chose had a smaller footprint and additional functionality for easy scalability.

Buy or License Only What You Need
When evaluating test software packages, ask the vendor about different licensing scenarios. With the many different models of purchasing software today, you should be able to purchase only what you need for your project. For example, a California consulting firm was able to benefit from a short-term license to run load and stress tests for one of its major clients. Depending on your project, you may not need long-term licensing.

Consider the Hidden Costs of Testing
How much time will a test engineer need for training? Will the engineer have to spend a week away from the office to learn a new system or a proprietary scripting language? Can they get up to speed and be productive quickly, or will it take a month or so of time, in the office, jammed in between other projects, which will slow the start of the Web application testing? Once testing is underway, will it require constant rework to continue to run? Look for testing software that offers flexible options and uses standard languages, such as JavaScript, as well as visual point-and-click recording to create and manage test scripts, in order to shorten the learning curve, jump-start the testing process, and keep it moving.

ROI Discovered Through Testing
Before Testing Begins-What's Your Hardware Capacity?
If you load test your Web application with the idea of optimizing your company's hardware configuration, you may find you have more capacity for testing than you thought. One international investment research firm was planning to outsource the testing of a new Web site to verify the performance and reliability of the application and to determine the hardware requirements. They decided to perform the testing in-house. Through the load testing process, they determined that its base hardware estimate was sufficient to operate the Web site.

Testing for Memory Leaks
Just as a leaky faucet drains money from a homeowner, memory leaks on a Web application can tie up a team with needless technical tasks. For example, a major financial services firm discovered through load testing that cookie verification in the production Web site was causing site instability. Prior to this discovery the testing team was scheduling daily reboots of twenty-two of the site's servers. With the new testing results, the company saved time and energy each day.

Benchmarking Information for Sales and Marketing

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.