Testers gather lots of metrics about defect count, test case execution classification, and test velocity—but this information doesn't necessarily answer questions around product quality or how much money test efforts have saved. Testers can better deliver business value by combining test automation with regression analysis, and using visual analytics tools to process the data and see what patterns emerge.
There is much published about the data we generate to assess product quality. There is less discussion about the data testers generate for our own use that can help us improve our work—and even less is said about recommended practices for data collection. Test data collection, management, and use all call for upfront planning and ongoing maintenance. Here's how testers can improve these practices.
Discussions on what constitutes a “proper implementation” of test automation often focus on what tool you should use, but that is only one part of the equation. Bas Djikstra details four other things you should consider, how they contribute to the success of your test automation, and what risks are associated with failing to pay proper attention to each of them.
Test data generation is an important preparatory step in software testing. It calls for a tester’s creativity as much as test case design itself. Focusing on the type of testing to be performed and designing data to support it yields the greatest success in finding defects. For example, security testing largely requires negative test data to attempt to gain access to a system as a hacker would. Localization testing requires very specific test data in the areas of date, time, and currency. Rajini Padmanaban describes how test data generation is a reverse engineering process, where one first focuses on the end goal and then works back to determine what kind of data should be created. Rajini describes data sets for various types of testing, ideas to keep in mind in reusing test data, and sharing data across the product team to save time while not trespassing on the team’s creative thinking.