How Much Should One Test?

Do you need 20 test cases or 200, maybe even 2000 depending upon the situation? As a test manager you want to provide some rationale around the scope of testing you plan to conduct. Having a well thought out testing scope helps to mitigate the risk associated with testing too less or too much. Here are some guidelines on how to land somewhere in the middle.

• Business complexity: Complexity means–more business requirements, more exception processing, more interfaces, more data requirements, more flows etc. This complexity leads to more business scenarios and thereby adds to the testing scope.
• Business criticality: More does not mean important. To compensate for complexity you need to narrow your focus on criticality. Criticality defines importance to the business and can be either a critical function and/or a heavily used function. Critical functions need to be addressed prominently and will allow you to offset testing some non-critical functions.
• Development maturity: This includes a variety of items that can influence the amount of testing that may be required. Development components that have an impact on testing include–adoption of a formal methodology, incorporation of various review steps, usage of formal and documented testing artifacts like test cases, definition and usage of a formal defect management process, change management process and metrics tracking. More maturity on these lines leads to less testing and vise versa.
• Team strength: A skilled and resourceful team that has been involved in the development effort will create less defects and issues. A new or inexperienced team will introduce more defects. In this situation the 'more testing' focus should be during early testing phases like unit testing but overall testing scope will be increased.
• Technology platform: Scope of testing may increase due to environment availability, infrastructure configuration challenges etc. This is especially applicable when dealing with new technology and tools. Technology challenges usually impact the testing timeline and can be addressed as appropriate contingency on the testing plan.
• Experience: In most cases there is some precedent to provide a basis for the testing effort. This basis could be a similar project, similar testing effort, similar client engagement etc. Leveraging previous experience helps to put this effort in perspective and provides the 'gut check' needed to finalize the testing scope.

Documenting your reasons for the test scope definition and associated assumptions creates a key test planning artifact. This strategy not only drives the planning process but also provides factual basis for the testing estimation model.