how to analyze test results (in manual testing)? what metrics do you try to provide ?

anjana prasad's picture

how to analyze test results (in manual testing)? what metrics do you try to provide ?

1 Comment

Depending on what you are trying to do. Also depends on your audience. What type of manual testing are you doing? Is it exploratory or test case driven? Do you examine risks? Do you have acceptance criteria you base your tests on? 

1 Answer

Christian Fernando Kedidjian's picture

Dear Anjana. In my unassuming opinion, i suggest you to perform a previous test plan you must agree with the key user on beforehand. This plan must be shaped keeping in mind main functionalities and project risks. To easy guide the key user approbal, it is recommendable to define an acceptance and exit criteria. It will lead the user acceptance test execution without issues, allowing you to avoid having to struggle with things that were not planned.

To perform a good acceptance criteria, you must define which kind of defects and impairments you are going to manage and then you'd better rank them according to their impact. For instance, Blocker, High, Medium, Low and Enhancement. Where Blocker means that you can access some functionality to be tested or a bundle of test cases cannot be executed because of the defect you found.

High Impact means for instance that you cannot release the product. Tipicaly, this kind of defects impact on the business continuity.

Medium impact means that there is a workaround which allows you to release the product, but the impairment must be solved in shortly. You have to detail this in the release notes.

Low impact means that a defect exists but it does not affect the business workflow. For instance, look&feel discrepancies, grammar errors, etc.

Enhancement is not a defect. It is something that can be planned for the next release to improve the product.

Once you have this definition wrapped-up, you must write the acceptance criteria down by defining the amount of accepted defects for each funtionality to be tested. For instance:

Complex Functionality: 0 Blocker, 0 High. 2 Medium, 3 Low.

Medium-Complex Functionality: 0 Blocker, o High, 1 medium, 2 low.

....and so forth.

Keep in mind this are just examples, so you'd better analyze your scenario to better define kind of defects, business impact and your test plan and acceptance and exit criteria.

Hope that helps!


StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.