Are there industry standard definitions for test execution results?

Mike Oliver's picture
Mike Oliver asked on April 14, 2015 - 5:55pm | Replies (1).

I'm working on refining our definitions for test results and have the following:

Pass - Test executes and all verification points pass (behavior matches requirements)

Fail - Test executes and 1 or more verification points fail (it does not meet requirements)

Blocked - Test case cannot be executed due to a defect in current or associated product (something like.. i cant even install the software)

Skipped - Tester determined that test did not need to be run and did not execute the test (He/She knew it would fail due to an already logged defect for example)

 

There is a contingent of folks who want to use the following result:

Pass with Exceptions -- Test case passed all verification points but failed due to an anomaly found in a related product or in an area not related to the requirements being tested.. for example -- i'm looking at load performance and i see an unrelated defect in the GUI.

 

I am opposed to this definition as it seems to be a little unclear with regard to intent and I have seen the definition get changed to allow failing test cases to "Pass with exception"

 

Has anyone seen any standards published that provide guidance in this area?

 

Thanks,

 

Mike

 

1 Answer

Matthew Heusser's picture

How about a simple red/yellow/green? Red means "needs rerun", yellow means "check the notes, yo umight want to rerun this", greem = "all good, but if you change the code a lot ..."

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.