Have you been measuring how many defects you find at unit testing and system testing per KLOC and how much may cost to fix them?
I can find some information like:
with code reviews + unit testing one can find up to 70% of the defects.
https://www.iiitd.edu.in/~jalote/papers/2WCSQPaper.pdf
Other studies say that the cost of finding and fixing a defect at unit testing is $428 and at system testing is $1045.
http://www.ppi-int.com/systems-engineering/free%20resources/Software%20Q...
Do you know more benchmarks per industry type ?
What is your experience ?
Thanks in advance
Code review, unit testing, and any code quality approach certainly make code a little better. Ideas around defect density per line of code have been debunked a long time ago though. Bugs are a social thing, there is no relation between a line of code and the number of bugs that might be there.
Hi Justin, thank you for your feedback.
You are right, bugs that are inserted in code "are social", depend on the team.
But the amount of bugs you find at each test stage (static analysis, code review, unit and system test) may match a pattern.
During a two years project in embedded software I implemented a metrics program and checked that the amount of bugs found matched industry standards.
In the second mentioned article is explained that defects per LOC is an incorrect measure because each programming languages requires a different amount of LOC to implement a functionality. Number of bugs found at each test stage gives better information about how effective is your testing and if you also measure the time to fix them, will give you extreme valuable information for future planning.
Defect density stats always should be analyzed carefully. If a tester reports that the Defect density had come down, it may be good as well as bad. Good because defects have come down and Bad because defects are not identified effectively.
Defect density can be used in a project where there is similar modules and same programming language. e.g. A project that rewrites developing 5000 policy statements that are sent to customer. Here the type of tests written for each policy statement is similar to the others and the metric can be used only to compare the effectiveness between modules.
Never ever use any industry standard comparisons for this metric please.
StickyMinds is a TechWell community.
Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.