When unnecessary information is entered into bug-tracking systems, they are not as effective as they could be. In this article, Adam Kolawa outlines what type of information should be entered into a bug-tracking system during development and after code freeze, and how to use the information when designing future product versions.
In my opinion, bug-tracking systems are poorly used. Most commonly, a bug-tracking system is used as a forum that allows the development and quality assurance teams to communicate with one another about product problems. This usually results in a system that is flooded with information about simple bugs that can be detected by the test suite. I believe this approach is misguided. I have found that bug-tracking systems should be used exclusively as a place to store feedback when you cannot immediately modify the code.
Using Bug-Tracking Systems during Development
When the product is in the development stage, developers ideally build test cases as they build code. They start by adding test cases that verify that each piece of code they add or change works correctly. These test cases are collected into a regression test suite that is run on a regular basis (preferably, every night). Each time a bug is found at this stage of the development process, a developer or QA team member develops a test case that verifies whether the bug is fixed, then adds this test case to the regression test suite. Test suites are thus constantly expanded and can be used to ensure that previously detected errors do not resurface as code is modified.
At this stage in the development process, the bug-tracking system should not be used to track defects. Test cases identify each error, so entering this same error information in the bug-tracking system would be superfluous. The bug-tracking system should be used only to store ideas at this point: every time somebody has an idea for a new feature or for a better way to implement an existing feature, this idea should be entered into the bug-tracking system as a feature request.
Using Bug-Tracking Systems after a Code Freeze
Once the product enters the beta testing phase, source code is typically frozen. Because code in this stage cannot be immediately modified to respond to feedback, there is an added need for an information repository. This repository should now include both error reports and feature requests. Many companies try to enter only "real bugs" in their bug-tracking system, and soon find that there is a fine line between bugs and feature requests. While some issues (such as program crashes) are obviously bugs, classification of other "problems" is more difficult. For one person, a program lacking an expected GUI feature might be a high-priority bug; for another, it is just a feature request.
We recommend that instead of being petty and quibbling over definitions, you enter all feedback and change requests in the bug-tracking system so that you can easily access it later. For classification purposes, use "bug" to refer to a product issue that prevents a user from using the product as advertised, and call everything else a feature request. Change requests should be recorded in the bug-tracking system throughout the code freeze stage—the time span from the initial code freeze to the day you start planning the next release. As you receive feedback, you can track the demand for each feature. The first time a feature is requested, you enter it in the bug-tracking system. Every subsequent time a user asks about that feature, you record another request for that feature. (You can store this information in any variable you like; for example, in Bugzilla, requests can be logged using the "vote" feature.) If you track requests in this manner, it will be easier to separate must-have features from eccentric ones.