Tips
Here are some specific ways you can use one fixed bug as inspiration for finding more bugs in a product:

• Look around the same area where the first bug was found. You already know to check that the bug fix works and didn't introduce new problems. Also look for latent bugs that may have been masked by the first bug.
• Look for the same bug on another platform. Your product might be ported to more than one operating system, or there might be overlapping features in a natively compiled application and a Web application. See if you can find a similar bug in places where the same feature is implemented.
• Look for a similar feature in other areas of the product, to see if a similar bug afflicts these areas. You may find that a user interface change that fixed one bug should be propagated to other areas in order to maintain consistency.
• If the original bug involves a corner case, push farther into the corner. If you make your test data more stressful or unusual than before, you may find additional bugs.
• Check the documentation. The bug fix might have changed user-visible behavior, and the user documentation might need to be updated to reflect this. If you've shed light on a limitation or subtle detail that wasn't documented before, consider whether it would help the user to mention this in the documentation.
• Look for bugs that have nothing to do with the bug fix you're testing. If you watch carefully, you will notice other bugs along the way that you weren't even trying to find.

How It Fits In
When I use a fixed bug to generate ideas for finding additional bugs, I often find a flurry of newly discovered problems that I need to report. If I don't find any additional bugs, and I actually allow the count of open bugs to drop, I feel like I'm missing something. If you get that feeling too, then you've successfully adopted the habit of making bugs multiply.

I usually end up on projects where I'm testing a product with many bugs. If you're testing a mature product, you may not find as many additional bugs. But you won't know until you look.

If you're required to follow a highly structured process, you might have difficulty finding sufficient time to explore the additional areas that I'm suggesting in addition to your scheduled duties. In that case, I recommend taking a minute or two to see if you find any new symptoms. If you do, tell your manager your concern about the problems you find, and ask to schedule some time to explore those areas. You may have uncovered an area of the application that's under tested. If you've wandered into an area that someone else is responsible for testing, share your findings with him, and take advantage of his specialized knowledge about the best way to address problems in that area.

You may be wondering why I didn't suggest using this checklist right after you've filed a bug report rather than waiting until the bug is fixed. Some of these tips may help you further isolate a bug you've found or to find more bugs based on a bug you just reported. However, I suggest using the checklist after a bug is fixed because a design decision has been made at that point. Once you know how one bug has been fixed, you have valuable information about the scope of the fix that is hard to predict when you first report a bug. Once you know the boundaries of where the fix has been applied, you're ready to explore those boundaries with additional testing ideas.

These ideas have helped me to submit a relentless onslaught of bug reports. The developers' best hope is that the severity of the bugs is decreasing on average as the number of open bugs continues to rise. I hope you can use some of these ideas to make your own bug reports multiply.

About the Author

Danny R. Faught rarely fails to find a bug when he goes looking for one. His independent consulting practice is Tejas Software Consulting, where he has recently launched a new blog http://tejasconsulting.com/blog. You can contact Danny at faught@tejasconsulting.com.

Back to Top 
 

	Comment:     by Swarna Latha 10/7/2005 Your suggestion is good. But what should we do if we go with the test cases given to us. We are asked to test only those areas which the testcases cover. Ofcourse we can do adhoc testing. If we dont find time to do adhoc, or if the client hadn't asked for that, what should we do in that situation? should we leave that application with hidden bugs ? Author's Response: 10/7/2005     I would recommend that you use your best instincts to sniff out where bugs might be lurking outside the reach of your scripted tests. Take a few minutes to see if you can find any problems, even if you have to do it on your own time. If your organization is disfunctional, don't act on the first bug you find - wait until you find a serious bug. Then ask if your management if anyone wants to know about it, and explain that you found it while doing exploratory testing. If you do this a few times, you might convince management of the value of exploratory testing. If you do it right, they won't think that the best way to react is to simply add one more scripted test. Organizations are usually difficult to change, so you may have to be patient, and learn more about how to be a change agent. Be careful not to alienate yourself in your efforts to improve things.
	Comment:     by Surendra Oturkar 10/5/2005 Once you find a bug in the system then it is almost impossible that you dont have an other bug, even if bug fixing is not done well then it itself introduces lots of bugs. So following your suggestion is best way to deal with bugs. -- Surendra Oturkar
	Comment:     by Darryl Hurmi 10/4/2005 The question is do you always have to enter a bug report to fix a problem. I am working with a development team that is adding a new protocol to an existing product, every time a programmer is added to the team to work on a piece of the code they make the same mistakes. They are so use to using library routines that they have used for years that they don't think that the routines aren't valid for this case. getting them in a room and explaining the issue and how it needs to be addressed saves a lot of time for other bugs. This saved me a day of writing bug reports and verifying bug per programmer, I still ran the test but it passed. Author's Response: 10/4/2005     As long as your organization doesn't have a metrics process that requires all the bugs to be officially reported, then I think it's often a great idea to streamline the bug reporting process like this. Just a few days ago I tested new feature, and instead of isolating and filing a dozen bug reports I emailed a list of findings to the developer. Some he fixed quickly (saving us both some effort), and the rest I'm filing in the bug tracking system so we don't lose track of them. Several resulted in discussion about the cause and possible fix, as well as the design goals, which also educated me about how the system was intended to work. Your approach to educate the developers is a good one, if you can get the message across effectively.
	Comment:     by Tony Beck 10/4/2005 These are great points Mr. Faught, and maybe that's because they work. In the educational software testing I've handled over the past five years I have employed this very same logic - if a developer has fixed a defect, he's introduced some more - and it continues to be a very successful method. Developers will often say "How did you find that?" and Project Managers will thank you. Still, it's the nature of QA that sometimes they will ignore it until the cutomer reports it. Because they're on a timeline, they have to make tradeoffs. Nice, thoughtful article. Author's Response: 10/4/2005     Sometimes a developer introduces a new defect with a code change. But just as often, they unmask a defect that was already there, and sometimes, when you're not sure what the cause of a cluster of problems is (and how many causes there are), when part of the cluster is fixed, you know what else is still left to report. Regarding customer reported problems - yes, several times I will have a bug languishing in the database, and the first time a customer reports it, I escalate the priority, and it gets quick attention. The response is faster than if no one had isolated and reported the problem before. I don't mind that some of my bug reports get a low priority until we get solid evidence that they're affecting users. But if you're seeing that most of your bug reports are being ignored, it's time to ask whether you're looking for the kind of bugs that the organization wants you to be finding.
	Comment:     by ritam sharma 10/4/2005 These thoughts are nice. But these are more good when you explore at the begining stage for bugs. If you are end of the release, testing the corners would help. One should understand there are endless bugs one can find, if the product is not very well unit tested. Above techniques will help you. Testers should always think of the ways the change is going to effect the test plans/cases. Most of the companies dont have a string assurance processes. I would still prefer to have a matrix where one can map the number of bugs found to find the weaker area and then plan the above kind of mindset for each funtionality. Author's Response: 10/4/2005     I realized after writing the first draft that some of these ideas are also relevant before you know what the fix is for the first bug you found, though I haven't fully thought out how this maps to my exploratory testing and bug isolation techniques. Perhaps we'll hear from people who work with developers who do good unit testing - I bet the bugs are endless for them too, but harder to find. For information about the weak spots in the application - if you have a bug tracking system with good data on which functional or structural areas the bugs are in, then you have all the information you need.
	Comment:     by Babu Narayanan 10/3/2005 When the boomerang returns, I would like to see the unit testcase (added for the defect by developers) and test report before validating. If the tester has good development knowledge, they shall write the cause of the bug too in the defect log. Author's Response: 10/3/2005     It's great if you're on a project where unit testing is done well. That probably means you have to work harder than most testers to find bugs. Regarding the cause of the bug - yes, testers sometimes know the cause, for example, I sometimes file Javascript code with a web page bug. But if the tester doesn't, it's good to make sure the developer adds a sentence or two explaining what they fixed. This can give you ideas about follow-on bugs to look for, e.g., maybe the fix was incomplete.
	Comment:     by Mike Whittaker 10/3/2005 Once I discover a "bug pattern", I think the best use of everyone's time is to explore for and report other instances of that pattern, since a local fix of one bug might not be carried over into the corresponding other areas. Case in point - testing portable audio devices: plugged in device, opened 'properties', unplugged device, clicked 'OK' - app gave GPF. All the places where you could have a dialog open on the device, and then unplug it before proceeding, needed a fix. Better for everyone to get all these fixed as part of the same task ! However, as you imply, if only one common piece of code required fixing, then the...Read On Author's Response: 10/3/2005     Choosing how many bug reports to open is tough when you have the same symptom in several places. If there's interest, I can cover this in a future column. In your case, maybe the failures all track back to a common piece of code and only one but report is needed. Opening a dozen bug reports is annoying for both the programmer and the tester if they all have the same root cause. But if you guess wrong and they're not all the same bug, you might delay the fixes when you lump them all together.

Back to Top