TrainingConferencesAbout UsContact UsAdvertiseSQE.comRSS Feed

StickyMinds.com: brain food for building better software

Log In
 Clarify Your Search Criteria

Tips on Using Our Search Feature(s)
 
StickyMinds.com Home
ResourcesTopicsCommunityPowerPass
Home  >  Detail: After the Bug Report



A StickyMinds.com Original
Article Picture
After the Bug Report

By Danny R. Faught

Send This Content to a FriendGet a Short Link to This ContentPrint This ContentSee User Comments About This Content

Summary: We crank out bug reports and expect them to return like a boomerang so we can check to see if the bugs were fixed. In this week's column, Danny Faught shares some ideas drawn from recent experiences that could make you a better customer advocate subsequent to filing a bug report.


AccuRev
Before the Bug Returns 
 
The people on your team may add comments to a bug report as part of an ongoing dialog about how to handle the bug. This gives you a nice history about decisions and additional details about the nature of the bug and the fix. After you file a bug report and send it on its way, there may be more opportunities for you provide additional useful information. 
 
On my current project, the programmers tend to add comments to the bug reports assigned to them when they learn new information. They also may add comments and forward the bugs to the development manager, seeking feedback. These are opportunities for me to see if there's more information I can provide about the nature of the problem that I didn't think to include when I first filed the bug. Also, I may comment on how well I think any proposed fixes will work. That greatly shortens the communication path compared to having the programmer completely implement a fix before I have a chance to review it. 
 
Our bug tracking system notifies the team when a new comment is posted, so it's easy to track these conversations. If your system makes it more difficult to see comments posted, or if the volume of activity would make it impractical to monitor the things you're interested in, you might have to wait until the boomerang comes back to you. Also, if the programmers are hostile to receiving feedback when they're working on a bug, you're better off following a more formal process. 
 
When the bug is fixed, more or less. 
 
The boomerang returns...In many organizations, the people who file bug reports are asked to verify the fix when the fix is completed. Or you may be testing the fix for a bug someone else submitted. My goal is to complete this step with at least as many bugs open as when I started. 
 
Of course, you should try to reproduce the bug the way you originally reported it. Sometimes, the problem is still there, as if nothing at all was fixed. Perhaps because of a configuration management glitch, the fix isn't in the build you have. Or maybe there's something specific about my configuration that I either didn't report or the programmer didn't think was significant. When I reject a bug fix outright, I try to include additional details about my configuration and exactly how I reproduced the bug. I use different wording than I did the first time around to help reduce any misunderstanding. And I write with a tone that expresses puzzlement rather than condemnation. In almost all cases, the programmer really did make an effort to fix the bug, and we need to recognize that effort. 
 
What if the fix did some good but doesn't address the bug to your satisfaction? You have a judgement call to make. If the changes to the software represent some sort of progress, and can stand alone if no further improvements are made, then I recommend closing the bug as fixed. I don't like keeping a bug report open for a long time, spawning many changes, and possibly changing its focus several times.  
 
Open one or more new bug reports, describing the additional changes you would like to see. Doing this before closing the existing bug will help make sure you don't get distracted and forget to follow up, and it will give you the ID numbers for the new bugs that you can reference in the comments when you close out the old report. It's good to leave a trail between related bugs. 
 
On the other hand, if the fix is really unfinished, go ahead and reject it; send the bug back to the programmer. Maybe an obvious new bug was introduced, or some small detail was overlooked. It would be easier for the programmer to clean things up while the code is still fresh on her mind than to wait for a new bug report to filter through the system. Deciding whether to reject a fix or file a new bug report is often a tough one, but if you have developed a good working relationship with the programmers, you'll get a good result either way. 
 
While you're at it... 
 
While I'm checking a bug fix, I sniff around for other bugs in the same area of the application. If the code is growing or changing frequently, I can often dig up additional problems to report without much extra work. Bugs tend to cluster together, and bug fixes tend to unmask latent bugs you couldn't see before.  
 
Even if you are under strong schedule pressure, it's worthwhile to spend a few minutes to get a few more bugs filed. Organizations that support exploratory testing are more likely to see the value of this additional effort. 
 
Not a bug? 
 
Ouch. It's not fun when someone says the problem you so carefully isolated and reported isn't really a problem after all. Here are a few recent cases I've been involved with: 
 
The software I'm testing has a key component that is provided by a third party. Several times I've filed bugs that flew right back to me, with a comment that not much can be done about it. This is the point where my hackles as a customer advocate go up. Customers won't care whether the problem is in our code or someone else's. They only care that the software doesn't work. In these cases, I'll do my best to explain the impact of the problem to the customer and maybe suggest a way we can work around the problem. I'll be sensitive to the fact that a workaround might be expensive to implement. 
 
Another example is when the programmer decides that something I think is a problem is not a problem at all. I reported a bug that could cause the user to be unable to log into the application for several hours. The developer said that this was the expected behavior, and the development manager agreed. I was out voted, but far from giving up. Rather than continuing the discussion in the bug report’s comments, I sent an email to the programmer and the manager explaining why I thought it was a mistake not to fix the problem. Not only did they agree with me, but the manager also validated my opinion by posting my message into the bug's comments. But the saga didn't end there. I thought that the fix that was implemented didn't go far enough to address the problem. Since the fix did work as designed and didn't introduce any new problems, I closed it out and opened a new bug report asking for more. Given the improvement that was made, we were all able to agree that this new request is valid but a lower priority. 
 
The key to being a good customer advocate is to consider the human elements of the development process. Practice the fine art of holding your ground on issues that affect customers while maintaining effective communication with the programmers and managers. Hopefully I've given you a few useful tips for doing that better. 
 
Resources 
 
How to Make your Bugs Lonely: Tips on Bug Isolation http://tejasconsulting.com/papers/bug-isolation-pnsqc-2004.pdf, Pacific Northwest Software Quality Conference 
 
An Elusive Diagnosis http://tejasconsulting.com/newsletter/2002April-May.html#feature, Tejas Software Consulting Newsletter


About the Author
Danny R. Faught has been throwing bug report boomerangs for twelve years. He is proprietor of Tejas Software Consulting (http://tejasconsulting.com/) and maintainer of testingfaqs.org. You can contact him at faught@tejasconsulting.com. Danny is a regular StickyMinds.com columnist.

Back to Top
 
 

Member Comments
Add Your CommentExpand Comments
 
Comment:    
by govind p 4/25/2006

Danny, U r cen percent right. i also faced the same problem which u explained in the topic. U r very good in expressing QA person's thoughts. Its a good topic for buddies and experienced QA's

 
 
Comment:    
by Surendra Oturkar 9/13/2005

Its Nice Article, But still i think bug classification makes bug soving more easier and makes the area to work on the bug specific, so I think this point should also be considered.

Author's Response:
4/25/2006    
Certainly there are fields in a bug tracking database that should be used to classify the bugs in various ways, including severity, priority, and functional area. I only gave a brief glimpse into a part of the bug tracking process.

 
 
Comment:    
by Rameez ali 5/26/2005

Dear Mr. Faught, It was nice to know that the Defect life cycle related concerns of testers all over the world are nearly the same. And you have indeed put them in a very clear way. You are right, the developers look at the application from a different angle than Testers. I think both Developers and Testers should work in the contrary department to better understand the concerns and constraints associated with that job. I would appreciate if you could comment on that.

Author's Response:
5/26/2005    
Hi, Rameez. I think you're asking about how the organizational structure affects the testers' ability to get bugs fixed. I prefer to work closely with the developers, if the developers and development managers have a good appreciation for quality. However, if the developers don't really understand what the testers are doing and why, and the development managers don't encourage sound development practices, you may need a more formal and independent organizational structure in order to get your point across when you find bugs that should be fixed.

 
 
Comment:    
by Imran Jah 4/20/2005

Hi Danny, your take on 'Bugs & Psychology' (if I may call it so) is excellent. Just one thought, it is said that anything that does not match requirements is a BUG. But as testers, we must also look at things from Customer's point of view, thus, also taking care of the unstated but desired requirements. However, its not easy to get the programmers to think in the same light. What can be done in such situations..??

Author's Response:
5/26/2005    
Get a good book that covers bug reporting, such as "Lessons Learned in Software Testing." That'll give you a lot of tips on how to be a good bug advocate. Filing an effective bug based on an unstated requirement requires a lot of understanding of the context that you're working in.

 
 
Comment:    
by Jehanzeb Riaz 4/12/2005

Excellent Article Danny, this is the reality and you explain it very nicely. Cheers mate….

 
 
Comment:    
by Yuriy Shvets 4/6/2005

Danny, I am doing exactly the way you have described in your article. I do have some issues I would like to post and maybe if possible to get some comment back from you. Consulting Company that have never had QA before and trying to establish processes and build QA department. Based on the project I am working on I have created the flow of Build deployment and notified all people on the team that testing will only be performed on QA environment. Lately my Project Manager has decided to implement some new changes. She wants me know to do a "Smoke" test on Development Environment Before I can actually deploy build to QA and do...Read On

Author's Response:
4/10/2005    
Hi, Yuriy. I think we need to have more of a discussion than this forum will allow. Feel free to send me email and I'll see if I can give you some tips. Doing a smoke test is not a bad idea in general, though you can debate who should be doing it. It sounds like there are some snags in the process you need to get ironed out, but it's hard to tell at this point where they are.

 
 
Comment:    
by Bahadur G. Asher 4/6/2005

Hi Danny, it's a very good and minute observation that you have done. We also many a times face a similar issue of programmers returns the bug sheet with comments such as "bug not reproducable" or bug not found. I found 2 major reason for this kind of thing happening 1st reason could be the bug is not explained properly or not enough steps are listed to reach to that bug and the 2nd reason is that the bug list is too big and there are other major bugs that need to be looked in and they did not had enough time looking into this bug. I also agre with you that if you look carefully then you will find bugs in clusters most of the time...Read On

Author's Response:
4/10/2005    
Thanks for your comments, Bahadur. One thing that helps me to write reproducible bug reports is to try to reproduce the bug on different systems - the more they differ, the better. Often I will be surprised that I don't see the bug on the second system, so then I have some additional isolation to do. I'll end up writing a much more useful bug report that the developer is more likely to reproduce.

 
 
Comment:    
by John Delaney 4/6/2005

Yes a very good article. On the subject of not reproducable defects, the defect should include steps to reproduce along with a system environment description.

Author's Response:
4/10/2005    
Thank you, John. I'm always making subjective decisions about how much detail to include. It's easy to think the ideal is to provide vast amounts of information, but if the bug is easy to understand, it's actually easier to understand with a succinct description. When I do provide lots of additional information, like a long stack trace, I'll try to put it in a part of the bug report where it won't clutter the main point.

 
 
Comment:    
by Venkatesan Kalingamurthy 4/6/2005

It is a good article. You are bringing a very good point on the relationship between tester and developer. The very important point I like is "to consider the human elements of the development process". Anyway both are working for develping a good product only. If we keep this in mind, bug triage meeting will not be a battle field Another point I may want to add is, sometimes bug may be returned to tester by saying "NOT REPRODUCIBLE". As a tester we should not close it just like that. We should take it with the programmer by explaining steps and environment that we have tested. In these situation we should cool our...Read On

Author's Response:
4/10/2005    
Hi, Venkatesan. At a bug triage meeting, my aim is to make sure the decision-makers understand the consequences of the bug. As long as I know I've provided the relevant information, it's easier to accept the decisions about priorities. Also, sometimes new information you find later will help escalate the attention a bug gets, so have some patience.

 
 
Comment:    
by Gandhali Samant 4/5/2005

ggod article. I liked the consideration of human element of the development process. Also the handlling of bugs if they are 'Not bugs' as per developers.

Author's Response:
4/10/2005    
Thanks, Gandhali. I do so much better now that I start with the assumption that the developer intended to do a good job among a variety of pressures, only some of which I'm aware of. But I'm not so nice toward the software itself - my attack on the software is unrelenting. :-)

 
 
Comment:    
by Paul Tsuda 4/5/2005

Danny, I really like this article, and I think you hit the nail on the head when you talk about good customer advocacy while remaining sensitive to the psychology of the programmers. Keeping this balance, doing this difficult dance, deserves the exposure you are giving it. Metrics and methodology are just empty words without this daily execution by experienced professionals.

Author's Response:
4/5/2005    
I'll give credit to Gerald (Jerry) Weinberg for opening my eyes to the people side of things. Technology is easy, compared to learning how to work effectively with the other people on a project.

 
 
Comment:    
by Robert Rose-Coutre 4/5/2005

Regarding a bug that is partly fixed -- If you close the bug, then open a new one to address the residual buggy behavior, be sure your test manager or some who attends meetings is up to date on it. In daily bug-report-review meetings (hopefully with at least a programmer, dev. manager, test manager, and doc person), the attendees are usually flying through the bug-report items as fast as possible. They will probably recognize the bug title and remember fixing "something like that" and drop it or "bury it," etc. This is especially likely in larger organizations with many testers reporting dozens of bugs per day, per...Read On

Author's Response:
4/5/2005    
Very good point, Robert. It's important to mention in the bug report the existence of the other similar bugs (so they don't think you're blindly filing a duplicate, and say explicitly why this one is different. Despite my best efforts to distinguish similar bugs, I've had bugs rejected in the manner you describe. In some cases, it's debatable whether it's really one bug or two, and the path of least resistance is to make to sure to track all aspects of the problem within one bug report. Then you can be ready to file additional reports if a fix comes in that doesn't cover everything. Timing is important.

 
 
Comment:    
by sushant mishra 4/5/2005

Hey,Its a ood article that we faced in our daily basis work.

Author's Response:
4/5/2005    
Thanks, Sushant! These subtle persuasive skills can make a big difference in how effective a tester is.

 
Back to Top


Marketplace

RESOLVE SUPPORT ISSUES from your Desktop!
Minimize downtime with a remote support solution that lets you resolve issues right from the desktop

TAKE CONTROL OF REMOTE COMPUTERS
Support, configure and install applications and updates remotely for greater efficiency.

Web based bug tracking - AdminiTrack.com
AdminiTrack offers an effective web-based bug tracking system designed for professional software development teams.

Need Agile Test Cases?
Create statistically complete test cases simply and quickly.

SOLVE SUPPORT ISSUES on the First Call!
REMOTELY CONTROL AND CONFIGURE SYSTEMS. Easily install applications, updates. All from your Desktop!

Get your product or service listed here.
Subscribe to Better Software Magazine
Subscribe to Better Software Magazine

First Name:

Last Name:

Email Address:


Home   |   Resources   |   Topics   |   Community   |   PowerPass



© 2008 StickyMinds.com. All rights reserved.
StickyMinds.com is a division of Software Quality Engineering.
Privacy Policy    Terms & Conditions    Link to StickyMinds.com    Feedback


Ravenflow



Better Software Conference & EXPO 2008