An Ounce of Goat or a Pound of Hero: Software Reviews and Inspections

[article]
Summary:
Being a hero is great, but only if your heroism doesn't involve a situation you could have avoided in the first place. In this week's column, Harry Robinson explains that goats are to forest fires as early detection tools are to the software development process. There is a lesson about software reviews and inspections in this comparison that testers and project heroes can benefit from.

When do your testers join the software development process? If you're like most teams, testing doesn't start until late in the development cycle. Once testing is engaged, you work around the clock finding the important bugs, fixing those you can fix, and trying to get the code ready to ship. In all, your testers and developers will put in several extremely late nights and probably a few weekends to get the code out the door.

Last-Minute Heroes
Testing that starts late in the software development cycle gives many bugs that were originally small and simple a chance to become deeply entangled with other bugs in the system. These entrenched bugs are often difficult to find and nearly impossible to fix well.

On the bright side, everyone who survives the dreaded days of debugging will likely be proclaimed a hero. But I wonder if heroism in these situations is as necessary as we are often led to believe.

Let's step back for a moment and look at similar heroics in a totally different context. Smokejumpers are firefighters who parachute into forest fires armed with shovels, picks, and little else. This matches up pretty well with my idea of heroism. Their work is important, exhausting, and dangerous. Fatalities occur on a regular basis.

But an ounce of prevention may really be worth a pound of cure--or hundreds of acres of forest. Some communities have begun using goats to deal with wildfires. Instead of waiting until a fire starts, these communities hire goats to munch away on the underbrush. This starves a potential fire of the fuel that would allow the flames to spread. This is not glamorous work, but the goats effectively clear many acres in a short time.

Mike McMillan/Spotfire Images

Goats vs. Smokejumpers

  1. Goats work cheap. 650 goats probably cost as much as one firefighter.
  2. Goats are not nearly as exciting as smokejumpers. Watching goats munch on underbrush is only marginally more interesting than watching the underbrush grow.
  3. Using goats requires planning. Goats are of little use once the fire has started. (Unless you plan to feed them to the firefighters.) To be effective, they must be used before the emergency hits.
  4. Goats are not a complete solution. Fires will still happen and you will still need firefighters-hopefully not as many or for as long.
  5. Measuring the success of goats is tricky. You can't talk about fires extinguished; you have to talk about fires prevented. Fortunately, if you have some idea of how many fires would usually flare up, you can get an estimate of how helpful the goats have been.

Now, let's return to your team's software development efforts. On your last project, did you get pulled into firefighting mode because your testing found serious bugs only late in the development cycle? Did you repeatedly send in your developers to contain and debug the problems? I'll bet your developers worked mainly in reactive mode, moving from one hot spot to the next since they had no time to create long-lasting solutions. In other words, your people were heroes heading straight into burnout.

But what if your team had taken a more goat-like approach, stopping small, isolated bugs before they could become large, tangled bugs? What if your developers and testers used cheap, simple methods during the development phase to lessen the presence of bugs later in the process?

 
Reused with permission: San Diego Union-Tribune

The truth is we have had such tools and methods around for a long time. We just haven't always used them as well or as often as we could have. For instance, to name

User Comments

44 comments
Michael Cummings's picture

Great article. It was well-received by both the test and dev teams here. One comment I would make is that we have to try to make sure that the heros are not the ones starting the fires - which seems to be the case all to often. MC

October 15, 2004 - 11:23am
Michael Cummings's picture

Great article. It was well-received by both the test and dev teams here. One comment I would make is that we have to try to make sure that the heros are not the ones starting the fires - which seems to be the case all to often. MC

October 15, 2004 - 11:23am
Michael Cummings's picture

Great article. It was well-received by both the test and dev teams here. One comment I would make is that we have to try to make sure that the heros are not the ones starting the fires - which seems to be the case all to often. MC

October 15, 2004 - 11:23am
Michael Cummings's picture

Great article. It was well-received by both the test and dev teams here. One comment I would make is that we have to try to make sure that the heros are not the ones starting the fires - which seems to be the case all to often. MC

October 15, 2004 - 11:23am
David Ramger's picture

I heard on ESPN recently a comment from one of the sportscasters who was discussing baseball's weekly Top 10 Plays of the Week. He pointed out that a large number of the exciting diving catches that were being highlighted on the show were a result of a very good athlete making a great play only because he was out of position. There seemed to be a lot of younger players who made the highlight reel simply because they were good athletes who *had* to make a diving reach to catch a ball that the older, more experienced players made easily because they were in the right spot in the first place, but made the same catch look routine. <br/><br/>The same thing applies to programmers. The younger, less experienced coders tend to expend a lot of energy fixing problems that older, more experienced coders would never have made in the first place. And yet it seems to be the flashy heroes that are rewarded for fixing the problem rather than the one who prevented it in the first place.

October 11, 2004 - 2:44pm
David Ramger's picture

I heard on ESPN recently a comment from one of the sportscasters who was discussing baseball's weekly Top 10 Plays of the Week. He pointed out that a large number of the exciting diving catches that were being highlighted on the show were a result of a very good athlete making a great play only because he was out of position. There seemed to be a lot of younger players who made the highlight reel simply because they were good athletes who *had* to make a diving reach to catch a ball that the older, more experienced players made easily because they were in the right spot in the first place, but made the same catch look routine. <br/><br/>The same thing applies to programmers. The younger, less experienced coders tend to expend a lot of energy fixing problems that older, more experienced coders would never have made in the first place. And yet it seems to be the flashy heroes that are rewarded for fixing the problem rather than the one who prevented it in the first place.

October 11, 2004 - 2:44pm
David Ramger's picture

I heard on ESPN recently a comment from one of the sportscasters who was discussing baseball's weekly Top 10 Plays of the Week. He pointed out that a large number of the exciting diving catches that were being highlighted on the show were a result of a very good athlete making a great play only because he was out of position. There seemed to be a lot of younger players who made the highlight reel simply because they were good athletes who *had* to make a diving reach to catch a ball that the older, more experienced players made easily because they were in the right spot in the first place, but made the same catch look routine. <br/><br/>The same thing applies to programmers. The younger, less experienced coders tend to expend a lot of energy fixing problems that older, more experienced coders would never have made in the first place. And yet it seems to be the flashy heroes that are rewarded for fixing the problem rather than the one who prevented it in the first place.

October 11, 2004 - 2:44pm
David Ramger's picture

I heard on ESPN recently a comment from one of the sportscasters who was discussing baseball's weekly Top 10 Plays of the Week. He pointed out that a large number of the exciting diving catches that were being highlighted on the show were a result of a very good athlete making a great play only because he was out of position. There seemed to be a lot of younger players who made the highlight reel simply because they were good athletes who *had* to make a diving reach to catch a ball that the older, more experienced players made easily because they were in the right spot in the first place, but made the same catch look routine. <br/><br/>The same thing applies to programmers. The younger, less experienced coders tend to expend a lot of energy fixing problems that older, more experienced coders would never have made in the first place. And yet it seems to be the flashy heroes that are rewarded for fixing the problem rather than the one who prevented it in the first place.

October 11, 2004 - 2:44pm
Jim Bernstein's picture

Harry<br/><br/>One of the most truly entertaining articles on Sticky Minds! The problem you describe plagues most industries however most companies are so busy firefighting they can't go to the market to buy the goats. I try to convince people each day that finding a problem in development costs much less than finding it in pre (or worse) post production.<br/><br/>Jim<br/>Solstice Software

October 11, 2004 - 3:35pm
Jim Bernstein's picture

Harry<br/><br/>One of the most truly entertaining articles on Sticky Minds! The problem you describe plagues most industries however most companies are so busy firefighting they can't go to the market to buy the goats. I try to convince people each day that finding a problem in development costs much less than finding it in pre (or worse) post production.<br/><br/>Jim<br/>Solstice Software

October 11, 2004 - 3:35pm

Pages

About the author

Harry Robinson's picture Harry Robinson

Harry Robinson is a Software Engineer in Test for Google. He coaches teams around the company in test generation techniques. His background includes ten years at AT&T Bell Labs, three years at Hewlett-Packard, and six years at Microsoft before joining Google in 2005. While at Bell Labs, he created a model-based testing system that won the 1995 AT&T Award for Outstanding Achievement in the Area of Quality. At Microsoft, he pioneered the test generation technology behind Test Model Toolkit, which won the Microsoft Best Practice Award in 2001. He holds two patents in software test automation methods, maintains the Web site Model-based Testing, and speaks and writes frequently on software testing and automation issues.

StickyMinds is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Nov 09
Nov 09
Apr 13
May 03