Test-Physicians, Source Code, and BSODs

[article]
The Ten Commandments of Software Testing, Part 3
Summary:

This is part 3 of a three-part list of nine "commandments" that can be used as a guide to improving your life as a test engineer. These are intended to be useful suggestions as well as for fun. Read on and see what you think.

As a response to numerous inquiries about these commandments listed on my Web site . This column is part 3 of the explanations. I mean these to be both fun and useful. You can also click below to read parts 1 and 2. Here is the full list of commandments so you can see them in one list. 

  1. Thou shalt pummel thy app with multitudes of input
  2. Thou shalt covet thy neighbor's apps
  3. Thou shalt seek thee out the wise oracle
  4. Thou shalt not worship nonreproducible failures
  5. Thou shalt honor thy model and automation
  6. Thou shalt hold thy developers sins against them
  7. Thou shalt revel in app murder (celebrate the BSOD!)
  8. Thou shalt keep holy the Sabbath (release)
  9. Thou shalt covet thy developer's source code

And now, here are my interpretations of numbers 7-9 (yes, there's really only nine).

7. Thou shalt revel in app murder (celebrate the BSOD)
I often make an analogy between testers and physicians. Physicians, I say in the story, treat their patients gingerly. They say "Does it hurt when I touch you here?" And then they promptly stop touching that spot when you say "Yes!" If testers were physicians, the story would be somewhat different.

Test-Physicians would also inquire "Does it hurt when I touch it here?" But when the pain is confirmed, a test-physician would then poke, prod, and probe, until the pain became unbearable. Why? Well, it isn't sadism, it's our job. No bad deed should go unpunished.

You see, every bug is a proud moment for a tester, but no bug should go without further investigation. So you found a bug that causes ill-formatted data to be displayed on the screen. Great, but can you go further and make that same bug corrupt data that the application is storing internally? If so, you have a better bug. And, can you then make that corrupt data be used by the application in some internal computation? If so, you have now turned a simple little formatting bug into a severity-one bug that causes data to be corrupted and the application to crash.

And, of course, the Holy Grail would be to crash not only your application, but to cause the entire operating system to hang. Ahh, the Blue Screen Of Death. I remember my first like it was yesterday, I anticipate my next every time I apply a test.

The moral of this commandment is that behind every good bug, there may be a better bug. Never stop exploring until you've discovered just how deep the bug goes and just how damaging it can be.

8. Thou shalt keep holy the Sabbath (Release)
Oh so many times I hear testers whine about release dates. Testers most often want to extend release dates and, more often than not, their reasoning for wanting to do so is right on the mark. But their reasoning sometimes doesn't matter.

The fact is that there are more factors than just quality that go into determining when to release an application to users. Quality is important but market pressures, competition, strength of user demand, staffing and personnel issues and many more nontesting issues must determine a suitable release date. As testers, we must simply get the most work done that we can in the amount of time allotted to us.

We should not complain about release dates. We should, instead, warn about consequences. That is the extent of our responsibility and it should be the extent of our concern.

9. Thou shalt covet thy developer's source code
I am not much of a believer in white box testing. I think

About the author

James Whittaker's picture James Whittaker

James A. Whittaker is is a technology executive with a career that spans academia, start-ups, and industry. He was an early thought leader in model-based testing where his Ph.D. dissertation became a standard reference on the subject. While a professor at the Florida Institute of Technology, James founded the world's largest academic software testing research center and helped make testing a degree track for undergraduates. He wrote How to Break Software, How to Break Software Security (with Hugh Thompson), and How to Break Web Software (with Mike Andrews). While at Microsoft, James transformed many of his testing ideas into tools and techniques for developers and testers, and wrote the book Exploratory Software Testing. For the past three years he worked for Google as an engineering director where he co-wrote How Google Tests Software (with Jason Arbon and Jeff Carollo). He's currently a development manager at Microsoft where he is busy re-inventing the web.

StickyMinds is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Apr 29
Apr 29
May 04
Jun 01