When ambiguity rears its nebulous head, how can we move our projects forward with certainty? According to Harry Robinson, one of the most useful things a tester can do is ask good questions early in the software development process to help the rest of the team members to think clearly about what they are doing. In this column, Harry offers us some weapons to defend ourselves against the misunderstandings, bugs, and rework that often result from ambiguities in the development process.
A few years back, there was a Dilbert cartoon where Dogbert would wave a wand and proclaim "Out Out!! You demons of stupidity!!" I don't typically have many problems with stupidity—the people I work with tend to be on the bright side—but I would love to have a wand like that to dispel ambiguities, because ambiguities plague my life. And they probably plague yours, too.
What Are We Talking About?
The main street in my town has a posted speed limit of 25 miles per hour. Near the local elementary school, however, there is a sign that says
Speed Limit 20 mph
When Children are Present
The intent of the school zone is obviously to keep kids safe, but I find the sign ambiguous. Like almost everyone else in this situation, I simply drive slowly as I get near the school, but if I had to explain the school zone rules effectively to a computer, I would have to understand the rules much better.
- Why does the sign use the plural "children" instead of the singular "child"? Does the rule only apply if more than one child is present?
- Who qualifies as a "child"? What would happen if I drove past a 17-year-old at 25 mph?
- What does it mean to be "present"? A child walking along the sidewalk is certainly present, but what if the child was behind a fence?
- Does this rule apply only when school is in session? What about weekends? Holidays? Nighttime?
- And what if these situations are combined: two 17-year-olds behind a fence in a school zone on a summer weekend night? Well, maybe that's a whole different story ...
I could, of course, drive back and forth to test out each of these situations, but it would be a lot cheaper to find out the answers without having the police pull me over.
Have You Helped Someone to Think Clearly Today?
One of the most useful things a tester can do is ask good questions early in the software development process to help the rest of the team members to think clearly about what they're doing.
I recently helped someone design a utility to start and stop a service on a remote machine. Even this simple utility yielded several good questions:
- What happens if you send a "start" request to a service that is already running, or a "stop" request to a service that is not running?
- What if the remote machine reboots - does a running service automatically restart?
- What happens if the connection drops when you are sending your request? Does the utility return an error message, or does it retry? How many times does it retry?
Asking these questions up front saved us from misunderstandings and bugs later in the process that could have required substantial rework.
Driving Out Your Own Demons
Here are some simple heuristics to help you drive out the ambiguities you encounter.
- Dissect the description. Are the words open to different interpretations? The book Exploring Requirements by Weinberg and Gause uses this approach to twist the phrase Mary had a little lamb into everything from Mary ate a small portion of mutton to Mary conned the naïve stock trader. The authors state that "although the nursery rhyme example may seem silly, we've used this heuristic for years in reviewing requirements documents, and it has often saved vast sums of money." (p.95)
- Poke at the borders. Terms often map imperfectly onto each other, so find the fuzzy areas. For instance, in the remote service example, how will we check that a service is running? Are we checking a flag somewhere? Could that flag be improperly set to indicate that the service is running when it really isn't?
- Walk through an example. Scenarios are very powerful tools because they present a concrete situation rather than vague abstractions. Testers particularly shine in thinking up error scenarios. What if the connection drops? What if the customer enters something incorrectly? What if the disk is full? What happens if I pull out this cable?
- Get others involved. You should be not only a source of good questions, but an inspiration for good questions from others. Developers, for instance, have a different perspective and often think of questions testers might not. Once you begin to ask the "what happens if" questions, others will find themselves joining in.