To Get Quality Software, Let Them Fail

Your job title doesn’t matter. Developer, analyst, manager, tester—at some point you decided to be an advocate for quality. You look at the product, take into account time, budget, and other business constraints, and recommend fixes to ship a product with the best possible quality.

You say all this to the customers, stakeholders, and decision-makers—and they don’t want to fix it.

Okay, maybe if you can’t log in, search, or pay, those bugs get fixed—unless they are in an odd browser variant that isn’t very popular. Maybe when people call in with problems, customer support will now be able to say that it’s “a known issue” instead of just looking confused. By and large, though, the good testing work you do doesn’t have much impact.

You find bugs, you write them up, the bugs get discussed, and nothing changes. For all the money invested in testing, the only outcome is a clogged tracking system.

What are we doing? Would we be better off conducting less testing and only reporting blocker issues?

Perhaps. Or we could talk about how to impact outcomes that are not within your control.

Power and Choices

Most people want to be powerful. At the very least, they want to be able to make their own choices. One way to demonstrate power is to disagree and win. The quality advocates do this by insisting a bug be fixed, and the people interested in the busy business of production do it by deciding to ignore the quality advocates.

The only way they can be powerful is by telling us no. The problem lies in how you framed the issue.

The stakeholders could be powerful by providing better software for the customer. They could be powerful by making a more scientific, long-term tradeoff on risk and reward. After all, five years from now, no one is going to remember that the software was released a day or five later; they will remember that it didn’t work.

Give your customer choices. While you’re at it, reduce their uncertainty. Framing the dilemma as go versus no-go leaves a lot of uncertainty around no-go. Provide a fixing plan, or even present two, that outlines the situation this way: If we fix these bugs, and retest this much, we can deliver in so long.

These aren’t my ideas. Dale Carnegie suggested that people want to be powerful (and they want to argue) in his book How to Win Friends and Influence People nearly a hundred years ago. Jerry Weinberg, the author of The Psychology of Computer Programming, suggested the rule of three: basically, If you haven’t thought of three possibilities, you haven’t thought enough.

With only one option, you are stuck. Two is a dilemma. But with three options before you, you really start to feel like you have a choice. Your customers will, too.

Justifications for Failure

I’ve framed these ideas as a “ship decision.” These days, many customers I work with want to deploy every day, or even several times a day. This also applies at the individual feature or story level.

This situation is actually better, because the pain of being “late” will be measured in hours, or a day or two at most. No product is going to fail and no one is going to go out of business if you wait a few hours to add better error-handling capabilities if the name fields are blank.

However, products do fail because of “broken windows.”

Broken windows go along with crime-ridden areas. But the broken windows theory goes a bit further: If a building or car window is broken and doesn’t get fixed, that actually encourages crime. After all, if the building or car is not being maintained, it looks like there will be no repercussions for damaging it.

Put differently, if a car is abandoned for weeks, it’s likely that no one would notice. However, once a car window is broken and not repaired, criminals might consider it fair game, going further and stealing the radio, taking the tires, defacing it, and so on. Lawful residents get fearful and withdraw from the area, which allows more serious crime to move in.

Another term for this is “normalization of deviance,” which is a commonly accepted cause of the Space Shuttle Challenger disaster. The space shuttle had so much redundancy that when something went wrong, it became routine to get a waiver. NASA officials got so used to these “deviant” practices that they became insensitive to them. Eventually, this happened so often that it became normal to break the rules. This continued and escalated until the consequences were tragic.

Normalization of deviance and broken windows are two common dysfunctional ways teams deal with software bugs. In the case of broken windows, once the software is buggy, what’s one more bug? In the case of normalization of deviance, the team may have a standard that any blockers and critical bugs make it impossible to ship, then they’ll simply argue that every bug is “major,” “will not be fixed,” “working as designed,” “a usability issue,” and so on.

Getting to Changed Outcomes

Software literature tends to prescribe two extremes for ensuring quality. The first is having a gatekeeper, which takes power away from everyone else. The second extreme is to collaborate on everything, which too often translates to “go along to get along.”

I am suggesting a third option: Educate the decision-maker. Make it clear to them that they are the one making the decisions, and meanwhile, provide all the relevant information and give them options.

Team with them about broken windows and normalization of deviance. Come up with products in the real world to use as examples—products that are good enough, highly reliable, or “you get what you pay for” (for instance, I once bought a pair of dollar-store sunglasses that broke before I left the parking lot).

Give your customer the thinking tools to make the right decision. Articulate the long-term consequences of the choices. Then step back and let them make the choice.

When They Choose Wrong

If the customer makes a choice that is wrong in your eyes, that is okay. They will have to live with that. You predicted what will happen, you explained the consequences of making that decision and made sure the customer understood the outcome, and the organization will get a chance to learn from its mistakes.

Now comes the hard part. You have to shut up, not say “I told you so,” and sit in meetings where your ideas are spouted off by other people as if they came up with them. The same people who told you “no thanks” before will be explaining broken windows to the vice president of IT and the CIO. They’ll be explaining why they need to change policy a bit, clean things up, slow down, and fix some bugs.

They might even listen to you this time.

Sometimes the only way to get things done is to give up the power.