Measuring Up: How to Preview User Satisfaction before Your Release


Why wait to discover how your users will react to your system when there are ways to measure such things during development? This column describes a simple tool to develop visibility into customer satisfaction. Learn how you can begin to manage expectations so that neither you nor the customer has an unpleasant surprise on release day.

It's 10 a.m. You're about to ship to five beta sites. You've met the date, you're within budget, and the defect counts have been steadily declining for the last four weeks. Still, you're a little nervous. How will the customers react to this new release? 

Carrie was unhappy to learn during beta testing that the product her team had been building for insurance underwriting reps had drifted from what the users had wanted.

"It's too slow," the beta test group said. "It takes forever to pull up the records when I have a customer on the phone!"

"But in test it performed within the parameters in the requirements," Carrie protested. "What changed?"

Just a small design tradeoff that affected the default display order of the records. It had seemed like a reasonable decision at the time.

If you're a project manager, you know how important it is to have visibility into the product and the project. It's bad news to find out in a beta test that you have built a product that doesn't meet your customer's expectations. How can you tell if you're still on track to meet customer expectations?

Agile methods are one option. The agile school advocates close customer contact and frequent deliverables that are directly tied to business value. Still, many projects follow more traditional models: the customers (or their surrogates) are involved up front in defining features and then come back in during testing. A lot can happen during the time in between. If you're not using an agile method and don't have the luxury of having a customer assigned full-time to your team, how can you stay in touch with what the customer expects? One option is to measure customer satisfaction as you design and build the product.

Take a page from marketing and use a survey to catch a glimpse of customer reactions before the product ships. During development, ask the people who are closest to the product—sponsors, designers, and customers (or surrogates)—how they feel about the project.

You probably identified a set of system attributes during initial project definition. Suppose your customers identified these four qualities that were most important to them: easy to use, fast, always available, and secure. Of course, these descriptions are too vague to be helpful in actually building a system. You probably negotiated more specific targets for the product as a whole and for specific functions within the product. But for the purpose of a satisfaction survey, these more general attributes are about the right level.

For each attribute, create a bipolar scale, with the desired attribute on one end of the scale and its opposite at the other. For example: the opposite of "fast" is "slow"; the opposite of "easy to use" is "difficult to use."

About the author

Esther Derby's picture Esther Derby

A regular and Better Software magazine contributor, Esther Derby is one of the rare breed of consultants who blends the technical issues and managerial issues with the people-side issues. She is well known for helping teams grow to new levels of productivity. Project retrospectives and project assessments are two of Esther's key practices that serve as effective tools to start a team's transformation. Recognized as one of the world's leaders in retrospective facilitation, she often receives requests asking her to work with struggling teams. Esther is one of the founders of the AYE Conference. She co-author of Agile Retrospectives: Making Good Teams Great. She has presented at STAREAST, STARWEST and the Better Software Conference & EXPO. You can read more of Esther's musings on the wonderful world of software at and on her weblog at Her email is

StickyMinds is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, is the place to go for what is happening in software development and delivery.  Join the conversation now!