Some years ago I worked with an airline that was writing some new booking pages for its website. Following good practice, the airline’s team tested the pages with sample users to see what they thought of the new design. On the whole the feedback was good, except one thing: The sample group of customers said the new pages made it harder to find the lowest price for a flight. The airline decided to go ahead with the new content anyway.

When the pages went live, naturally, the airline’s revenue went up. Customers were spending more money, but they might have been upset about that fact.

Ideally, I’d like to see companies put a dollar amount on each planned business decision, but to be fair, pinning down the financial value can be hard—especially in a corporate IT setting. And as this airline example highlights, it is not just a question of how much value is anticipated, but also how sustainable it is. One could argue that while the airline increased revenue in the short term, in time, customers would start to consider their flights more expensive and take their business elsewhere.

I want to look at ways to assess business value and some of the considerations to think about.

Calculating Business Value

The obvious way to put a business value on an agile user story is to consider what difference it will make and what financial benefit that will bring. Typically, we would expect new technology to either increase revenue or reduce costs.

Let’s consider the following story:

As a widget seller, I would like to sell my widgets online.

That might look easy, but some analysis is needed. For a start, how many widgets might we expect to sell online? That requires us to ask, do people buy widgets online? And if the answer is no, we need to ask, would people buy widgets online?

Once we get through these questions, suppose we determine that a hundred thousand people might buy widgets online. Then we need to ask what the selling price is likely to be. We might know what a widget sells for in a shop, but would people expect to pay less online?

When we eventually reach a figure, that breeds more questions, such as: If we sell widgets online, will we lose offline sales? This known as cannibalization: when one product or channel takes sales from another of your products or channels.

Answering these questions may well involve some guesswork, but herein lies another problem: Guesswork is open to questioning. If someone doesn’t like your answer, they can attack the guesses you made to get to it.

The more background research and analysis you do, the less guesswork will be required, but eliminating guesses actually might not make calculations more accurate. In some cases, intuition and rough rules of thumb can be more accurate than complex formula and data. (Gerd Gigerenzer’s book Risk Savvy contains many such examples.)

If you get lucky, then as you do the analysis, you’ll discover some driving forces that render all the others insignificant. For example, if in analyzing online and offline widget sales you were to find that over the last five years online sales were rocketing while offline sales were plummeting, then it becomes clear the future is selling widgets online.

Assessing Cost Savings

Making a rational assessment of the cost savings in an organization should follow a similar structure to the revenue example:

As a supermarket manager, I want self-checkout lines so that I can reduce the number of cashiers.

Faced with such a story, we need to ask, how many customers will use the self-checkout? And how many cashiers might that replace?

One should also consider whether self-checkout might lead to increased theft or reduced opportunities to upsell additional products.

Some of the same catches apply, generating the need for more guesses or further research. However, in situations like this, additional questions also come into play: Will the customer experience be better or worse? Such a qualitative change is even more difficult to value. Creating a worse shopping experience may not immediately affect the bottom line, but over time it can make a big difference.

A second question to ask is, how quickly will these changes happen? It might take years before customers are happy to use the self-checkout regularly and the staff can be reduced. Researchers have found cases where IT changes took decades to realize cost savings.

Time as a Business Factor

Time plays a still more insidious role in these calculations: The more analysis and research you do in order to tighten up your business case, the longer it takes, and time is money.

Initially, analysis and research costs money because someone gets paid to do it. It may also entail extra costs such as accessing research reports. But it also costs because such analysis and research delays development and release of a product into the market. Delaying product release means the benefits are delayed, too. Delay also leaves opportunities for competitors to release a product.

Back when new IT systems took months or years to develop—think about the days of COBOL and C programs—it made sense to do lots of upfront analysis. However, in a world of tools and frameworks with far higher productivity, such as Ruby on Rails and Amazon Web Services, it might be possible to develop a product, deploy it, and analyze the results in less time than it would take to do a detailed analysis.

The Importance of a Company Strategy

Answering some of the questions outlined here requires referencing your company’s overall strategy, and perhaps the specific IT strategy. This provides one shortcut to evaluating user story requests: Does a story align with the business and technology strategy of the company? If not, you’ve saved yourself some analysis time.

However, this requires that companies have a strategy and that it is clearly communicated. Let me leave you with a story with three possible endings:

As a taxi company, I want customers to be able to book a taxi using a phone app so that …
a) I can reduce the number of people employed answering phones to save money.
b) I can take more bookings for more taxis and generate more revenue.
c) I can disrupt the business model of taxi companies and build a single global company.

Valuation of a story is going to be vastly different depending on which ending your story has, and the ending depends on your company strategy.

When presented with a new user story, these are all some major factors that should be considered when evaluating the important question of business value.