Software Project Estimation

Using Uncertainty Factors

Tired of guesstimating your estimation process just to create a completion date management will accept? Jonathan Kohl takes the guess work out of estimations by focusing on uncertainties. It may sound counterintuitive, but the idea is to focus on the fact that all projects face unforeseen delays. The rigorous estimation process Jonathan describes here provides your team a way that ensures enough time is scheduled for development and a date for completion management can agree upon.

Steve McConnell's Software Estimation: Demystifying the Black Art is a fabulous resource for software projects. If you haven't read it, pick up a copy and work your way through it. It is full of great ideas, interesting concepts, and some tips. The topic of software estimation is large, so in this article I'm going to focus on one of McConnell's topics: using uncertainty factors in estimation work.

McConnell writes, "Accurate software estimates acknowledge that software projects are assailed by uncertainty from all quarters." [1] Most software development professionals I've talked to agree with this statement, and there are plenty of estimation tools available with this concept built in. However, I rarely see uncertainty factors applied to project estimation. Management feels the technical staff is uncooperative and inflates estimates, and techies feel that management doesn't have a true appreciation for how much work is required. Using a bit more rigor in our estimation process can make estimation efforts visible, defensible, and more accurate. (Also see Joel Spolsky's feature article Beat the Odds in the March 2007 edition of Better Software magazine. [2])

Example: Scrum Team
A Scrum team was struggling with their estimation process. They'd tried different techniques, including using story points instead of time blocks, but in the end, a senior manager would get a date in mind and the estimates would have to be massaged to meet that date. The fact that they never met that date was a cause for concern.

To combat the confusion, I proposed a new methodology. I would use an estimate-uncertainty factor and Monte Carlo simulations (statistical methods of generating possible future outcomes, used in scientific studies and other fields [3]) to come up with a target release date range which we would present to management for further discussion. Each item in the estimate date range would have an associated probability of completion attached to it. Early items start out with a zero percent probability, and dates further in the future start to move slowly towards a one hundred percent probability.

The uncertainty factor is a number that is associated to a raw estimate (estimate team members provide for a task) and used in the Monte Carlo simulations to generate possible outcomes. It maps to our confidence in the estimate value the team comes up with. In our case, if we had little confidence in the estimate, it was considered a high risk estimate and had a larger uncertainty factor assigned to it. If we had a lot of confidence in the estimate, it was deemed a low risk estimate, with a smaller uncertainty factor.

Prior to meeting as a group to generate raw estimates, I met with each individual team member to determine if they added time to their own raw estimates. For example, when determining a raw estimate of effort, did they secretly pad that estimate by doubling or tripling the time? People often learn to add uncertainty factors without telling anyone. This can distort estimation efforts, particularly if more than one person on a team does this. I told them in our estimation brainstorms to just provide raw estimates but to communicate them in terms of probability or confidence. Are they highly confident in their estimate (a low risk), somewhat confident (medium risk), or not that confident (high risk)? In statistical terms, we could express them as "most likely" (the mode), "average" (the mean), or a "50/50 chance" (the median). The tool uses our estimate, our assigned uncertainty factor, and a random number generator to simulate probable futures.

User Comments

1 comment
Doug Shelton's picture

Great article, Jonathan.  

I think this area is continually overlooked by agile purists who maintain that "knowledge work [i.e., SW development] cannot be estimated".  Of course it can indeed be estimated, just as laying bricks for a wall can be estimated.  It is purely a matter of to what degree of accuracy - and precision - it can be estimated [i.e., typically much less well than laying the bricks :-].  

February 6, 2014 - 5:29pm

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.