Stories, Epics, and Tasks: Organizing Agile Requirements
Making user stories small is hard. I recently ran a story-writing session with a team and their clients. We brainstormed a dozen stories and then prioritized them. We took the top one and talked about how we could make it smaller and simpler. I knew little about the system and the background, so I asked some obvious questions and challenged everything. I bet I was a pain.
The first cut started with an and. The card said “real time and historical reports.” Whenever you see and, or, either, or but, look to split the card.
Fortunately, everyone in this meeting was open to splitting the stories. After awhile we had a multitude of stories derived from the first one.
Looking beyond Stories
Some teams only work with stories. Their stories make sense to the business and to the technical team, are deliverable quickly—a few days at most—and have value. This is probably the ideal situation, but it doesn’t work every time, and it can be difficult when a team is new to agile.
An alternative is to add epics and tasks. An epic is some big piece of functionality the business wants that is delivered via multiple smaller stories. Epics, by definition, break the rule that stories must be small, but they have the most business benefit. Tasks are smaller work items that build a story. They usually can be accomplished in a day or so, but by themselves they are devoid of business benefit.
Go Large with Epics
Sometimes, the business doesn’t want something small; it wants something big! Indeed, it might want more than one big thing. An epic might represent a milestone or notable delivery, or an epic can be a placeholder, something to break down into smaller pieces in future.
It makes sense to use the user story “As a …” format for epics, but there is no law that says you must. Just remember that who, what, and why are useful things to know.
Go Small with Tasks
On the other hand, it sometimes makes sense to break a story down into the work that needs to be done. In calling out the tasks needed to build a story, the development team engages in an act of shared design. Tasks are not normally written in user story format. Instead, they are written by the team, for the team, so use language the team will understand.
A task is a piece of work that needs doing, usually in order to build toward a bigger story. As such, it does not have independent deliverable functionality or generate business value, and, unlike a story, it normally is not a vertical (end-to-end) slice. Most tasks tend to be for programmers, but they could be for anyone on the team.
Not all stories get broken down into tasks. Breaking a story down is an act of design, so whether a breakdown is required or not depends on existing knowledge, the technologies under development, the architecture, and, most of all, the size of the story to start with.
Tasks are useful because they allow the technical team to call out discrete pieces of work without pretending they mean something to the business. Having the whole team examine a story, especially when that brings different points of view, opens up the story. Different team members see different ways of breaking a story down, either as tasks or as smaller stories.
Three Organizational Levels
When I used to run teams, I found that two levels were enough: stories and tasks. However, I regularly meet teams who use epics and stories and seldom, if ever, do tasks. Just because there are multiple organizational levels, don’t feel you have to use all three. If you only need two, good, and if you only need one, then better still.
Avoid the temptation to introduce more levels. Adding levels increases administration and gets more confusing. When you are using an electronic tool, hierarchies can get complicated quickly, and extra complications distract from the real goal. So avoid tertiary epics, intermediate stories, subtasks, and the other names given to additional layers. Whenever I see teams who have added to the hierarchy, the effort and complications introduced are greater than the benefit. If you feel the need to group several stories, remember that you can do this without a hierarchy. Paper clips work well!
That being said, epics and tasks sometimes go by other names, especially natively within tools. Tasks can be called subtasks, stories sometimes go by features, and epics can go by themes. Some teams like to organize their work around user activities, which may be stories, epics, or something else.
If your team, organization, or tool wants to use different language, that’s fine; names are not important. Just ensure that your entire teams understand what is meant by each term, use the terms consistently, and stick to three or fewer levels.
Color Coding and Planning
I like to code tasks by putting them on white cards. That makes obvious what they are. I use blue cards for stories—you might say “Blue is for business.” I’ll also use blue for epics sometimes, but epics seldom make it to the board. Epics are big things, and for a team working in the small, the focus should be on getting the small things done.
These three levels line up well with planning: White tasks are for now and the iteration that is about to happen, and they are mainly for the team. Blue stories are what the product owner is lining up for the near-term iterations, and epics are for the future. Epics touch on strategy and may even be beyond the product owner.
Making Use of Your Choices
Being able to choose among stories, epics, and tasks brings flexibility to an agile team, but don’t think you must use epics and tasks. Sometimes when stories are decomposed into multiple smaller stories, each stands on its own and no hierarchy is needed. When a story is really small, knowing what needs doing can be trivial. Having a clear understanding of the differences between each level and knowing what size story to use for each situation will improve the accuracy of your sprint planning.
User Comments
The biggest challenge we face is that we typically need at least three things to be done: database, framework, front end, more and more also API changes to go with it. Often times the schema changes impact other parts that require code changes. We do have a lot of "ands" in the stories, but they are there because we cannot do only one part and not do the other at the same time without breaking the entire application.
One approach might be using shelf sets in source control. The development work is done, but it is not in the integration branch. That also means it cannot be tested and thus that story cannot be called done.
Is there any practical advice on how to split that up?
Tim, this is exactly the kind of case were I suggest you use tasks.
You have a story, you need some UI, some DB and something in the middle. That immediately sounds like 3 or more Tasks; the story is what the business want, the tasks are what you need to do to build it. If that means db, ui, etc. then each is a task.
The next question - one you hint at - is how do you get all this work into the system in pieces without breaking whats already there while when yo haven't finished the next task? e.g. you've changes the db schema but haven't changed the code which queries it.
Well...
- if yoi are using git you could just check the changes in locally and not push to the trunk until you have all the tasks done
- if you have an older source code system you could use feature toggles to switch bits of the code off until the whole feature is in place (and later remove the toggles.) However if your design isn't already running with feature toggles that could be a challenge, once you start using them your design will adjust
- or you could use a branch. I know many people advise against branching - and I do myself! - I do think there is a big different between long lived branches and short lived branches. If your branch only lasts for a few days then it doesn't store up big problems. Its not that different from local repositories
The other thing to think about is: how can you make the original story itself smaller? - reduce the work
Hope that helps
In this example how do you suggest to estimate the story (in hours or story points) when the story has different dev tasks such as UI work, DB work, coding... It’s a challenging to estimate when tasks cover different roles and type of work.
Option #1: Don't estimate. Seriously, if its so challenging no estimate is going to be accurate so why bother?
Option #2: Estimate
Never, ever, estimate in hours. You can't. Pshycologists have shown this time and time again in the lab. Human's can't estimate time. Of course people do estimate i hours and then wonder why they are so far out.
You can estimate in effort points - relative sizing - and if you track your past performance you can get a pretty good idea of your capacity.
Pshycological studies also show we are actually better at estimating how long it will take someone else to do a given task than they are at estimating how long they will take to do the same task.
Such studies also show that people in authority and with expert knowledge underestimate by a greater degree.
Plus, "wisdom of crowds" logic shows that you are better off with multiple independent estimates than one expert estimate.
Therefore: ignore all your role differences, ignore the type of work problem, involve the whole team in estimating the work. But do not assign the work to anyone until the last possible moment, i.e. when the task needs doing who ever is available and feels they have the confidence to do it picks it up.
I say a lot more about the estimation process in Xanpan.
At the end of each sprint count the points you did, draw these on a graph. After a few weeks you will have an idea of your capacity.
It's important to recognise, as you do implicitly in responses to comments, that these "levels" are not levels of the same type of thing. The Epics and Stories are about what needs to be delivered in terms of capability. In traditional terms, they represent requirements. The tasks are units of work. If you are building non-trivial systems, it is quite likely that your tasks will need to be decomposed, but that's OK. A developer takes on a task as a unit of work, and realises that to implement it they need to do some design, understand interfaces to other parts of the system, do some coding, do some testing, etc. There is no reason why they should not, for their own convenience, break these down into sub-tasks. But that is the point, it is for their convenience.
Spot on Keith,
I'm not really keen on sub-tasks, two reasons really.
Firstly each time you add an extra level to the heirarchy (Epics->Stories->Tasks->Subtasks?) you make administration more complex. In my experience three levels can model just about anthing and in fact most teams of my experience only use two of the three (Epics & Stories or Stories & Tasks).
Second there is nothing to stop you breaking tasks down into more tasks: e.g. Task "buy milk" can be broken down into Walk to the shop, buy milk, walk home without the heirachical element.
Just because you can form a heirarchy doesn't mean it it necessary.
Its call Agile for a reason. Attempting to catpure every development circumtance into a rigid set of defnitions is futile and forces development teams away from the spirit of the agile process and sprints in order to "wing it" to get the application architecture developed prior to the bells and whistles of the user stories. These are stories and tasks. Tim's position is the perfect example. How can you build a compotentized, pattern based application on the strength of user stories? I don't see how you can. The same way you can't build a room without a foundation, and frame, and internal wiring, none of which the homeowner has any clue when they asked for an open concept with a mudroom. So while user stories from the business owner tell the team what the user want, user stories from the perspective of the product itself born of the knowledge of the development team, tell what the application need to apply those stories. Thus, there may be iterations that are ripe with tasks created by the team and not the business owner crucial for the business owners requirements. That demo may encompass one feature or even just a part of the UX...
We currently have Epics, that are broken down into Features, which are broken down into Stories, which then have tasks in order to complete those Stories. I find that the Features have very little value, and as long as the Epics have enough detail then we can go directly to the stories.
I can't find any documentation showing to use the Epic, Feature, Story, and task model. Have I been trained on doing something that isn't necessary?
Dump the features.
Every attempt I've seen to add an fourth layer only adds complexity. And as you say, where is the value?
In my mind 3 is enough to model just about everything: Normal size, bigger (to be broken down) and smaller (which build the thing.)
And many teams seem to manage just fine with 2, either Epics and Stories or Stories and Tasks.
If we just had an Epic that was written by the Program or Portfolio Team, then we could just have the Scrum Teams breakdown that Epic into stories, and their prioritization would naturally build 'Features' as they developed the MVP?
Pages