I've long advocated test managers having a mission of assessing the state of the project and reporting on it. This assessment compares where the project
is to where it should be—not including audits of the project or the process. To be honest, I don't care whether the project manager or the test manager assesses the project on a regular basis—only that someone does it.
A project dashboard explains all the details about the project state . And, in most organizations, people need to understand what the data means. What better way to lift the fog of confusion than to present your information in a
concise way everyone will understand? And everyone knows weather. Weather reporting is great technique to explain the project state assessment.
Some may use the traffic light model--red, yellow, and green--to denote the project’s state. This model shows today’s state, but it’s hard to see where the project is headed. I haven't found the traffic light useful, due to the static nature of the assessment and the limit of three levels to denote project state. And unlike traffic lights that automatically change, projects don't change unless the project manager and the team act to change them. Projects tend to continue in the direction the team is heading.
Most senior managers want to see a weekly project status, but don't want to wade through a project dashboard. They prefer a quick status update that shows where the project is and where it’s headed.
That is an assessment, and it needs to help both you and your audience understand where the project is and where it’s going. A weather report gives you a quick perspective about the project—where the project is versus where it
should be—the project "weather." If the weather declines over time, and nothing is done to resolve the issue(s) one could predict that the project will continue to get worse (forecast future project weather).
Say you're on a project that has few risks and is proceeding on schedule. You'd give that project a full sun. But imagine your risk list is increasing daily, and you're not sure whether two features will be completed on time.
Although the schedule hasn't slipped, you are sure it will. Say it’s early in the project, and your test team can't run--never mind run successfully--the number of tests they thought they could. You'd probably give that project a
partly cloudy designation. After several weeks of partly cloudy, you might move to cloudy. And if the project risk list is increasing, the developers are spinning their wheels, and you're finding more and more defects, you might
select rain.
The weather report model uses assessment to predict project progress, as if the project has a season. The prediction arises from our experience with the weather; seasonal weather doesn't change much day by day. Even if there are days with rain, snow, or abnormally high or low temperatures, the weather generally continues on as expected for the season.
A project progresses in the same way. You may encounter a problem you can fix on a project, but if you encounter problem after problem, you're not going to stay with your original assessment. As weather icons change (or remain the
same), readers will be more aware of the project’s status and may want to understand the dashboard data in more depth.
|
Status |
Description |
|
Sunny |
The project
schedule is on target. |
|
Mostly Sunny |
There is minor project schedule concern, but the schedule can be met. |
|
Partly Cloudy |
There is schedule concern; the schedule can be met by putting extra efforts. |
|
Cloudy |
There will
be difficulty meeting the schedule. |
|
Rainy |
To meet the schedule would take great difficulty. |
|
Severe |
The schedule cannot be met in any situation. |
Figure 1: A sample list of criteria for a weather report. Provided by Rodney Thompson.
Credible Weather Report
Weather reports models, like actual weather forecasts, can lose credibility if they change dramatically from week to week unless something dramatic has changed for the project. Problems that could change a weather report in one week
include losing a significant percentage of people to other work, a missed vendor deadline, or realizing late in the project that the architecture won't support the planned feature set.
Another way to hurt credibility is to use less-than-professional weather icons. In the same way that your project dashboard needs to be clear to your readers, the weather report icons need to add to your credibility, not diminish it.
From Project Data to a Weather Report
If you're already gathering a variety of project data--schedule data, velocity charts, defect trends, test coverage, people assignments, and risk list--then the weather report is your best assessment of the overall picture. If you're not collecting that data, resist the temptation to use a gut feeling for the weather report. Instead use progress toward release criteria for the weather report.
Frequency of Weather Reporting
The goal of the weather report is to help people understand the project assessment and avoid surprises. Projects with more than two months left should have a weekly weather report. At some point--certainly by the project’s final month--or near major milestones, you may need a weather report a couple of times a week.
Because managers are busy and don't always have time for the details, consider weather reports to communicate regular project assessments. Select your weather icons based on analytical data and make the criteria for the icon clear.
Acknowledgements
I thank Rodney Thompson and Esther Derby for their reviews of this column,
and Thompson for graciously providing his criteria for each of his weather
report icons (see table above).
Further Reading
"What Do They Pay You to Do?" STQE, September/October 2001.
"Are We There Yet?" Better Software magazine, January 2006.
Check out a variety of weather icons online at:
http://www.jdawiseman.com/papers/trivia/weather-icons.html.
