How to Read a Program Dashboard Without Fooling Yourself
A program dashboard is written by people who are graded on what it says. Keep that in mind and you will read every one of them more accurately.
This is no accusation of dishonesty. Most program managers report what they believe. But belief bends under pressure, thresholds get set generously, and a status that would trigger an unpleasant meeting tends to get one more week to improve. Project people have a name for the result: the watermelon program. Green on the outside, red all the way through.
Executives get fooled by dashboards for a simple reason. They read them the way they were designed to be read, top to bottom, color first. Reading them well means looking at the three things below.
Takeaway 1: Read the movement, and ignore today's color
A single report is a snapshot, and a snapshot tells you nothing about direction. Ask for the last eight reports and lay them out in order. You are looking for three patterns.
Dates that drift a little each period. Risks that have held the same score for months. Milestones that were "at risk," then disappeared from the page without ever being marked complete.
Example: A milestone slips one week in each of six consecutive reports. It never turns red, because the reporting rule says a slip under two weeks stays yellow. Read one report at a time and the program looks slightly behind. Read six in a row and you see a milestone that has moved a month and a half with no recovery plan, on a program that has never once reported red.
If your team cannot produce the last eight reports in a comparable format, you have learned something about the reporting, and it is worth knowing.
Takeaway 2: Separate work done from results accepted
"Percent complete" is the most quoted number on any dashboard and the least reliable. In most cases it measures effort spent or tasks started. It says little about whether anything works.
The old joke in software is that a project spends the first 90 percent of its schedule getting to 90 percent complete, and the second 90 percent of its schedule finishing the rest. The joke survives because the measurement problem behind it has never gone away.
Replace percent complete with a count of things that are finished and accepted by someone other than the team that built them.
Example: A dashboard has shown "integration build: 90% complete" for two months. Ask for the count and the answer changes shape: twenty interfaces in scope, fourteen built, three tested end to end, none signed off by the business. The 90 percent figure was a fair description of how much coding effort had been spent. As a forecast of go-live it was off by months.
Apply the same test to money. Compare the share of budget spent with the share of deliverables accepted. A program that has burned 70 percent of its funding and delivered 40 percent of its scope has a problem that no status color on the page will show you, because budget and scope usually sit in separate boxes with separate, individually defensible ratings.
Takeaway 3: Look for what the dashboard leaves out
Every dashboard is a selection. Somebody chose which measures made the page, usually at kickoff, and usually the ones that were easy to collect. The items that sink programs are rarely among them.
Four things are almost always missing. Decisions waiting on someone and how long they have waited. Dependencies on teams outside the program. Changes in vendor staffing, especially senior people leaving. And the size of the change request queue, which is where scope disputes go to wait.
Then ask one question the page cannot answer. A good one: "What would have to go wrong for us to miss the date, and how likely is it?" Another: "What are you most worried about that isn't on here?" The quality of the answer tells you whether the person in front of you is managing the program or managing the report.
Example: A dashboard shows green across schedule, scope and budget. The sponsor asks what is not on the page. The program manager mentions, almost in passing, that the vendor's lead architect resigned two weeks ago and no replacement has been named. That one fact outweighs everything on the dashboard, and it had no box to live in.
A ten-minute reading routine
Before the next steering committee, do three things. Pull the previous two reports and mark everything that moved. Circle every percentage and ask what was counted to produce it. Write down one question about something the dashboard does not cover.
Ten minutes of preparation changes the meeting. The team learns that the report will be read closely, and the reporting starts to improve before you have asked for a single change.
A dashboard works like a gauge, and gauges can be miscalibrated. Treat a long run of green the way you would treat a fuel needle that has not moved in three hundred miles.