Deathray Research

Are story points about complexity or time? Mike (Agile Estimation) Cohn was explicit:

point-based estimating is about the time the work will take

In short, story points are a flimsy, undersized hospital gown draped over real, time-based estimates. As much as you want to hide them, they’re gonna show through.

Some processes associated with Agile (continuous integration, unit testing, short, functional delivery cycles, customer representation, etc.) help us build better software. But not story points.

Jeff Sutherland, founding father of Scrum, is a proponent of story points. Jeff argues that because one programmer does a task faster than another you can’t forecast based on hours completed. He seems not to recognize that the performance variation between hackers is the same whether you estimate in points or hours. The unit of measurement is irrelevant. 

He continues:

we estimate everything in points for the Product Owner so that he create a release roadmap based on team velocity and adjust the plan if velocity changes.

Or, you could estimate in days[1], calculate a velocity, and adjust the plan if the velocity changes. Again, points adds nothing.

In his defense of story points, Jeff sites research that showed:

Not knowing the velocity of production of the teams is the root cause of 100% failure of release plans to be accurate in their board meetings.

In other words, story points are good because velocity is good, — but velocity and story points are two different things. Using velocity for scheduling is a massive step forward. If you’re not doing it, start. Look closely at Jeff’s arguments and you see they hold if you substitute velocity for story-points. 

Beyond that, that statement ignores all sources of schedule error not covered by velocity, a constant error on the known tasks. When schedules go to hell, it’s usually because:

• tasks are added or requirements change - same velocity, different end date
• people leave the project (or get added) - velocity shifts because the project structure changed, or
• things seemed to be on track, but big hidden pile of bugs gets found, and panic ensues - measured progress (velocity) wasn’t real. 

These must be addressed by other means.

Castles in the air

Another common argument is that it’s better to estimate in points because you’re comparing one task to another: It’s twice as long, or whatever. This argument is typically illustrated by the Parable of The Two Runners, wherein we have two runners; one finishes a course in 5 minutes (“That’s a five-minute course”, she says), the other in ten (“That’s a ten minute course”). When they see a course that’s twice as long, they disagree about how long it will take, but they both agree that it’s twice as long. Happiness.

But they can agree because distance is measured in clear, universally understood units. Just like days. Everyone knows how big one is.

Measuring with points isn’t like using with meters instead of yards, it’s like everyone has their own “meter-long” rod they calibrate with, and they’re all different. Seriously, how big is a point? Each rod is based on some estimate of how long some task might take. (If we could do that reliably we wouldn’t be having this discussion.) On different projects you’re not even talking about the same task. 

Is my database task bigger than your two point design task? The only way to know is to convert each to common units (like hours) and then decide. Estimates don’t get more accurate by being more abstract. 

Another common argument is that It’s faster to estimate story points. Maybe, but I think part of the benefit comes from the sound practice of using an exponential (1, 2, 4, 8, etc.), or Fibonacci scale, instead of striving for pointless, unachievable accuracy (111.5 days).  This is another practice that should be widely emulated.

In any case, what percent of your project is given to estimation?  Don’t introduce a new, fuzzy unit of measure to optimize a small fraction of the plan. In his classic work The Diffusion of Innovations, Rogers identifies ‘compatibility’ with existing technologies and beliefs as one of five factors influencing the adoption rate for innovations. Calendars are a key technology. Introducing a new measurement system based on fuzzy concepts will not inspire confidence in your customers. 

Story points mean never having to say you’re sorry

But if people like Sutherland, Cohn and Martin Fowler all use story points they must be good for something. They’re good for this:

When a one story point task takes two days, no one says you were wrong. You didn’t say it was a one day task; you said it was a one story-point task. If you do your initial estimate in days, someone will inevitably come back to that first, raw estimate and try to force you to meet it. These people are everywhere. They learned everything they know about management from The Apprentice, and actually believe they’re “playing hardball” or “results oriented”, rather than “counter-productive”, or “stupid”.

I feel for you, but it’s not worth the trouble of introducing a new “scale”. The reason is that it’s a one-time fix. The first time you generate dates from your points, you have a baseline some savant-idiot from “the business” can club you with. 

Success in engineering management lies mostly in not doing stupid things to try to meet unachievable dates. That requires education, communication, political ability, and a skin like rhino hide. Metrics sleight-of-hand won’t cut it. 

Conclusion

Agile estimation provides two tools everyone should use: Velocity, and an exponential scale. Your estimates will always be wrong. If you calculate velocity based on real progress and use that to make predictions, they’ll be better. They’ll be about as good, in fact, as fallible humans can make them.

Footnotes:

[1] All references to days mean ideal-hacker days. Let velocity handle the conversion to calendar time as well as any estimation error.