On the dark art of software estimation

“How long will it take?” demand managers, clients, and executives. “It takes as long as it takes,” retort irritated engineers. They counter: “Give us an estimate!” And the engineers gather their wits, call upon their experience, contemplate the entrails of farm animals, throw darts at a board adorned with client/manager/executive faces, and return–a random number. Or so it often seems.

It is well accepted that software estimates are frequently wrong, and all too often wildly wrong. There are many reasons for this. I am very fond of this analogy by Michael Wolfe on Quora:

Let’s take a hike on the coast from San Francisco to Los Angeles to visit our friends in Newport Beach. I’ll whip out my map and draw our route down the coast … The line is about 400 miles long; we can walk 4 miles per hour for 10 hours per day, so we’ll be there in 10 days. We call our friends and book dinner for next Sunday night, when we will roll in triumphantly at 6 p.m.

They can’t wait! We get up early the next day giddy with the excitement of fresh adventure … Wow, there are a million little twists and turns on this coast. A 40-mile day will barely get us past Half Moon Bay. This trip is at least 500, not 400 miles …

Writing software is rarely a matter of doing something you already know exactly how to do. More often, it involves figuring out how to use your available tools to do something new. If you already knew exactly how long that would take, it wouldn’t be new. Hence “it takes as long as it takes.” Non-developers often seem to think that we engineers just look at a proposed task and think “we shall implement A, B, and C, then do X, Y, and Z, and that should require N hours, plus or minus a couple!” Sometimes it is like that. But not often.

More typically, the thought process is more like: “I can see how I’d do it if I were rewriting that whole controller from scratch, but that would take days … is there an elegant hack where I can change the inputs to this function in such a way that I don’t have to rewrite its code? … what if I monkeypatch it at the class level? … wait, maybe there’s an API call that almost does what I want, then I can tweak the results — hang on, what if I outsource it via an asynchronous call to the external OS?”

In which case, the result is: “I can confidently estimate that this will require less than two hours of typing. However, working out what to type is going to take me/us anywhere from one hour to several days. Sorry.”

Another analogy, from my sideline in novel-writing: publishers always want a synopsis of your unwritten novel. This is insane, because, as an extremely successful author friend puts it, “writing a synopsis requires everything that actually writing the novel requires.” It’s like asking for a map of what is by definition terra incognito. So it often is with software estimates.

That said: if you’ve ventured into a lot of terra incognito in the past, and you’ve heard legends and tales of this new territory, and how similar it is to your past ventures, you can at least offer up an educated guess. (People seem to like the word “estimate” better than “educated guess.”) There are better and worse ways to estimate, and to structure estimation, and you can’t give up on the task just because software is hard.

What’s more, the form in which an estimate is requested is crucially important — and that’s often what managers, clients, and executives get so terribly wrong. More often than not, when a low-ball estimate balloons into a terrible reality, it’s their fault, not that of the engineers.

First, some managers/clients/executives (hereafter referred to as MCEs) still think that task performance can be measured in fungible engineer-hours. Not so. It was never so. We’ve known for decades that, for instance, “adding engineers to a late software product makes it later.”

Second, some MCEs tend to think that they can increase the accuracy of estimation by adding precision and granularity — by breaking down large projects into as many small tasks as possible, maybe even hundreds of them, and asking for individual estimates of each small task. This is a catastrophically terrible idea. Increasing precision does not increase estimate accuracy. In fact it does the exact opposite.

Let me explain. Software estimates are always wrong, because the tasks being estimated are always, to some extent, terra incognito, new and unknown. However, sometimes the errors are in your favor; an obscure API, a third-party library, or an elegant hack condenses what you expected to be a week’s worth of work into a single day or less. Your hope, when estimating a project, is not that all of your estimates will be 100% dead accurate. That never happens. Your hope is that your overestimates and underestimates mostly offset each other, so that your total estimate is roughly correct.

But the more precise and granular you get, the more likely you are to reduce your overestimates — which is disastrous for your overall accuracy. What’s more, the estimation of a hundred-plus features quickly leads to “estimation fatigue.” Estimation is important. It is the basis for your schedule for the next weeks or months. So, like all important things, you want to minimize, not maximize, its cognitive load.

Furthermore, a feature is not the work that goes into it, and not the unit that should be estimated. What engineers actually estimate is the size of the technical foundation building block that makes a feature possible, which is often/always shared among other features at that scale. It is very difficult — and doesn’t really make sense — to try and work out exactly how much of the work going into such a building block will be apportioned to individual features.

This is all pretty abstract. Consider a concrete example. Suppose you’re building an app that logs in to a web service. Don’t have individual server-side estimates for “user can create account,” “account email address can be confirmed,” “user can log in,” “user can sign out,” and “user can reset password.” Have a single “user authentication” task, and estimate that.

In hours? …Maybe, maybe not. I’m fond of using “T-shirt sizes” instead: S (an hour or three), M (a day or two), and L (a week or so.) Each project has its own pace, depending on tools, technical debt, the developers working on it, etc., so after a week or few you should be able to start figuring out how your hours map to your T-shirt sizes, rather than pretending to know that a priori.

T-shirt sizing is especially good because it comes with built-in warning signs. If you find yourself with all Ls and some XLs, it’s a very visible sign that you probably do need to deconstruct your project a little further; if you have mostly Ss, it means you’ve fallen into the granularity trap and need to abstract your tasks out a little. If you have a roughly even mix of S, M, and L, you’ve probably structured things so that you’ll have pretty good — well, least bad — estimates.

Which are, to belabor a point that needs belaboring, only estimates. The sad ultimate truth is that it takes as long as it takes, and sometimes the only way to find out how long that is is to actually do it. MCEs hate this, because they value predictability over almost all else; thus, they nearly always, on some level, treat estimates as commitments. This is the most disastrous mistake yet because it incentivizes developers to lie to them, in the form of padding their estimates — which in turn inevitably slows down progress far more than a few erroneous task estimates might.

So: accept that estimates are always wrong, and your hope/goal/aim is not for them to be correct, but for their errors to cancel out; estimate by feature group, not by feature; and never even seem like you treat estimates as commitments. Do all these things, and while estimation will always remain a gauntlet, it will at least cease to be the lethal minefield it still is for so many.