Everyone working is Software industry have been involved in estimates in a way or another. Most of the people have known for ages that the one thing that's sure for estimates is that those are wrong. More complex the software or organization creating the software, more difficult it has been to estimate how long creating software takes.

There's been buzzing around for some time the #noestimates movement. The basics of it, as I understand, is that software teams should get rid of making estimates, since those do more harm than good. But I think there goes so much discussion under #noestimates, that I'm not fully sure what all it contains.

I've seen estimates to do lot of harm in many organizations. Estimates have caused poor quality, unhappy people and lot of unnecessary waste on creating those and reacting to those. But then on the other hand, I couldn't see organizations to live without any estimates on software.

Value of estimating versus the time used to it

Value of the estimates versus the time used on creating the estimates is one of the key questions on estimating. Too often estimating takes lot of time. People use hours or easily even days to figure out how much time creating some software takes.

I guess it's not valuable to discuss too much about what's wrong with estimating. People are known to be terrible at estimating. One excellent source to understand more is Nobel prize winner Daniel Kahneman's Thinking, Fast and Slow which introduces at least WYSIATI, anchoring effect and lots of other reasons we can't estimate. So I'm not going to use more time on arguing about the subject, but use is as a given fact that people are bad at estimating.

So what is the reason estimates still exist. Why estimates are still done, even though those are always thought to be wrong. I believe estimating goes to the same category as planning. As Eisenhower said: "Plans are worthless, but planning is everything", there's something similar with estimates. Estimates as such are not accurate, but the journey is valuable and some figures are always needed.

Sometimes the time spent doing the estimates doesn't match the value that estimates can give. Estimates should be used as a tools for decision making. Most often the decision is that should something be tried to do or not. The decision should be about the start, not about the completion. That's what Agile and iterative thinking should help on, there should come multiple decision making points and clarifications about the completion when knowledge increases during the development.

Companies can't live without estimates

Liked it or not, companies can't live without estimates. Most of the companies have limited resources in developing software (and everything else too). Those few who currently seem to have unlimited resources, still need to serve their customers quite often to keep their positions.

In real life, there are customer commitments and internal commitments that need to be handled somehow. Most of these are based on estimates, things that are most probably not accurate. Still those don't change the fact that sometimes these commitments need to be given. Think about yourself, would you be willing to purchase a house building project, if your contractor would say that we don't really know how much it will cost, when it will be ready and what it will contain. Of course you want to get an estimate, or even a fixed price with fixed timeline and features.

Many companies actually are in estimating business. They estimate the costs of development, they estimate their sales, they estimate the timelines and then take known risk with price and profit margin. This is how most of businesses still work. The luxury of making fixed profit is with fixed hour rate businesses. So basically some of consultants, lawyers and couple of others. There the risk is in not getting clients and not in making bad estimates.

Other than actual businesses relying on estimates, many companies have internal dependencies to SW development estimates. Product marketing for example, can't wait for product to be finished before marketing activities can start. They might need easily few months head start to get all the activities ready when product would be ready to ship. There easily are tens of these dependencies inside companies to the software team estimates: sales , marketing, documentation, hardware creation, operations, training, partners and so many others.

Software estimates, as much as software teams hate those, and as much those are wrong are mandatory part of all businesses. Those of you who still hesitate, think about Lean value stream map. For customer, software is just a small part of product. Even a plain software product. I can't even invent a software that wouldn't need other activities to go parallel to software creation to shorten the full cycle.

What's the solution then

Estimates are problematic. Sometimes estimates cause waste, sometimes lack of estimates cause waste. When sales uses SW development estimates as promises, those can create business, but at the same time those might cause quality issues, stress and unhappy people. So what could be done to estimating, when to do estimates and how.

I can reveal already now, that I'm not a believer of black or white solutions, so I don't think there is a one solution for all. But here's couple of thoughts how situation with estimating could be improved.

Estimated targets and predictions

One possible solution is to go towards having targets from early estimates and then predictions separately. At least when estimates are not in the team level, but in the project or feature level, it would be beneficial to have a target and then constantly update the predictions based on progress. Both of these figures should be always visible and that would give to everyone else idea what is the real situation.

In this approach there are many potential problems also. People tend to keep the first estimated targets as the goals and then predictions as a follow-up of the situation. This might give the feeling of failure, when the first estimates will most often be wrong. Still this approach should give everyone the idea, that decisions are made with something that will most probably be wrong.

Software aided and fact based estimating

As discussed, people are terrible at estimating. In a way it's easy to say that computers can't be worse. Saying this, I've never seen a good software that could do this properly. There are quite many variables that needs to be taken in to account to have a proper estimate. Still I believe that it would be possible to create framework that would give rough estimates of projects or features.

This framework should benefit from the historical data that has been stored to many software tools that has been used in the organizations. As an input, the framework would need to get software components that require changes. Then based on throughput times in past and historical progress of projects estimating software could give an idea how long something will take. Even thought these estimates would be quite rough ones, still it would give valuable data for human made estimating. And I believe it still would be closer to the truth than many other estimates. As said, unfortunately I haven't seen system like this yet in place. I would be surprised if there wouldn't be any software projects ongoing in this field. I just haven't bumped in to those yet.

It's ready when it's ready

There are few companies who have been good on preventing the harm what bad estimates have on markets. Apple is the prime example of this. I'm hundred percent sure, that even the guys in apple are bad at estimating. They concentrate on the product quality, so they can't guess when they are ready. They have taken the approach on telling when things are ready when those are ready. In Apple case it works for their benefit, but many other companies need to be able to have some timelines beforehand.

Still the approach it's ready when it's ready has something to learn from. It starts with understanding, that estimating software project length is difficult. When this is realized, then companies should build their activities based on this. Sales shouldn't be selling exact dates too early, handovers and dependencies between different teams should be minimized and people shouldn't be rewarded or punished because the estimates they have given.

Software, even the minimum viable product, won't be ready before it's ready. There isn't any amount of estimating activities before hand which can tell for sure when the ready will be. Activities in companies need to be able to react to each others timelines and not work with fixed timelines.

Summary

Estimating is a key aspect of business in software business (and basically in any other business). There are too many aspects and ideas to improve estimating for a one blog post. Some people believe in time boxed development, others think Minimum Viable Product will help and some believe tools will be the thing that will fix the issues. I don't believe there is one solution to fix it. I see the key thing to be that people start to understand that estimating is really hard and people are bad at it. Most businesses need estimates and can't live without those.

As a feedback to many of Agile and Lean discussion ongoing in different forums, I have to say that too many people forget the business behind software. It's so easy to talk about no estimates thinking, Agile practices or Lean values without understanding the effects of it to the businesses. Software is a key part of many businesses, but it won't make money without the other activities in the companies.

Everyone involved in the estimating processes should acknowledge what is the decision estimates are used for. Minimum Viable Estimate could be a good guidance for everyone. What is the minimum level of estimate we need to know in order to make the decision in hand. And as reminder to everyone, estimates don't get better with estimating, those get better with implementing.

Written by +Henri Hämäläinen