When we start a project, as engineers we tend to understimate the effort to deliver that project and give an optimistic delivery date, then that date is turned into a commitment and it leads to overtime work on the eng team and frustration on the customer or stakeholders when there is a delay. The planning fallacy explains this phenomenon, and we might be tempted to not give any estimates at all but customers and stakeholders want to know when a feature will be delivered, because based on that they can make some plans.
The goal of this document is to define an effective and consistent estimating process, as well as show a clear way to transform estimates into time. The elements we need are: effort, story points and velocity. These are familiar terms, but worth reviewing because they might have different meanings to every developer.
Effort
For certain activities like flights or food delivery we have accurate estimates because they are processes that have been done thousands or millions of times and variability is low. But in software development we work with unique tasks that have their own complexity, risk or uncertainties and getting an estimate is not trivial.
One way to decompose the effort in software development is considering it’s the sum of risks or uncertainties, complexity and amount of work required to fully implement and potentially release a user story.
Amount of work
The amount of work is the number of defined tasks or changes that are required to complete the story. Those taks are sometimes overlooked because they are straighforward and minor activities, however in some cases they represent a big protion of the feature.
Risk or uncertainties
This is the most variable factor of the estimation process, when some uncertainties are not sorted out they could lead to scope creep or when risks are not addressed, the project could derail. Here are some scenarios that we should avoid before giving an estimate
- we don’t have full specs
- the implementation approach is not clear
- we work with legacy code that is hard to understand or set up the environment
Complexity
The complexity of a story can increase when:
- there are dependent components or repositories
- we work on new projects
- we have multiple business rules
There is not an exact formula to combine the previous factors, but as we get more practice and also partipate in the discussions we can improve our estimates. The important thing is not to forget to include the 3 factors in the estimate.
Story Points
To measure the effort, we’ll use the story points which are an objective and relative unit that we give to each user story. Story points are a function of the effort, this is the formula:
story points = f(amount of work, risk or uncertainty, complexity)It’s important to note that story points are a relative unit, it means that every project or team needs to define their own scale. We can define a scale for the team so that we can create a roadmap or timeline.
A common practice is to use Fibonacci numbers to give estimates because uncertainty grows with the size and complexity of the work, making precise estimates less meaningful for larger tasks.
Since the story points are a relative unit here are some examples of stories of different sizes.
1 story point stories
This is the smaller size and a story to be 1 pointer should have low risk or uncertainty and be a change of a existing component.
2 story points stories
A change to a existing component in multiple repositories or files. Also new components build on top of existing components.
3 story points stories
Could be a mix of changes to existing components and creating new components.
5 story point stories
Stories that create a new section or with a high complexity or uncertainty.
8 and more story point stories
When we have a story of 8 points or more we have to try to split into more manageable stories. But if by splitting the story doesn’t provide value to the user or doesn’t have a deliverable we can work on it.
Velocity
We can define the velocity as the number of story points completed in a given sprint. This metric gives us an estimate number of sprints it could take to complete a story based on the numbers of points, the formula is:
number of sprints = story points / velocityUnder normal circumstances, we can say that when the estimated number of sprints are completed we’ll have our feature delivered and turn into a potential release date.
On the other hand, we can also use team velocity to define the commited work in each Sprint.
Cone of uncertainty
We have defined some formulas, but the result is only as good as the inputs. The problem is that we cannot know all the inputs with 100% certainty until the project is completed. At the beginning of a project there are more unknowns, and as we work on it we learn more about the scope, the risks and the technical approach.
This is the cone of uncertainty. The more we know about the project, the better estimate we can provide. The cone should not be a reason to avoid estimates, but it explains why an estimate at the beginning of a project has more variability than an estimate for a story that is ready to start.
To deal with this variability, we can:
- Provide regular updates and adjust estimates as we learn more about the work.
- Use ranges instead of fixed dates. For example, we can say that a feature will take 12 to 16 days, rather than exactly 14 days.
The important point is that an estimate is not a commitment made once at the beginning of a project. It is our best prediction based on the information available at that moment. As uncertainty decreases, we should revisit the estimate and narrow the range.
This makes uncertainty explicit instead of hiding it behind an apparently precise number. Early estimates should have wider ranges, while work that is well understood and ready to start can be estimated with much greater confidence.
More effective estimates
So far we have focused on estimating the work, but an estimate also needs to account for the team and the resources available to deliver it. Once we understand the estimated effort and the team’s velocity, we can compare the estimate with stakeholder expectations and business constraints.
The project management triangle helps explain these trade-offs. Scope, time and cost are connected: fixing or changing one of them usually requires adjusting at least one of the others. Quality should remain part of the definition of done, rather than becoming an implicit trade-off.
Depending on which variable is more important, we can use different strategies.
Fixed Time
If the most important constraint for the company is to deliver a feature by a specific date, then we can adjust scope or cost. In my experience, this is the most common request from stakeholders. The scope needs to have a clear line between what is required for the release and what is nice to have.
Possible strategies are:
- We’ll deliver the required functionality by the target date and move the nice-to-have functionality to a later release.
- We’ll deliver the full scope by the target date, but we’ll need to add X more engineers to the project.
Fixed Scope
When the full scope is required, we need to accept variability in the other factors. Since we are committing to delivering all the features, the delivery date needs to remain flexible. In this case, it is better to provide a range rather than an exact date, especially when there is still uncertainty in the estimate.
Possible strategies are:
- We’ll deliver the full scope in approximately X days, with a variability of +/- Y days.
Cost and Opportunities
Companies naturally try to optimize costs, but in my experience estimates are rarely expressed in terms of cost. We usually talk about how long a project will take or what scope we can deliver, rather than how much it will cost.
Cost also overlaps with time, since development time is itself a resource and usually one of the most scarce. However, thinking explicitly about cost can open up alternative ways to solve the problem instead of only adjusting scope or delivery time.
For this factor we can consider options like:
- We might not need to develop the feature in-house and could use a third-party service instead.
- We might simplify the process, explore a different technical approach, or involve an engineer with more experience in the problem domain.
“Precise Estimates” Is an Oxymoron
An estimate is, by definition, an approximation based on the information available at a given moment. If we knew exactly how much effort a project would require and when it would be completed, it would no longer be an estimate.
This does not mean estimates are not useful. Their value is not in predicting the future with laser precision, but in making uncertainty explicit and providing enough information to plan, prioritize and make trade-offs.
Estimates should help teams and stakeholders make decisions, not create artificial commitments. When expectations and estimates do not align, scope, time and cost give us different levers to find a realistic plan.
The goal is not to estimate perfectly, but to communicate what we know, what we don’t know, and what options we have.
Resources: