Move Aside Story Points! Hello CFP
Story points are unsuitable for use as the primary metric on software projects. We recommend that you supplement them with COSMIC function points and give 14 reasons why.
What are Story Points?
In Agile software projects, a story point (SP) is a team’s agreed amount of effort to do some work. Tasks are estimated in the number of story points needed to do the work. Story points are an arbitrary “local currency”. There are no hard and fast rules as to how big a story point should be. Story Points are not even part of the Scrum Guide. The magnitude of a story point varies from team to team. It can vary over time too. A story point and the estimation of the size of a piece of work in story points is just the team’s “collective guess” at a given point in time using an inconsistent metric. Essentially story points are a proxy for hours according to Mike Cohn. For even more challenges with SP, see Problems with story points.
Where did they come from?
Story points are a consequence of adopting Agile software development methods. One of the fundamental principles of Agile software development is that teams self organise. Consequently, it has become accepted practice that they should also invent their own standard measure of effort.
Allowing the team to determine their own metric for size/effort has value in team cohesion. Furthermore, it is useful to have a means for estimating ancillary work (tasks that are not specific to delivering functionality) although simply estimating in hours might also be used.
The Problems with Story Points
- Story points are inconsistent, over time and from team to team.
- Story points cannot be used as a foundation metric on projects
- Story points are unsuitable for contracts
- Story points are generally unsuitable for process improvement
“Agile software needs to grow up, no other professional engineering practice uses inconsistent metrics”
No other activity is measured using a system where every team uses it’s own (ambiguous) standard
The most significant factor in determining a software project is the size of the software to be delivered. There are many other significant factors too. But size is the #1. Without a common currency for software size, projects are hard to estimate, they are even harder to measure and if you cannot measure them, they are very hard to manage.
Story point estimates are specific to the team doing the job. A story point is actually a proxy for the estimated effort to do some work. There are typically just a “feel” for size. As team members come and go, the opinion of the amount of effort to do some work can fluctuate, so the magnitude of a story point can change. Consequently, estimates of size in story points are meaningless for comparisons with another team, let alone another project or organisation or another industry.
Agile teams typically estimate size in story points only a few weeks before doing the work. This makes the project manager’s longer term planning work very hard indeed.
“The most important factor on a software project is software size. And the best way to measure software size is COSMIC function points. “
Organisations need to improve
It is virtually impossible in most organisations to be able to look forward or back at a project and use story points for comparison or learning about the characteristics of one project over another. Story points inhibit learning.
The Principal Metric on a Software Project
The key concerns when managing a software project are: software size, effort, time, quality and risk. Organisations need consistent metrics for all of these. If you know the size you can quickly determine appropriate metrics for quality, schedule and resources. Size is the pivotal metric. Story points conflate software size and effort, so cannot measure either properly.
Organisations need indisputable measures to estimate and learn.
If you can consistently estimate the software size to be delivered and you have performance data on past comparable projects, you can quickly determine estimated effort and schedule and hence resources needed. This cannot be done with story points when it’s not clear if they measure size or effort. What makes project management far easier and more effective is to have an indisputable set of metrics to work with. Relying on story points alone inhibits organisational learning and improvement.
“Relying on story points inhibits learning and maturity “
Story Points are Unsuitable for Outsourced Development
Story points have no place in outsourced development contracts. For example, you cannot have a contract to deliver 100 story points, because there is no absolute consistent understanding of the magnitude of a story point. The result of this is that the customer has no visibility of value for money nor any way of controlling it.
Agile contracts tend be characterised by uncertainty and a lack of detail on requirements, prior to engaging a development contractor. As a result most contracts for developing software are essentially time and materials based. In a T&M contract it is hard to negotiate a price for functionality at a given quality for a given cost. Many vendors will agree to stand up a team that will deliver whatever functionality the customer asks. In many circumstances that the author has seen, the development organisation is also entitled to charge for the effort to fix bugs in the software they have delivered. They can make as much as money for fixing their own bugs as developing the functionality in the first place. To address this problem. we propose contracts based on firm fixed price per function point with quality constraints based on defects discovered per function point.
Cause Inflated Costs
On an outsourced development contract we often see delays and cost overruns caused by poor quality work. Often the quality problems are caused by the customer who has provided poor quality requirements. It may not be in the commercial interest of the development organisation to challenge the quality of the requirements or to even point out the dangers of continuing. Irrespective of which party is at fault for creating the extra work, it is often in the development organisation’s commercial interest to allow the quality problems to cause the delay, and then get paid extra to do the repair work. How is this the fault of story points, you ask? Story points perpetuate ambiguity, and that ambiguity leads to T&M contracts instead of a contract based on a firm unit price for delivered functionality.
“Use of Story Points alone in outsourced contracts perpetuates poor quality and higher costs.”
STOP, there is a better way, and it has been here all along.
COSMIC Function Points – the Development Manager’s Best Friend
Few software managers are actually aware that there are proven universal standards for measuring software functionality size. The idea of measuring functional size, has been around a long time. The Function Point is the software equivalent to measuring weight in Kilos.
The most up to date edition is the COSMIC Function Point (CFP). It is the modern refinement of a proven technique. It is free (open source) and it is based on fundamental software principles and so is universally applicable.
The process of measuring is called functional sizing and it is a way of measuring the size of software functionality using a metric that is always the same, and therefore comparable across teams, projects, organisations and industries.
- Why is it better?
- It overcomes all of the limitations of story points as mentioned above.
- It is proven and mature (an ISO standard)
- Why haven’t I heard of it before?
- In short, it is not been fashionable
- Until now, it has been time consuming to do the measurement work.
- Should I use CFP as well as story points?
- Yes you may continue to use story points alongside CFP.
- Story points are working for me, should I change?
- It depends on who you mean by ‘me’. In some (mature and stable) in-house software teams, story points can work quite well for estimating User Stories and sprints, and there is little reason to change as far as an individual team is concerned. But, for all the reasons listed above, the weaknesses remain at an organisation level. These weaknesses will never be overcome if you continue to use story points alone.
|Purpose / Benefit||COSMIC Function Points||Story Points|
|Good for sprint estimation||Yes||Yes|
|Good for Application Sizing||Yes||No|
|Time spent sizing user stories improves quality||Yes||Yes|
|Consistent across teams||Yes||No|
|Consistent across organisations and industries||Yes||No|
|Suitable for Development Contracts||Yes||No|
|Development Contracts with uncertain scope.||Yes||No|
|Suitable for Organisational Learning||Yes||No|
|Helps to reduce costs on outsourced development||Yes||No|
|Suitable for predicting and reporting on quality||Yes||Somewhat|
|Suitable for early size and cost estimation||Yes||Somewhat|
|Good for project estimation||Yes||Somewhat|
|Good for sizing non-functional work||No *||Yes|
|Good for software portfolio sizing, high level planning||Yes||No|
|Sizing can be automated||Yes||No|
|Good for board-level reporting (of development and maintenance activity)||Yes||No|
|Good for comparing team productivity||Yes||No|
* Many non-functional requirements can actually be re-articulated as functional requirements.
Conclusion and Recommendation
We recommend that any team that is already using story points should continue to do so. Alongside this, they should also measure their software in Cosmic Function Points. Confine the use of SP to work within the development team. When looking at estimation, contractual metrics, sprint capacity and project management, use CFP only. In time, you may find that you no longer use SP, instead focusing on the functional size of delivered software.
Find out more about COSMIC Function Points
All the COSMIC method’s comprehensive documentation is available for free download at www.cosmic-sizing.org. We suggest you start by reading this introduction to COSMIC sizing . The sizing method is based on fundamental software engineering principles and is very easy to learn.
By Colin Hammond and Charles Symons