What is software productivity?
It’s actually pretty straightforward: productivity is simply output over time.
The tricky part, however, is finding a suitable measure for outputs. Outputs—at least in the context of software—are hard to standardise. In most cases, we can think of these outputs in terms of business value and/or how we deliver user-recognisable capabilities.
Why do so many companies want to measure software productivity? Is it really that important?
For any software venture, managers have a duty to seek quantification and some degree of certainty about cost, schedule, and scope. They will also be looking for means and measures to…
- …benchmark against other organisations
- …track progress over time
- …assess team and individual productivity
- …use productivity to determine offshoring decisions
- …use productivity to inform prioritisation decisions
General productivity measures
We tend to focus primarily on developer productivity, as developers are generally the most active and most expensive members of a software project. However, it is also useful to understand the productivity of others, including testers, analysts, project managers, designers, architects, and anyone else who contributes to the success of these projects.
Criteria of a good software productivity metric
Sadly, there is no such thing as a perfect metric for “output” on software work. Therefore, we have to find alternative, proxy indicators that get us closer to quantifying the output.
Ideally, for software, we want an “output metric” that achieves the following:
- strongly correlates to business value
- is ungameable, as well as consistent, objective, repeatable, and independently verifiable
- is independent, functioning regardless of programming language
- is valid for comparing one project to another
- can account for the more complex work being assigned to more capable team members
- can be collected cheaply and easily
As a side note, ideal metrics for individual productivity do vary slightly from those ideal for determining team productivity. Use whatever works best for your scenario.
Ranking of software metrics
The following table was produced by Steve McConnell and is discussed in this presentation for ranking suitable software productivity metrics. COSMIC function points were less well-known and automated sizing was unavailable when Steve first created this table. We have added the column for COSMIC function points here. This extra column is subjective and is, in our own opinion, not part of Steve’s original assessment.
LOC/SM
|
FP/SM
|
CFP/SM
|
Story Points/SM
|
360 Peer Reviews
|
Manager Evaluation
|
Task completion predictability
|
Test Cases Passed
|
Defect Rates
|
|
---|---|---|---|---|---|---|---|---|---|
Measurement truly reflects productivity | 3 | 4 | 4 | 4 | 3 | 2 | 1 | 4 | 3 |
Directly or indirecty accounts for all or most work output | 4 | 4 | 4 | 5 | 5 | 5 | 2 | 4 | 3 |
Useful for measuring work of non-programmers | 1 | 4 | 4 | 4 | 5 | 5 | 5 | 4 | 4 |
Resists “gaming” by individual contributors | 2 | 5 | 5 | 4 | 3 | 3 | 4 | 4 | 5 |
Strongly correlated with business value | 2 | 3 | 4 | 4 | 3 | 4 | 2 | 4 | 3 |
Objective, independently verifiable | 4 | 4 | 5 | 3 | 2 | 2 | 4 | 4 | 4 |
Measures output the same regardless of programming language used | 1 | 5 | 5 | 3 | 5 | 5 | 5 | 5 | 5 |
Supports person to person comparisons within a team | 3 | 4 | 4 | 4 | 5 | 4 | 4 | 4 | 4 |
Accounts for best people getting the most difficult assignments | 2 | 4 | 5 | 4 | 5 | 5 | 1 | 5 | 2 |
Data can be collected easily and cheaply | 3 | 1 | 5 | 3 | 3 | 4 | 3 | 3 | 4 |
25 | 38 | 45 | 38 | 39 | 39 | 31 | 41 | 37 | |
Rank | 9 | 5 | 1 | 5 | 3 | 3 | 8 | 2 | 7 |
In Steve’s original assessment, the conclusion was a close run between several approaches, with “Test Cases Passed” ranked highest. Now, with the advent of a simpler approach to functional sizing (COSMIC Function points vs. IFPUG function points) and automated sizing, there appears to a be a clear leading metric for assessing productivity: COSMIC Function Points.
Misleading productivity measures
While there is no single perfect metric for software work, some are more useful than others. Some metrics exist that are likely to mislead, be abused, or instigate unhelpful behaviour.
Lines of Code (LOC)
Developers can vary wildly in the lines of code written for any given functionality. It is easy to measure, but not necessarily a good measure, as different programming languages require different amounts of coding, and the variances between can be vast. Using lines of code as a measure can also encourage developers to write superfluous lines simply to pad their productivity. This has a counterintuitive effect, as it can generate additional maintenance costs.
Story Points
Story Points are a subjective, non-standard proxy for effort estimation. Some will claim that they measure complexity; the reality is that, at best, they might give an indication of difficulty for a particular task. In essence, these are an indication of “ideal” days for staff. They are unsuited for contract work and cannot be used to compare one team to the next, nor one project to the next. There is no learning or continuous improvement achievable by adopting story points.
Story Counts
User story size can also vary significantly. One story may be a ten minute technical task for an individual; another story may require a whole month of work for an entire team. In our experience, stories range from 0 CFP to 120 CFP, which renders count an inconsistent metric.
Other team productivity measures
Score Card-based productivity assessment can be useful in some circumstances especially if the card values are both normalised across teams and chosen to reflect all the criteria that the organisation wants to use.
It is important to choose criteria that focus on organisation’s values (for quality and output). It is also important to get teams aligned with the true objectives of the business.
The winner: COSMIC function points
Our recommended output measure is COSMIC function points (CFP). Here’s why:
- CFP is rooted in fundamental principles of software behaviour (inputs, outputs, reads and writes).
- CFP is user-focused; it aligns to what is needed to deliver value to the user.
- CFP is a reasonable proxy for business value.
- CFP aligns well to actual effort.
- CFP has a definition for an individual unit of size.
- CFP provides a measurement of size, which is the most significant factor in determining cost and duration of a project.
- CFP allows managers to manage non-development activities, with CFP as a base metric.
- CFP is open, free, and independent.
- CFPs are suited to incomplete requirements (i.e. Agile).
- CFPs are suited to almost any type of software.
- CFP early estimation is almost entirely automated (by ScopeMaster, of course).
Software development productivity measurement is the activity of recording the metrics and attributes of a software endeavour for comparative purposes.

How productive are our software developers?
Want to know if your team is more productive than other teams or than industry norms?
You can compare the activities of one team vs. another within an organisation, which is internal benchmarking. Comparing against another similar organisation is referred to as external benchmarking. Although industry benchmarks may be interesting, local and internal benchmarks are generally considered the most reliable.
Software development productivity benchmarking and assessing your team
If you have a great team, why not publish your productivity index? It could help you win development work. All you need is a set of user stories from a previous project, about one hour, and access to ScopeMaster.
- Take a set of requirements from a recent project. Upload them into ScopeMaster and let the analyser work out the size.
- For the productivity calculator, enter total project effort (in personal days), project duration (in months), and the defect removal you ultimately achieved on the project.
- ScopeMaster will then determine the productivity index of your team.
…and that’s all there is to it!
If you rank above average (approximately six on the index), then you are on the track to greatness. If you’d like, we can even publish your team’s productivity index on our league table!
Remember: be wary of misleading metrics. Story points, lines of code, T-shirt sizes, and man-days are all gameable, easily manipulated techniques for estimating time and effort. Stick to ISO-standard functional sizing methods for best results.