Digital transformations, or large software projects have a reputation for exceeding their original budgets and schedules. It turns out that they are almost never early and nearly always late. A new study on project failures looks at a new set of projects data from a statistical perspective.
It is essential to work proactively to reduce the factors that will cause a project to go over budget and overschedule.
One of the biggest risks on large digital transformations is with requirements. Requirements clarity, consistency and volatility are all risks that require pro-active management and here’s and graphic to illustrate why:
A team working on a very large government digital transformation who are responsible for cost estimation chose to use ScopeMaster to analyse and size the requirements. They were delighted and amazed with the results. Their goal was used to automate the sizing in COSMIC Function Points in order to generate a reliable cost estimate. Not only did ScopeMaster estimate the likely functional size (and hence cost) but it also exposed defects in the requirements, one of the biggest digital transformation risks.
The digital transformation programme is divided into several projects. The first phase of the first project consisted of over 2300 requirements.
The project is an HR package implementation that requires complex legal rules being embedded into the behaviour of the software. The software is mostly a package implementation with modest amounts of custom coding. There are hundreds of pages of business rules (legal documents) to be implemented. Each business rule affects many citizens, potentially hundreds of thousands being affected by each rule. It was established that a mistake in a requirement that incorrectly implements a rule, is likely to cost at least $100k with consequential costs. Some rules, if implemented incorrectly could lead to costs of many millions. Any defect exposed to the users would cost a minimum of $100k.
ScopeMaster analysed 2300 of the requirements (in under 1 hour) and exposed an over 3000 requirements defects. It also estimated the functional size of the project. In this ongoing case study digital transformation use of ScopeMaster, the exposure of the number and specifics of the defects is both valuable and timely.
The Risk Exposed
ScopeMaster found over 3000 legitimate defects in 2300 requirements. What does this tell us and how can we use this information to help the digital transformation project succeed?
We previously established that each defect will lead to an incorrectly applied business rule, which is likely to incur at least $100k of total cost. It might be reasonable to assume that most of these defects will be found and fixed anyway before release, 85% is a reasonable assumption based on the typical test-only and risk-based testing approaches. That leaves 450 defects that would go live, with a minimum consequential cost of $45,000,000.
Another way of looking at this risk is to consider the risk of cost overrun. We will assume that the average implementation cost per COSMIC function point (CFP) is $1000. Although normally the cost per CFP would be lower for a package implementation (circa $200/CFP), the costs are usually higher in the government context AND the complex nature of the legal rules that are being encoded may require costly legal expertise to review each one.
The cost of implementing a requirement correctly is $1000/CFP. If the requirements errors are not fixed, Each requirement will be need to be implemented twice, once incorrectly and then correctly. Software rework is less efficient than “getting it right first time”, usually productivity drops by 50%. So the reworked cost is $1500/CFP added to the initial cost of building it wrong in the first place $1000/CFP, gives us a total cost of $2500/CFP – or a budget overrun of 150%. In other words, if these requirements problems are not fixed prior to coding, there is a risk that the project will cost 2.5x its initial estimate. As we can be certain that these defects will cause the increased rework cost, this is not a project risk but an issue.
The likely functional size (as estimated by ScopeMaster) of the 2300 requirements in the digital transformation project is about 20,000 CFP.
By fixing the requirements quality and avoiding the rework, we can avoid the extra development cost of $30m (1500 x 20,000). The work required to fix the requirements before coding (using ScopeMaster) would be about $300-500k (licensing and effort). For less than $0.5m, the project can avoid a nearly certain $30m cost overun.
The project is ongoing and the concern about requirements quality remains in active discussion across the team. We are very proud that ScopeMaster has been able to expose this risk before coding even starts so that it can be mitigated. The saving to the government in question as a result of mitigating this risk exceeds $30m.
