Technical Debt - exposed

The Meaning and Origin of “Technical Debt”

The metaphor of Technical Debt (TD) in the context of software was coined by Ward Cunningham who at the time was working on coding a financial system. He used the metaphor to explain to his financial colleagues the reasoning for refactoring and rewriting parts of codes as the knowledge of the requirements evolved.

“When taking short cuts and delivering code that is not quite right for the programming task of the moment, a development team incurs TD. This debt decreases productivity. This loss of productivity is the interest of the TD.”

Ward’s original intended explanation was fairly narrow and specific, but the metaphor now has taken on a life of it’s own:

Nowadays Technical Debt means several things:

• Code that needs changing to eliminate problematic shortcuts, that were created before our evolved understanding of what is needed. (original meaning).
• Code that is buggy.
• Defects in architecture and designs that were short cuts created by tactical decisions
• Code that is hard to maintain
• Code that depends on out of date libraries, operating systems or other technical components
• Defects in requirements that were created because shortcuts were taken in understanding the requirements.

The distinction between Tech Debt, Legacy and Software obsolescence:

Legacy Systems

TD is usually about technical shortcuts having been taken and that are causing problems. Ageing software applications and components that are built on technology that one would not now choose, are often classed as legacy systems. At the time that they were built, shortcuts may not have been taken, therefore there would be no tech debt there. Often people will misclassify a legacy system as tech debt, whereas in fact it might be just approaching obsolescence. If a legacy system has suitable interfaces to other systems and it performs its core role faultlessly, it may be legacy but it may be neither TD nor obsolete. When a system no longer serves the business, although it did for a while, we consider this to be an obsolete system, not necessarily TD.

Software Obsolescence

The value to an organisation of a piece of software will diminish over time. After years of use, the business case for maintaining the old system is likely to diminish. The application has a lifespan and eventually it is more economical to retire or replace it than to keep it going. Software becomes obsolete either when the operating environment that it needs to run become unsupported (hardware, operating system, libraries) or when the software itself no longer serves the purpose for which it is needed.

Once a new piece of software is written and deployed, it is prudent for executives to plan for a few years of upgrades and to plan its eventual retirement/replacement. The economical lifespan of a piece of software varies. Some kinds of software become obsolete after a just few years (many consumer software), whereas others may only become obsolete after decades, (e.g. COBOL banking systems or systems of record in general). Obsolescence is not the same as tech debt. A very reliable piece of software may become obsolete and yet have no TD.

The Cost of Technical Debt

Continuing with the financial metaphor we can say that for some software applications, the interest rate on that TD can be very high, whereas for others it might be lower. If the cost to the business of not fixing the problem is low, then the interest rate on the TD is low. The overall cost of TD varies depending on its impact on the business operation and directions. Take for example a project where shortcuts are being taken with requirements quality. The consequence of this is that the software will HAVE to be rewritten before it goes live. This is foolish creation of TD, which is likely to be avoidable – and goes a long way to explaining why some software projects that neglect requirements are late or fail.  A case by case approach is needed for the impact assessment but a more generic approach can be considered for the sizing of the debt.

COSMIC Sizing of Technical Debt

TDin code can be measured in CFP.

How the TD arises can be from poor requirements, architecture and design decisions and shortcuts.  This then transpires into code which either does the wrong thing or does it in a way that will cause inappropriate functionality.

Quantifying Technical Debt

These are the questions that prudent executives should be asking:

• How big is the TD?
• What is the cost if we don’t fix it?
• What is the cost if we do?

For organisations that are running hundreds of applications with operational and strategic data, it is helpful to normalise the response to these questions so that the investments to address TD can be prioritised.

Size and repair cost

Functional sizing is an excellent approach for standardising the sizing of TD.  The unit of measure is the COSMIC Function Point or Function Point. For example, application has a total size of 1000 CFP of which 100 CFP are interfaces that need rewriting (TD).  Cost per CFP or cost per FP to replace or rewrite the shortcut can usually be determined readily using benchmark data.  Repair cost might be $800 – $1500 per CFP.   Using a universal sizing approach means it can be applied across TD in code, designs, architecture and even requirements.  It is very important to consider also the cost of migrating from the before and after state as well as any non-technical impacts, such as people and process changes.

Impact on Business

When determining the overall cost of the TD, you need to consider both the amount to which it is impacting your business and the cost to repair it.  The case-by-case approach to impact assessment is appropriate.  And this can usually only be determined by business and technical folks collaborating to answer “to what extent are technical shortcuts preventing the business from achieving its goals”

Exposing Technical Debt

Exposing TD is useful as it can help provide early information for strategic decisions about software systems. For example a system that is burdened with TD may be a candidate for early replacement. Whereas another piece of software that has limited TD may be better suited for extending its lifespan.

Spotting technical debt in your organisation

If a software application is constantly being described as inflexible because of the way it stores/handles/structures data, then this may well be TD that is holding you back.  In other words, technical shortcuts or poor design decisions that have never changed to meet the needs of the businesses.

If a project to implement some new or changes to software experience extended periods of bug fixing or refactoring, this may be caused by poor requirements work that generated technical debt.

Avoiding Technical Debt

This is the most prudent approach at all. It is the notion that it is cheapest to develop the software right first time. However, respecting Ward Cunningham’s original definition of the term, this not be possible. If the (strategic) need and use of the software was not knownable when the code was first being written, then some TD was unavoidable. However, all efforts should be made to keep this rework to a bare minimum. The approach for doing so is to allow sufficient time and resources to do the requirements, architecture and design work adequately before coding starts. This should expose the knowable unknowns so that the creation and elimination of TD is avoided.

Reducing Technical Debt

There are two general approaches to reducing TD:

1. Do the detailed rewrite to eliminate the problematic shortcut or defects.

2. Leapfrog the Debt. Do a wholesale replacement of TD with suitable re-usable code. (“Leapfrog the Legacy”, the idea of leapfrogging the legacy is not new, although I coined this phrase as I think it helps non-technical folks understand the case for some involved technical decisions.)