Technical Debt is not real

In software development, “Technical Debt” often emerges as a foreboding specter, casting a long shadow over codebases and development teams alike. Yet, herein lies a provocative truth: technical debt is not a tangible entity lurking within lines of code. It’s a metaphor, a way of thinking about the accumulated consequences of past decisions and shortcuts.

In this article, we’ll examine this metaphor and see how technical debt has less to do with the code itself and more about the choices and compromises that emerge from the challenges within software development. We’ll delve into two forms of it: emergent technical debt, which arises from evolving system requirements, and deliberate technical debt, a strategic choice to prioritize rapid development over code quality. By looking at technical debt in this way, we not only refine our understanding of the concept but also identify effective strategies for managing it.

The conceptual origins of “Technical Debt”

The term “Technical Debt” can be traced back to Ward Cunningham, a notable figure in the software development world and one of the original authors of the Agile Manifesto. Cunningham introduced this term to describe a phenomenon in software development akin to financial debt. He explained that just as it’s sometimes necessary to incur financial debt, it can be strategically acceptable to accumulate technical debt:

It’s OK to borrow against the future, as long as you pay it off.

— Ward Cunningham [source]

His analogy was simple yet profound: incurring a small amount of debt and repaying it promptly can be beneficial, but allowing it to accumulate unchecked can lead to a crippling cycle, where one is overwhelmed by the burden of merely servicing the interest. Cunningham’s comparison was specifically targeted at the challenges from technical debt arising “because of the way the system is.” However, the term has evolved, often used to describe a broader range of challenges in software development, including the consequences of expedited development.

Because of the way it is

In the realm of software development, the inception of a project is often marked by a paradoxical certainty: we know the least about the project at its beginning. This is an inherent trait, not a flaw, of the developmental process. It’s a recognition that emerging requirements are both inevitable and unpredictable, a fundamental principle at the core of the Agile movement.

We build just enough to address current understanding without overreaching. This is essential to building a useful product with the least amount of waste. We try to avoid the pitfall of developing features that, while impressive, may not align with the actual needs of the end-users. We must come face-to-face with the crucial question: What value does a feature-rich software hold if its core functionality fails to serve its intended purpose?

However, this pair of ideas has a complex knock-on effect. When a new requirement emerges that is antithetical to our previous understanding of the system, we are faced with a dilemma between two choices. The first is to throw up our hands and declare the requirement impossible “Because of the way it is”—this of course is not realistic. The second is to somehow shoehorn this new requirement into the system by means of a nasty hack. In this view of technical debt, borrowing against the future is an acknowledgement that “we know we don’t know, so let’s not pretend”. This loan will come due when we learn what the future holds.

There is, of course, a third less taken route—adapting the system to make it as if the requirement had always been known. Performing this type of change is analogous to repaying our debt in our financial metaphor. The more we defer paying off these mismatches between what we initially believed and what we know now, the more interest we accrue. This interest takes the form of bugs, of extra time necessary to understand the system, and of prolonged development time for new features.

Buying on credit

There is another, more common, usage of the term “Technical Debt” that seems to have originated from the startup world, a realm where customer needs and business models are often unclear initially. This form of technical debt draws parallels to the consumer credit boom of mid-20th century America.

In the 1950s, amazing new inventions were hitting the market, including vacuum cleaners for the home, dishwashers, microwave ovens, and washing machines. This amazing plethora of new products granted a lifestyle leap to many, previously exclusive to the ultrarich.

To top it all off, acquiring all these amazing products was possible without having the cash upfront; they could all be purchased on credit. But of course, this led to a significant portion of the middle class saddled with debt, living a life beyond their means.

In this startup context, technical debt is akin to buying time on credit. Companies may rush to release products, compromising on code quality with the hope that success will grant them the opportunity to rectify these shortcuts in the future. This mindset – “we will write bad code now and pay for it later” – represents a distinct category of technical debt, differing fundamentally from the kind driven by evolving project understanding.

This situation mirrors the nuances in financial debt. While corporate debt can be a leveraged tool for growth, personal debt, especially unsecured debt like credit cards, can have devastating impacts. However, unlike people, startups frequently die at a young age so a heightened risk appetite at the outset is the norm.

If we have this kind of debt at our organization and it isn’t on the verge of failure, congratulations are in order, we’ve made it. Our best bet is to now try to develop an understanding of the systems we’ve built and to “refinance” our debt.

Paying down debt

A common sentiment among development teams is the desire for a dedicated refactoring sprint, with the belief that this will restore the system to a more manageable state and improve development efficiency. This often stems from a scenario where changing requirements have prevented the team from tidying up the codebase.

However, this reflects a deeper issue: a fundamental lack of solid understanding of the system from the outset, leading to an ad-hoc approach akin to buying on credit. This mindset can dangerously lead to the illusion of a “big rewrite” as a panacea, which, more often than not, results in failure and an exacerbation of existing technical debt.

The concept of refactoring sprints, while seemingly a solution, is often misguided. The root problem often lies not in the code itself, but in the processes that built that code. In cases of what might be termed “corporate technical debt,” a continuous, proactive approach to managing technical debt is more effective than sporadic refactoring sprints. This approach requires a rethinking of how we structure our work—every time we get a feature request that doesn’t fit with the system in its current state, we are in fact getting two stories: change the design of the system to be as though it was always meant to have that feature, and make the actual feature.

This aligns with Kent Beck’s philosophy (creator of Extreme Programming and author of Test-Driven Development by Example) who said:

for each desired change, make the change easy (warning: this may be hard), then make the easy change.

— Kent Beck [source]

Neglecting this practice leads to being mired in code that we resent working on. Working on a system in this dilapidated state is a slog, and the addition of successive features only adds to the problem.

Ignoring this approach can lead to a dreaded scenario where the codebase becomes burdensome “Legacy Code,” causing developer burnout and eventual attrition. In this context, “just one sprint” is insufficient for extricating a project from a technical debt quagmire. It requires a sustained, concerted effort with each feature request to incrementally align the system with the desired state. This involves not just adding new features but also ensuring each addition seamlessly integrates as if it were always part of the system’s design. This strategy transforms the way technical debt is managed, moving away from reactive measures to a more holistic and sustainable approach to software development.

Striking a balance in managing technical debt

This exploration into the realm of technical debt and its dual manifestations should provide a foundational understanding of this complex concept. It’s important to recognize that a moderate amount of technical debt, whether emergent or deliberate, can be manageable and sometimes even necessary. However, an excessive accumulation of either type can lead to significant challenges.

Armed with the knowledge of the two forms of technical debt and how to manage them, when responding to concerns about technical debt from your team, you can ask them to elaborate on what specifically they mean when they use the term. If you are in a technical leadership role, the insights gained here may guide you towards more effective strategies than relying solely on the elusive “magical refactoring sprint.” Ultimately, the key lies in striking a balance and pursuing a sustainable approach to managing technical debt.

Ben Levy is a Partner at Foxhound Systems. We build fast, reliable, and maintainable custom software systems across a wide variety of industries. Want to improve the effectiveness of your development team? Take a look at our Technical Guidance subscriptions or for a larger project, contact us.