Discover more from Tech World With Milan Newsletter
In this issue, we are going to talk about the following:
What Is Technical Debt?
Types Of Technical Debt
How to Measure Technical Debt?
Strategies to Fight Technical Debt
A Recommended Strategy To Deal With Technical Debt
Tools to track Technical Debt
Keeping Technical Debt Low
What Is Technical Debt?
Technical debt can cause so much frustration and burnout to development teams. Software engineers can be aware of the side effects of technical debt. However, they often need to explain to the product team why quick and easy solutions to coding development are risky.
So, instead of stabilizing the situation, the business keeps adding more features, and the technical debt grows. Because of this, we usually say that technical debt is not a technical problem. For that reason, every software development team must try their best to prevent the technical debt from accumulating so it doesn't result in the worst-case scenario, which is a project halt.
Ward Cunningham coined the term in 1992 at the OOPSLA conference [1] as a metaphor for developing a software asset. He concluded that the development process leads to new learning, as it depends upon artifacts he coined as technical debt.
Technical debt is everything that stops us from developing software quickly. Martin Fowler explains in his technical debt quadrant four different pathways that lead to technical debt creation [4], but there are more. We can see this, especially with startup companies that want to go fast and don’t think about quality first.
One may intentionally or unintentionally create technical debt. Using the Technical Debt Quadrant as an example, Martin Fowler explains:
In studies on developer productivity [2], developers are usually forced to introduce a new technical debt as companies keep trading quality for short-term goals, such as new shiny features. Yet Technical Debt doesn’t only impact the whole organization but also developers’ happiness, job satisfaction [3], and morale [8].
The main problem with Technical Debt is that code is an abstract concept, so it’s hard to explain what happens to business and management. So, companies could easily ignore what they don’t see or understand. So, here, we need to be explicit and visualize our Technical Debt for the company and management.
Usually, during the development of our software systems, the capacity available for new features per development cycle decreases (lead time is increasing) while complexity builds up. So, we spend more time fighting the complexity than developing new features.
The image below shows how technical debt reduces teams' ability to develop new features.
We want a high-performing organization that will give us a competitive advantage to be more responsive and innovative. We need a high-quality software development process to do this quickly.
Technical Debt will occur anyway, yet we need a strategy to reduce the creation of new debt while allowing us to reduce existing debt. Investing in this will build technical wealth for our company.
Types Of Technical Debt
We usually think bad code = Technical Debt, but there is more to it. There are different types of tech debt:
Code quality—our code could be clearer to understand. It lacks coding standards, is poorly designed, and is very complex. In addition, there are many code comments explaining what something does.
Testing - we need a proper testing approach (check Test Pyramid), from unit tests over integration to E2E tests.
Coupling - between modules that block each other increases the time to deal with such code.
Out-of-date Libraries or Tools - we are using some legacy libraries or tools with several issues, such as security or inability to update to new technologies and platforms. We, as developers, always want to work with the most up-to-date technologies and efficient tools.
Manual Process—Some processes in our delivery need to be automated. There are no automated builds and no CI/CD process.
Wrong or no architecture - we don’t have proper architecture, or it’s just a big ball of mud. Our architecture doesn’t reflect what we want to achieve with our system, or it doesn’t scale well.
Lack of Documentation—There is no documentation, or it has not been updated to reflect the system’s current state.
Lack of Knowledge Sharing—We don’t have a culture of knowledge sharing, so it isn’t easy for newcomers to get up to speed. We should always document our decisions and specifications during our work.
And there are more types, too, regarding low-value work, issues with monitoring, etc.
Yet, many organizations need more processes and strategies to manage and reduce technical debt, which signifies a proper engineering culture.
How to Measure Technical Debt?
When you ask experts how to measure Technical Debt, you will get different answers. Sometimes, it is the opposite. During many years working as a software engineering expert and consultant, I figured out a few practical ways to measure it.
1. Technical Debt Ratio (TDR). This compares the cost of fixing technical debt to the original development cost. You can express this in terms of time or money spent. TDR helps get buy-in from business stakeholders by translating technical debt into financial terms.
A ratio of the cost to fix a software system (Remediation Cost) to the cost of developing it (Development Cost). This ratio is called the Technical Debt Ratio (TDR):
Technical Debt Ratio = (Remediation Cost / Development Cost) x 100%
We want to see this ratio at 5% or less. If it is higher, the software is in a bad state. This score indicates, in time, how long the engineering team would take to improve their codebase to the desired quality.
A ratio between 20% and 50% indicates significant technical debt, where remediation costs begin to question the feasibility of continuing without substantial investment in resolving these issues. Projects within this range often require targeted debt reduction strategies to prevent escalation.
Ratios exceeding 50% often signal that the cost of fixing the debt is half as much as or even exceeds the initial development cost. This is typically the threshold at which businesses should seriously consider rewriting or abandoning the project. Yet, before deciding, you need to know a few things, such as the business impact, future-proofing tech, cost-benefit analysis, time to market, etc.
2. Code Quality Metrics. Use tools to analyze code complexity, duplication, and violations of coding standards. Metrics like cyclomatic complexity, code churn, depth of inheritance, and lines of code ratio can provide quantitative data on the state of your codebase. Automated tools like SonarQube can analyze code and identify issues like code complexity, potential bugs, and duplication.
3. Defect Ratio and Lead Time. Track the number of new bugs introduced compared to the number fixed (defect ratio) or the Change Failure Rate in Dora metrics (what percentage of your changes cause a failure). Lead time measures the time to deliver a change. High defect ratios and long lead times can indicate technical debt, making development slow and error-prone. You can create a "tech debt" label for new tickets related to it and use it to measure the technical debt.
4. Code Coverage. This metric measures the portion of code executed by automated tests. Higher coverage indicates a more robust codebase is less likely to have hidden technical debt.
5. Surveys. You can also regularly ask your people how bad technical debt in the project is in their view. You can ask them to scale it on a scale of 1 to 10, which doesn't mean much, but if you track it over time, you can see apparent trends.
Strategies to Fight Technical Debt
So, what are some strategies to reduce technical debt? We should all start by prioritizing Technical Debt. First, we must analyze our development process and codebase to find bottlenecks and create a Technical Debt Map (shown in the image below). Then, once we have such a map, we need to review each piece by identifying causes.
When we have this list, we can go through it and check what happens if we do nothing (could it become worse or not) and if this part of our system is used for new development now and in the future. So when we mark these two things, we can prioritize and keep stuff with worse tech debt, and we will build on those now and in the future.
Another alternative is to build this map based on the effort and pain involved in fixing issues or using the RICE Scoring model.
You can also organize a workshop or Tech Debt retros to build a map. You should also regularly audit debt across all of these categories.
Read how Google measures Technical Debt.
A Recommended Strategy To Deal With Technical Debt
One recommended strategy to deal with Technical Debt consists of the following steps:
We need to be transparent about our development, bottlenecks, and why we need to move faster. Then, we need to summarize this in a language that allows all stakeholders (engineering and management) to share the same understanding. Visualizations help a lot here.
Our systems need clear ownership and responsibility. Unfortunately, if some parts of our system aren’t right, no one cares for them.
We need to prioritize stuff based on impact, as we showed on the Tech Debt Map above.
We must empower teams to fix problems and resolve technical debts in the natural flow of product development. We need a healthy balance between reducing tech debt and adding new functionality with a pragmatic approach. What works well is to continuously invest 10-20% of development time into this. How you can do this:
For smaller debt (up to a few hours), do a refactoring when you touch it. But, then, let people be Good Boy Scouts.
For medium debt (from a day to a few days of work), you can try the following:
Tech Debt Fridays are working only on this.
Mark tech debt as a feature, prioritize it and work on it.
For more significant debt (from a few weeks to a few months):
Take fixed time for tech debt. E.g., take a story and work two days on it.
Have a special task force (dedicated team) to handle it for some time.
Use metrics, such as code issues (e.g., using SonarQube) or lead times. These metrics can help us make better decisions about fixing Technical Debt.
Tools to track Technical Debt
There are different kinds of tools that can track technical debt. However, I recommend always using some existing tools because introducing new ones can be costly. In general, Technical Debt can be followed in the following ways:
Project management tools include Jira, Asana, GitHub issues, Azure DevOps Board, or more specific tools such as Stepsize, CodeScene, or NDepend (for .NET).
Automated code analysis tools—you can use software tools like SonarQube to track and detect code issues, security issues, or other problems.
Manual tracking method—here, you can use the most suitable method, such as an Excel table or whatever works for you. It can even be a combination of the above methods, such as tracking strategic Technical Debt in Miro, tracking tactical Debt in Jira, and using metrics from SonarQube. Just make it accessible and easy to update.
Keeping Technical Debt Low
We want to produce code with the right speed, quality, and complexity, not in a quick and dirty way. As software evolves, our systems' architecture becomes more complex, making maintenance difficult. However, if we started with poor design and code quality, we would be in trouble later.
Our goal is to produce systems that are easy to maintain, that we can avoid errors, and that issues and further improvements can be added at a low cost. Software architecture and craftsmanship should be appropriately set to achieve this. We want to balance quality, speed, and complexity (as shown in the image below).
Yet, the faster we develop new functionalities, the more difficult it is to ensure software quality. Tech debt tends to pile up, leading to a slower development process and more errors. So we want to go slower from the start, which will give us more speed in the future.
The question is how to avoid creating tech debt in the first place. The answer is to have a high-quality software development process. Here are some things we can do:
Architect systems for the desired outcome. Involve architects from the start in creating teams by defining their topology [7] because otherwise, Conway’s Law [5][6] will make our architecture an image of our organization’s communication channels, not the desired architecture.
Write documentation for all your architectural decisions (ADRs) and technical documentation. Make it easy to find (place it in the repo near the code).
Write tests on all levels of the Testing Pyramid. Aim at 60-80% of code coverage.
Encourage a code review culture. Involve everyone in code reviews and check for code readability, design, performance, security, testability, and documentation. It is one of the best ways to share knowledge.
Pair Programming works where one developer writes an ode while another provides feedback and suggestions. This results in higher code quality; with it implemented, we don’t need code reviews.
Allow developers to refactor. We should build a culture without asking permission to fix technical debt, especially if it can be done in a few hours.
Follow codebase standards, such as design, naming, and architecture. Then, automatize this through tools such as linters or your CI/CD process.
TDD (Test-Driven Development) is a software design process in which each functionality is designed by writing a test and then implementing it properly.
Write high-quality code. Educate your developers first to know their programming language in-depth and understand OO concepts, design patterns, refactoring, clean code principles, architecture patterns and styles, and SOLID, YAGNI, KISS, and DRY principles.
Continuous Delivery. Code is adjusted in short cycles, so it’s always easy for other team members to verify. This method avoids significant issues in production by employing frequent releases and fast feedback.
References
[1] “The WyCash Portfolio Management System.” Ward Cunningham, Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.
[2] “Software Developer Productivity Loss Due to Technical Debt, “Besker, T., Martini, A., Bosch, J. (2019).
[3] “Happiness and the Productivity of Software Engineers,” Daniel Graziotin, Fabian Fagerholm, 2019.
[4] “Technical Debt Quadrant,” Martin Fowler, 2009.
[5] “Conway’s Law,” https://en.wikipedia.org/wiki/Conway%27s_law.
[6] “Conway’s Law,” https://martinfowler.com/bliki/ConwaysLaw.html.
[7] “Team Topologies: Organizing Business and Technology Teams for Fast Flow,” Matthew Skelton, Manuel Pais, 2019.
[8] “The influence of Technical Debt on software developer morale,” Terese Besker, Hadi Ghanbari, Antonio Martini, Jan Bosch, Journal of Systems and Software, Elsevier, 2020.
Liked the idea of managing Tech Debt through "Tech Debt Map", thanks for sharing that.