Technical Debt

This document sets the process by which we will improve visibility of technical debt and how it will be monitored moving forward.

Problem Description

When our codebase was young, understanding what the code was doing and adding new features was fast and straightforward, but as time passed by, and because there were no procedures in place for handling technical debt, technical debt started to accumulate. As technical debt increased this caused many issues like:

  • Making adding new features slower or even impossible at some times
  • Increasing the number of bugs in the codebase
  • Increasing the number of incidents.
  • Bugs and incidents take time away from the developers, time that could be used to add new features and provide more value to the business

All of these resulted in work estimation getting harder meaning projects are most likely not going to be completed on time, which causes developers to potentially add even more technical debt in order to meet the deadlines.

As the technical debt increased, it got to a point in which developers started to fear changing the code (e.g., New Deal in BOS). When this happened, there were no longer any attempts to improve the quality of the codebase, typically because only the minimum necessary changes were performed in order to avoid breaking the system in unexpected ways.

On top of the aforementioned problems, technical debt also made onboarding times longer than needed.

Background

Technical debt is a metaphor coined by Ward Cunningham in 1992 to talk about the trade-offs that development teams make in order to be able to deliver value to the business at the expense of quality, which ends up costing the business more in the long term.

There are two major types of technical debt: the one we take on purpose, knowing the consequences of doing so, and the one we take on unintentionally.

For the unintentional case, we might incur debt because of ignorance of good design and development practices, but we can also incur debt incrementally, even after applying good design and development practices, because the changing requirements of the project or even the deeper understanding of the business domain and issues, that arises during development, might make the team realize there is a better way to do the same thing they have done already.

Solution

This RFC focuses only on giving visibility to technical debt and how it will be monitored. It is the responsibility of the teams to effectively plan delivery of technical debt items along with their Engineering Managers and working with their relevent product owners to ensure the correct prioritisation is applied.

Whenever we talk about technical debt, we are going to be referring to intentional technical debt and unintentional debt generated even by following good development and design practices.

For handling the unintentional debt that is generated by ignorance of good development and design practices, the best solution is having a continuous learning culture in place, but that is outside the scope of this RFC.

In order to allow teams to keep a steady development speed in the long term, the technical debt must be handled continuously, as the more technical debt that we have the harder it is to fix it.

In order to be able to fix the technical debt, first, we need to be able to tell how much is our debt. In order to do this it is necessary for all teams to consistently capture identified Technical Debt items.

Finding technical debt is everyone’s responsibility, so every developer must be encouraged to find places in their codebases that can be improved and create the necessary tickets to make it happen.

A new type of ticket will be made available to all team's Jira projects to be able to track technical debt. This will be called Tech Debt and will give us more flexibility in the future in terms of fields and available transitions.

Using JIRA for this purpose has the following benefits:

  • It can be planned and estimated in the same way we estimate other types of tickets.
  • We can associate commits to technical debt tickets
  • We can link tech debt tickets to other tickets
  • Developers and product owners already have access to JIRA so anybody can create a ticket.
  • We can reuse our current tooling to calculate metrics based on tech debt tickets.

Future Changes

Our intention is to be able to examine the known Technical Debt and see how much we have amassed by service. To do this we will utilise the list of components available in Jira and align it the services as defined in the Service Ownership list. When Technical Debt is created we will then apply the component (service) to the ticket and then have a method to report on it at a more granular level.

This additional change will be applied in the near future and this RFC will be updated to make this part of the process and responsibilities. This will include the additional actions;

  • The associated component(s) should be added to the ticket at the point of refinement and prior to it being moved into a planned iteration.
  • The creation of an exception report to list all completed technical debt WITHOUT an assigned component for a given period.

Exclusions

As described previously, in some scenarios Technical Debt is purposefully generated to overcome a challenge in the delivery of a project. This is most likely some corner being cut to speed up the implementation of business benefits at the cost of agreed best practices.

This is a reasonable decision and we would expect this to have happened at the design / RFC stage of the project in the majority of cases. When Ebury incurs Technical Debt as part of a project in this way then this is a commercial decision and we would not define this Technical Debt in the same way as other cases and should not be identified as such. The removal of this “project” debt would be a subsequent phase of the overall project delivery and the related tasks defined as standard issues in those phases.

Roles and responsibilities

Team Leads

  • Based on the current aligned services in the service ownership register, migrate items from the previous technical debt project to the relevant teams.
  • Move all identified technical debt labelled items to the new Tech Debt ticket type as well as historically completed items since, and including, Q1 2022/2023.
  • Exception reporting should be monitored by the team leads per iteration to ensure the quality of the data for performance and financial reporting which will be easily accessible to the PSO.

Engineering Teams

  • When a technical debt item is identified a new technical debt ticket is created and added to the backlog.
  • When planned the ticket should also be placed in a quarterly team technical debt epic based on the Ebury financial calendar.
  • Technical debt should be planned for a minimum of 10% of overall quarterly capacity where there is technical debt in the teams backlog.

PSO

The following information should be consumable from dashboards that are created to monitor and measure technical debt within the tech teams within Jira;

  • List all completed technical debt items for a given period.
  • List all completed technical debt items per component / service for a given period
  • List all completed technical debt items per team for a given period
  • Exception report to list all completed technical debt WITHOUT an assigned epic for a given period.
  • List all outstanding technical debt.

Alternatives

There are some specialized tools for tracking technical debt like Stepsize starting at 15USD per developer.

Using labels to identify Technical Debt

Caveats

N/A

Operation

All developers and product owners have access to JIRA, so everybody will be able to create, and track the technical debt for a project.

Security Impact

N/A

Performance Impact

N/A

Developer Impact

There should be no changes to the identification of Technical Debt performed by engineers apart from the change of ticket type at this point of time although an increase in the information prior to planning the ticket will be required in the future (e.g. associated component).

Data Consumer Impact

N/A

Deployment

Techical Debt items should follow the same principals of the existing SDLC.

Dependencies

None

References

None