The reducer: a new agile role

With very few exceptions, the codebases you’ll encounter in paying jobs are big. A more direct to say it is this: most paying codebases are complex. Very complex, judging by my experience and standards.

What qualifies as a complex codebase?

  • Lines of code are measured in the millions.
  • There’s dozens of direct dependencies.
  • Bonus points: several programming languages.

But the real measure is more subtle. Looking closer, it can best be seen in the nature of what devs do every day. In a complex codebase, developers and testers spend most of their time struggling with the existing codebase. Struggling with the existing codebase means that:

  • Easy, even trivial features, take days of work to retrofit in the existing codebase, even when the most senior devs are working on them.
  • Certain sections of the code are untouchable and devs work around them.

The sad reality is that most codebases converge to this state in a short amount of time. It happens in just a few years, and it also happens to startups. We still lack the conceptual technology for writing non-trivial systems without incurring in horrid complexity by default.

This article is written for devs that are aware of this, and yet they need to work in these codebases for their own sustenance. You’re not alone. If you are aware that most complexity is avoidable, and that most of your team’s effort is a wasted effort against a coding quagmire, you are presented with the following options:

  • Find another company with codebases that are more tolerable.
  • Find solace in your own projects, where complexity is actually contained, and try to make money from them instead.
  • Plod along, trying to incrementally improve your company’s codebase. Resist against the introduction of new paradigms or tools that will increase the complexity instead of decreasing it. Carve sections of the codebase where you have ownership and where you can decrease its complexity.
  • Propose a big refactor

The first two are fine choices, but often not enough. It’s hard to find companies that are as mindful of complexity as you. And building a business is no easy task either, especially if you’re on your own.

The last two are also good, but they seem to represent the extremes of continuity and revolution. Plodding along can be a wonderful way of bringing your morale down to zero. And refactoring the entire code entails a large responsability and enormous pressure – it can be done, but usually will not work unless there’s wise dev leadership in your company, because you need to be protected from these pressures to make the refactor work. And if there was such wise leadership in the first place, the complexity probably would have been contained more.

Note: no approach, paradigm or framework will save you. If this was possible, it would already have happened.

I am hinting at another option, which is to propose a new role: the reducer. The reducer is someone whose work is to take an existing codebase and incrementally work on it to reduce its overall complexity while maintaining its functionality. The reducer is someone who is constantly refactoring the code, piecemeal but making fundamental architectural and stylistic changes to the codebase.

The measures of the success of a reducer is:

  • Decreased lines of code.
  • Decreased use of libraries.
  • More tests pass and the overall number of bugs is reduced.
  • The time taken to deploy new features is reduced.
  • Onboarding of new devs becomes quicker and easier.
  • All devs have a more complete mental map of the application.

Note that a reducer is not writing new features, nor fixing bugs (although both might happen as a secondary result of their work). The strict mission of a reducer is to reduce the complexity of a codebase, while maintaining its functionality.

A reducer is someone that’s draining the swamp in which most codebases are mired. When the swamp starts to be drained, stuff gets done easier and less people die of malaria and attrition. Morale and velocity improve. Dread decreases.

But if you want to sell the idea, it’s easier to say that a reducer brings down the technical debt of the codebase while not blocking the team (as a refactor would).

And to drive the point home: the factor that determines the speed of a dev team is complexity. A simple system allows a small team to go very fast. A complex system will slow down the brightest and boldest, as the war stories from Microsoft verify.

If you’re a team lead, or a manager, and want to increase the speed of your team, don’t hire more devs. Make your codebase simpler.

Note: automation won’t save you. Adding automation to a complex system will only create more chaos. This was lucidly determined by Taiichi Ohno, the father of the Toyota Production System.

The reducer can work alone, or in pairs. It can be part-time, or full-time. I think that not everyone can be a reducer, and I intuitively feel that rotation the role of reducer doesn’t make sense either. But a reducer works closely with the other devs, because they are the source of the knowledge of the system, and they will be the ones that will either thrive or suffer from the consequences of the reduction. The team itself needs to choose their reducer or reducers.

Very recently I realized that within organizations, most of my work as a programmer has been in fact as a reducer. I’ve done my best work by taking existing systems and making them simpler. I can only understand complex systems only when I have the initiative and permission to simplify them – but my cognitive tolerance for complexity vanishes as soon as I’m told that I have to contribute to an existing codebase using the same practices that have made it a quagmire.

I don’t think I’m alone in this respect.

Now, why do I propose the reducer as a role within agile, if I don’t like agile? Because most jobs and paying gigs nowadays use agile, and this is meant to bring about change within the context of existing organizations. I realized the chances I land a paying project that’s greenfield are low – but I might pull off convincing the team on letting me work in the direction of simplicity.