The Tech Debt Rewrite Trap: Why Starting from Scratch Usually Backfires

The Ultimate Temptation: Tear It All Down and Start Over

Every engineer who has wrestled with a legacy codebase has had the thought: rather than patching this rotting pile of code, why not scrap it entirely and build something clean from scratch? The idea carries powerful emotional appeal — a greenfield project, no historical baggage, a fresh start. Yet in a comment on the Lobste.rs tech community, veteran engineer Simon Willison drew on personal experience to lay out a brutal truth: full system rewrites succeed at a surprisingly low rate.

This isn't a personal bias — it's a pattern that has been validated repeatedly across the software engineering industry. Understanding why it happens, and what more viable paths forward exist, has real practical value for any team.

The Three Death Spirals of a System Rewrite

The Old System Becomes a Moving Target

The moment you declare the old system "too debt-ridden to save" and kick off a rewrite, the first problem arrives immediately: the old system doesn't stop running. It still carries the core business, still needs to respond to new requirements. Meanwhile, the developers maintaining it know their work is about to be replaced, so they naturally tend to do the bare minimum for new features. Rather than halting, technical debt accelerates.

To make matters worse, the fact that the old system keeps humming along in production continuously erodes management's patience with the rewrite project. After all, a system that's "broken" but still running makes a more convincing case than a replacement that's "under construction."

The New Team Underestimates Hidden Complexity

The team tasked with the rewrite is often the most ambitious engineers in the company. Early progress on the greenfield project is fast and everything feels great. But as time passes, a harsh truth emerges: no one truly understands the full behavioral boundaries of the old system.

Willison put it plainly: if the old system had solid documentation and tests, it wouldn't need to be rewritten. Precisely because it lacks those things, the new team will continuously step on hidden landmines — an edge case handled in a non-obvious way, a business rule passed down only through word of mouth among senior staff, a bug fix that only triggers in a specific timezone.

Compromise Under Delivery Pressure

After months or even years of delivering no visible value, pressure from management spikes sharply. The new system ultimately ships in an awkward state: it handles only a subset of the old system's functionality, or it's used only for some new feature that was "too hard to build in the old system."

The result is the nightmare scenario Willison describes: you end up with two production systems — a legacy system nobody wants to touch, plus a new system that handles a small slice of functionality and whose codebase is 80% code "intended for replacing the old system someday." If the company is unlucky and "strategic priorities shift" before the full migration is complete, both systems coexist permanently, doubling overall complexity.

A More Viable Alternative: Incremental Migration

Will Larson's Migration Framework

The authoritative reference Willison recommends is Will Larson's article Migrations: the sole scalable fix to tech debt. Larson's core argument is that the fundamental path to resolving technical debt is not a rewrite, but rather planned, verifiable, incremental migration.

The key to this framework is breaking migration work into a series of small, independently deliverable steps — each one leaving the system slightly better than before — rather than betting everything on a single big-bang replacement. Every migration step needs clear success criteria and a rollback plan.

Cover with Tests First, Then Refactor

Based on his own experience, Willison offers concrete advice: cover the existing system with as many automated tests as possible, then bring it to the desired state through targeted, incremental refactoring.

The logic here is straightforward: tests aren't just a quality assurance tool — they're precise documentation of system behavior. When you start writing tests for a messy legacy system, you force yourself to genuinely understand every behavior it has, including the "weird logic" that nobody remembers the reason for. With test coverage in place, refactoring transforms from a gamble into a controlled engineering process.

Compared to a full rewrite, incremental migration offers several distinct advantages:

• Continuous value delivery: Every refactoring step directly improves the production system — there's no long waiting period for "when the new system launches"
• Controlled risk: Tests provide a safety net, and every change can be verified immediately
• Knowledge retention: The refactoring process itself forces the team to deeply understand the old system, rather than work around it
• Avoiding the dual-system trap: There's always only one production system to maintain

When Is a Rewrite Actually Justified?

Of course, this doesn't mean rewrites are never the right call. There are a few situations where one may be justified:

The tech stack can no longer support business needs: For example, when the runtime or framework the system depends on is no longer maintained, security vulnerabilities can't be patched, and incremental migration is technically infeasible.

The codebase is very small with clear boundaries: For a service with a few thousand lines of code and a single, well-defined responsibility, the risks of a full rewrite are manageable. The "never rewrite" stance Joel Spolsky argued in his classic essay Things You Should Never Do was aimed primarily at large, complex systems.

A thorough test specification exists to drive the rewrite: If the old system has a comprehensive test suite, or business rules are clearly documented, the new system can use those as acceptance criteria — and rewrite success rates improve dramatically. But this is precisely the point: good tests and documentation are a prerequisite for everything.

The Core Judgment for Technical Leaders

For technical leaders, the "should we rewrite?" decision is rarely just a technical question — it's also an organizational and prioritization question. Rewrite projects require sustained resource investment, require maintaining management trust through long stretches with no visible progress, and carry coordination costs during the period when two systems run in parallel.

Before making this call, it's worth honestly asking yourself a few questions: Do we truly understand the full behavioral scope of the old system? Have we covered the critical paths with tests? Is incremental refactoring genuinely impossible, or are we just reluctant to do the tedious work of writing tests for legacy code?

Willison's insight serves as a reminder: the impulse to tear everything down and start over usually comes from exhaustion with the existing system, not from a deep understanding of the problem. Truly responsible technical decision-making means thoroughly evaluating the feasibility of incremental improvement before reaching for more drastic options. Technical debt can be paid down systematically — but what that requires is patience and discipline, not a high-stakes bet on a complete rewrite.