Merging 10 Legacy Repos into a Monorepo: An AI Coding Assistant Retrospective

At the AI Engineer conference, an engineer from WiseDocs, a medical technology company, shared a real and somewhat controversial engineering decision: spending six months restructuring 10 repositories — each more than six years old — into a single Monorepo, with AI coding assistants used throughout the entire process. Was the restructuring worth it? How reliable are AI coding tools in real-world, large-scale refactoring? This retrospective offers remarkably pragmatic answers.

Why Undertake This Monorepo Migration

The speaker's company primarily processes complex medical claims documents — some PDFs exceed 10,000 pages and are even larger than video files. The entire processing pipeline involves multiple machine learning models, and scaling any part of it is no easy feat.

Business growth hit three core bottlenecks: First, delivery speed couldn't keep up with customer demand; second, the AI pipeline was too complex to update; third, a legacy codebase spread across more than 10 repositories made nobody want to touch the code, creating a terrible developer experience.

The speaker compared technical debt to financial debt: it compounds in mysterious and sometimes inexplicable ways. The metaphor of Technical Debt was coined by Ward Cunningham in 1992, likening shortcuts taken in software development for short-term speed to financial borrowing. But unlike financial debt, technical debt's "interest rate" is often hidden and nonlinear: a temporary hack might have zero impact initially, but as system complexity grows, it entangles with other debts to create exponentially increasing maintenance costs. Martin Fowler further categorized technical debt into four quadrants: reckless/prudent × deliberate/inadvertent. Deliberate and prudent technical debt (such as clearly documented temporary solutions) represents a reasonable engineering trade-off; inadvertent and reckless debt (such as messy code resulting from unfamiliarity with design patterns) is pure burden. WiseDocs' six years of accumulated multi-repo legacy code had clearly evolved from manageable debt into a systemic risk that was suppressing business growth.

There's nothing inherently wrong with a company taking on technical debt in exchange for business ROI (such as shipping new features or landing new customers), but once the rate of introduced complexity outpaces the rate of ROI growth, the entire system spirals out of control. He cited Anthropic's case studies on Spotify and Stripe, pointing out that these companies had achieved tremendous improvements in both delivery speed and refactoring capability.

But he also raised a sharp observation: while tech products have indeed become more powerful over the past 20 years, in the rush to "ship faster," we've lost something — customer-centric product polish, code maintainability, and reliability have all been degrading. He showed availability data from two leading companies falling below 99.9% or even 99.999%, demonstrating that faster delivery doesn't automatically mean higher quality.

Monorepo vs. Polyrepo: Two Architectural Paradigms

To understand the significance of this refactoring, you first need to understand the two code organization paradigms: Monorepo and Polyrepo. A Monorepo is an engineering practice where all of an organization's projects, services, and libraries are stored in a single version control repository. Tech giants like Google, Meta, and Microsoft have long adopted this model. In contrast, the Polyrepo model gives each service or module its own independent repository. Monorepo advantages include easier code sharing, atomic commits for cross-project refactoring, and unified dependency management; disadvantages include repository size bloat, increased CI/CD complexity, and coarser permission management granularity. In recent years, the maturation of build tools like Nx, Turborepo, and Bazel has significantly reduced the management overhead of Monorepos, enabling small and mid-sized teams to benefit from this architecture as well. For WiseDocs, the cross-repository coordination costs of 10 independent repos had severely slowed the team's iteration speed.

How Fast Has AI Model Capability Evolved in 18 Months

The refactoring officially kicked off in April, with preparatory work beforehand. The team spent roughly two months evaluating orchestrators for their AI pipeline, comparing 5 open-source projects and scoring them against 17 criteria — at the time, Google and OpenAI's Deep Research hadn't been released yet, and there was no systematic web search capability, so all research had to be manually compiled into Confluence documents.

The speaker admitted: If done today, the same research could be completed 90% faster. With today's agentic workflows, starting from Deep Research, creating subagents for each evaluation criterion and candidate product, then automatically building and evaluating POCs — the efficiency is in a completely different league. Agentic workflows represent one of the most important paradigm shifts in AI engineering since 2024 — unlike traditional single-turn Q&A, AI agents in agentic workflows can autonomously plan tasks, invoke tools (such as search engines, code executors, file systems), spawn subagents for subtasks, and verify results through multiple iterations. The technical foundations of this paradigm include function calling capabilities, long context windows, and reasoning-action frameworks like ReAct. Deep Research is essentially such an agentic search system — it can automatically decompose research questions, retrieve from multiple sources in parallel, cross-validate findings, and generate comprehensive reports.

However, the speaker also warned against "AI psychosis" — seeing a 20-page Deep Research report and taking it at face value, only to discover that product features described in the report don't actually exist, ultimately dragging down the project. This risk is particularly acute in agentic systems: an agent's confidence level often has no correlation with its accuracy. They excel at generating fluent, well-structured reports but may "hallucinate" nonexistent API endpoints or feature specifications on critical details.

The most compelling evidence came from a model capability comparison experiment. Initially, using O3 for Temporal-related refactoring took three hours of back-and-forth conversation, produced 10 major errors, and required constant manual intervention — guiding the model and manually editing code throughout. Temporal, mentioned here, is an open-source distributed workflow orchestration engine that originated from Uber's internal Cadence project. Its core idea is to let developers write complex, long-running workflows in ordinary code without manually handling state persistence, retries, compensation, and other classic distributed systems challenges. In medical AI pipeline scenarios, processing an oversized PDF might involve chained calls to multiple ML models, asynchronous waits, and failure retries — Temporal orchestrates these steps into reliable workflows, ensuring recovery from any node failure. This also explains why Temporal-related refactoring code was challenging for models — it requires deep understanding of distributed state machine semantics.

When the same task was rerun with modern models: Sonnet 4.6 solved the problem with just one additional iteration, and Opus got it essentially right in one shot.

More importantly, the interaction pattern had changed. In the O3 era, tool calls were rare, whereas modern harnesses automatically spawn subagents, generate plans, execute shell commands, and perform multi-round validation. While the per-execution cost is slightly higher, human intervention is dramatically reduced. The speaker estimated: The same refactoring task now takes only one-fifth of the original time.

Building the Right Mental Model for AI Coding Capabilities

Stronger models have profoundly changed how we think about the software development lifecycle. Previously, people were feeding models specific code snippets and making small edits; now, as long as you provide a well-structured spec, models can typically execute to high quality.

The speaker particularly emphasized the correct interpretation of METR's famous "task duration" curve. METR (Model Evaluation and Threat Research) is a research organization focused on AI capability assessment, and their "task duration" benchmark has become an important reference for measuring AI coding agent capabilities. The test collects a large number of real software engineering tasks, grades them by the time a human would need to complete them (ranging from minutes to tens of hours), then has AI agents independently complete these tasks while tracking success rates. This evaluation method is far more representative of real engineering scenarios than traditional code completion accuracy metrics.

The industry commonly cites the 50% success rate version, claiming models can complete tasks that would take humans 18+ hours; but the speaker argued that we should focus on the data at 80%, 90%, or even 99% success rates — that's where the most effective mental model lies. The critical distinction: 50% success rate means the model "might" complete the task, while 99% success rate means engineers can actually "trust" and "delegate" to the model. This gap is the core bottleneck in AI coding tools' transition from "assistive" to "autonomous."

His logic is practical: if you kick off an agent at 11 PM for a task that takes an hour to run, and it only has a 50% success rate, you've likely wasted both that hour of compute and your attention. What you want is to build a solid plan and spec, hand it to the agent, and have reasonable confidence it will deliver. Using Mythos Preview as an example, success rates begin to drop significantly at the 4-hour task mark, and even for short tasks under 15 minutes, there are certain categories the model can't reliably complete.

The conclusion: AI is advancing rapidly, but we're still far from the point where you can "fire up an agent and get reliable delivery." For engineering teams and every IC (individual contributor), understanding how to contribute within this system is crucial. This also requires a complete framework and set of primitives to support high-quality agentic development and avoid falling into an endless "doom loop" with the model — a vicious cycle where developers continuously submit change requests, the model repeatedly generates flawed code, and both sides burn massive amounts of time in error-correction loops with no real progress.

Real Productivity Gains from the Monorepo Migration

The core action was straightforward: merge 10 repositories into one Monorepo and build new features on top of it. The results were impressive.

The previous repositories had been around for over six years with slow progress — partly due to accumulated technical debt, partly because AI coding tools didn't exist yet. In the first six months after the rebuild, just to reach feature parity, the code growth curve was already strikingly steep — and it didn't slow down after achieving parity. The team continued adding new features, with both code volume and commit frequency far exceeding previous levels.

Even more important was the change in participation: more and more developers joined in contributing. Today, nearly every developer in the company is committing code to this new Monorepo, even when changes fall outside their area of expertise — such as modifying schemas, API calls, or other parts of the tech stack. This phenomenon validates a core advantage of Monorepos: when all code lives in the same repository, the psychological and technical barriers to crossing module boundaries drop significantly. Developers no longer need to clone another repo, figure out another build configuration, or submit coordinated cross-repo PRs.

The speaker mentioned an easily overlooked soft benefit: Developers actively want to work in this cleaner codebase. People voluntarily ask "Can I work in this repo?" and the patterns established during this refactoring have spread to other repositories across the company. This improvement in Developer Experience (DX), while hard to quantify, has a profound impact on team morale and talent retention — when the codebase is well-structured and the toolchain runs smoothly, engineers can devote more energy to solving business problems rather than fighting build systems and code rot.

Can Modern LLMs Handle Large-Scale Refactoring in One Shot

The speaker ran a bold experiment: having GPT-5.5 (extra high) perform a zero-shot refactoring of the entire codebase, providing only repository names and information about the underlying models and components.

The model claimed to have "completed the objective" in 10 minutes and 22 seconds, but only wrote 2,000 lines of code — a suspicious number. Digging deeper revealed that it only built scaffolding without actually implementing the model components, even admitting in its own comments that "Ray Serve deployment and bootstrap commands have not yet been added." Ray Serve is a model serving component built on the Ray distributed computing framework, developed by UC Berkeley's RISELab team. It allows developers to deploy machine learning models as scalable online services via HTTP endpoints, supporting model composition, dynamic batching, and auto-scaling. In scenarios like WiseDocs, where multiple ML models need to be chained together to process oversized files, Ray Serve provides a unified deployment and resource scheduling framework. GPT-5.5's skipping of this implementation illustrates precisely that AI models still struggle with infrastructure integration layers — the "glue code" involving distributed system configuration, environment variables, and process management.

This demonstrates: Current models still can't self-validate and complete large-scale refactoring in a single pass, but they're getting close. The speaker predicted that within roughly six months, models will be able to reliably complete refactoring at a scale comparable to the Stripe case.

Was This Monorepo Migration Actually Worth It

Faced with challenges like "shouldn't you have waited another year for models and harnesses to mature before refactoring," the speaker gave a nuanced answer.

The counterargument has merit: models keep getting stronger, tool calling keeps improving, sandboxes and monitoring frameworks keep getting better — the cost of carrying technical debt and refactoring later is dropping exponentially.

But the risks are equally real: under AI-native development practices, if you let generation run unchecked, it's easy to pile up large volumes of low-performance, low-quality code that nobody truly understands — which is fundamentally no different from legacy code. When problems arise or customer needs require adjustments, it becomes even harder to intervene. Therefore, whether doing a full or partial refactoring, reasonable guardrails must be in place. In the AI coding context, guardrails include both technical measures (such as type checking, automated test suites, lint rules, CI gates) and process-level constraints (such as mandatory human code review, spec review before implementation, and diff review of agent output). Without these guardrails, AI-generated code might pass surface-level functional tests while planting seeds of performance issues, security vulnerabilities, or architectural decay.

The speaker's final answer was yes, it was worth it. The team first used the multi-repo approach to quickly meet customer needs and achieve business goals, then circled back to refactor and significantly accelerated. The concrete benefits from the refactoring included: reduced pipeline processing time, lower costs, support for larger files, and features that previously took months can now be delivered within a week.

During the Q&A session, he added several pragmatic details:

• Polyrepo vs. Monorepo: Modern models are already quite good at navigating across multiple repos (as long as they're placed under the same parent directory), but end-to-end testing, validation, deployment, and running a complete "AI factory" in a sandbox remain more cumbersome with multiple repos — cloning and environment setup take more time.
• Requirements accuracy: During the refactoring, 15 out of 17 requirements were met, and the validation process continued to evolve — for example, plan mode had just been introduced to Claude Code at the time and Cursor didn't have it yet, but the team later incorporated it into their development lifecycle. Plan mode here refers to having the AI agent output a structured execution plan for human review before writing any code, confirming the direction is correct before entering the implementation phase, thereby preventing the agent from generating large amounts of useless code in the wrong direction.
• Code review: All PR reviews during the refactoring were done manually, supplemented with local skills for code checks. PRs were one of the few effective ways for developers to build repository context and understand refactoring details at the time.

The core takeaway: Models will keep getting stronger, but sometimes pausing to build a proper Monorepo, clean up technical debt, and then push forward aggressively is still the right engineering decision. The key lies in rigorously evaluating the business value of large-scale refactoring and weighing the trade-offs between "doing it now" versus "doing it later."