AI Migrating COBOL to Java: Why It Copies the Bugs Too

When AI Meets 60 Years of COBOL History

In the world of enterprise IT, COBOL is both ancient and indispensable. This programming language, born in 1959—designed under the leadership of U.S. Navy Rear Admiral Grace Hopper and others, originally intended to create a business data processing language close to natural English—still quietly powers the core systems of banks, insurance companies, and government agencies worldwide, processing trillions of daily transactions. According to a Reuters survey, approximately 220 billion lines of COBOL code are still running globally, with 95% of U.S. ATM transactions and 80% of in-person financial transactions relying on COBOL systems, processing $3 trillion in business transactions daily. However, as engineers proficient in COBOL grow increasingly scarce—the average age of skilled COBOL developers worldwide exceeds 55, with many already retired or approaching retirement—how to migrate these legacy systems to modern technology stacks (such as Java) has become the number one challenge for countless enterprises.

Recently, a discussion on Hacker News caught the industry's attention: a team attempted to use AI to automatically migrate legacy COBOL programs to Java, with unexpected results—the AI not only migrated the business logic but also carried over the original bugs. This seemingly absurd phenomenon actually reveals profound technical and philosophical issues behind AI code migration.

"Faithful" Migration: Functional Equivalence or Defect Replication?

Why AI Migrates Bugs Along with Code

From a technical perspective, the core objective of AI code migration tools is typically behavioral equivalence—meaning the migrated code produces the same output as the original code given the same inputs. Behavioral equivalence is a core concept in formal methods, originating from the fields of process algebra and program semantics. In code migration scenarios, it requires that for all possible input sets, the migrated program produces output sequences, side effects, and state changes completely consistent with the original program. This is more rigorous than simple "functional equivalence" because it also requires error handling paths, boundary behaviors, and non-functional characteristics to remain consistent. Mainstream AI migration tools such as IBM's Watsonx Code Assistant for Z and AWS Mainframe Modernization typically employ LLM-based code translation combined with automated test verification to approximate this goal.

This objective sounds impeccable, but the problem lies precisely here. If the original COBOL program contains a bug—say, an incorrect boundary condition, a precision calculation deviation, or a flawed exception logic—then a "faithful" migration tool treats these behaviors as part of the intended behavior and precisely reproduces them in the Java code. For AI, it cannot easily distinguish between "features" and "defects"—because without original requirements documentation, the code itself is the sole source of truth.

The Core Paradox of Migration Work

This creates the central dilemma in COBOL-to-Java migration:

• If AI fixes what it considers bugs, it may break downstream systems that depend on those buggy behaviors. In software engineering, this phenomenon is known as "Hyrum's Law": when an API has enough users, all observable behaviors of that API—whether explicitly defined in the specification or not—will be depended upon by some users. In COBOL legacy systems, this phenomenon is particularly severe. For example, a bank's interest calculation module might undercount by $0.0001 due to a rounding error, but after 30 years of operation, reconciliation systems, reporting modules, and tax calculation systems have all adapted to this "incorrect" value. If rashly fixed, it could trigger system-wide cascading failures or even regulatory compliance issues.
• If AI preserves all original behaviors, then these historical defects are carried wholesale into the new system, making the migration merely a "technical debt transfer" in a different language.

In other words, AI migration tools face not just a technical problem but a judgment challenge about "what is correct."

Deeper Challenges in Legacy System Migration

The Harsh Reality of Code as Documentation

Many COBOL systems that have been running for thirty or forty years have long lost their original design documentation, and the initial developers have retired or passed away. In such cases, the code itself becomes the only trustworthy "specification." Any effort to understand the system's true intent must begin with reverse engineering the existing code.

This means that during migration, AI is effectively translating a "specification" that contains errors. It has no omniscient perspective to judge whether a piece of logic was carefully designed or a historical mistake. In this situation, conservatively preserving all behaviors becomes the "safest" choice—at least it won't introduce new, unknown problems. It's worth noting that many COBOL systems have undergone decades of patch iterations, with developers from different eras leaving their marks in the code, forming intricate layers of stacked logic. Some seemingly redundant conditional branches were actually added to address a long-forgotten production incident; some apparently unnecessary data conversions may exist to maintain compatibility with an external interface that was decommissioned long ago. This "archaeological-level" code complexity far exceeds the contextual understanding capabilities of today's AI models.

The Precision and Semantic Gap Between COBOL and Java

COBOL and Java have fundamental differences in data type handling. COBOL is renowned for its fixed-point decimal arithmetic—through PICTURE clauses (such as PIC 9(5)V99), developers can precisely define the number of digits and decimal point position of numeric values, with all operations performed in decimal fixed-point format, eliminating the precision loss inherent in binary floating-point. This is crucial in financial calculations, enabling exact decimal handling without any rounding errors.

Java's float and double types follow the IEEE 754 floating-point standard and cannot precisely represent certain decimal fractions (for example, 0.1 is an infinitely repeating number in binary). Although Java provides the BigDecimal class for precise decimal arithmetic, its performance overhead is significantly higher than native floating-point operations, and its API usage style is fundamentally different from COBOL's declarative approach. During cross-language migration, the precision requirements and rounding rules for each numeric variable must be individually verified. These underlying semantic differences can cause the old and new systems to produce inconsistent results in edge cases, further amplifying migration risk. A seemingly trivial precision difference can accumulate into massive accounting discrepancies in financial systems processing billions of transactions daily.

Rational Thinking About AI-Assisted Code Migration

AI Is Not a Silver Bullet, But an Accelerator

This case offers an important reminder to the industry: while AI code migration tools can significantly improve efficiency, they cannot replace human domain knowledge and judgment. Migrating COBOL to Java is not fundamentally a pure translation task—it's a complex engineering endeavor involving business understanding, risk assessment, and architectural decisions.

The ideal migration process should be human-AI collaborative: AI handles the heavy, repetitive syntactic conversion work, while human experts are responsible for identifying which historical behaviors need to be preserved, which defects should be fixed during migration, and rigorously validating critical business logic. Currently, more mature modernization methodologies in the industry include: the Strangler Fig Pattern, which gradually replaces old functionality with new services rather than migrating all at once; the API wrapping pattern, which encapsulates COBOL programs as microservice interfaces for new systems to call; and the complete rewrite approach. With AI assistance, some leading enterprises adopt a "three-phase" strategy: first using AI for initial code translation, then having engineers with business domain knowledge conduct logic review and bug annotation, and finally verifying migration quality through shadow running—where old and new systems run in parallel and their outputs are compared. While this approach takes longer, it significantly reduces migration risk.

Post-Migration Verification Is Equally Critical

The phenomenon of "migrating bugs along with code" also emphasizes the importance of comprehensive testing. Enterprises cannot rest easy simply because AI claims to have achieved "behavioral equivalence." Instead, after migration is complete, a robust regression testing framework must be established, using extensive real data comparison and verification to ensure the new system's behavior meets current business requirements—rather than blindly replicating potentially defective old behaviors.

In practice, this means enterprises need to build test sets containing millions or even billions of real historical transaction records, execute them in parallel on both old and new systems, and compare outputs transaction by transaction. Any discrepancy must be adjudicated by business experts: is it a new error introduced by AI migration, an old bug in the original system that should be fixed, or an acceptable deviation caused by semantic differences between the two languages? The workload of this verification process is often no less than the migration itself, but it is an indispensable component of ensuring migration success.

Conclusion: Technical Debt Cannot Vanish Through AI Alone

AI copying COBOL bugs directly into Java—this slightly ironic story fundamentally reflects an eternal truth in software engineering: technical debt does not automatically disappear just because you change languages or tools.

Legacy system modernization is a systems engineering effort filled with uncertainty. AI is a powerful assistant, but by no means an omnipotent savior. For enterprises considering using AI to migrate core systems, true wisdom lies in: embracing the efficiency gains AI brings while soberly recognizing its limitations, and using human professional judgment to safeguard the technology. Only then can migration truly become a system's "rebirth" rather than a defect's "reincarnation."