ProgramBench: An AI Evaluation Benchmark for Reverse-Engineering Program Logic from Runnable Binaries

A Blind Spot in AI Capability Assessment

When we evaluate the programming capabilities of large language models, the vast majority of benchmarks focus on a single direction: generating code from natural language descriptions. Whether it's HumanEval, MBPP, or SWE-bench, they all essentially test the "read requirements → write code" pipeline. HumanEval, released by OpenAI in 2021, contains 164 hand-written Python programming problems and measures the probability of a model generating correct code using the pass@k metric; MBPP, proposed by Google, includes approximately 1,000 entry-level Python tasks; SWE-bench uses real GitHub issues as test cases, requiring models to locate and fix bugs within complete code repositories. Together, these benchmarks form the mainstream framework for evaluating AI programming capabilities, but they share one commonality: they all provide explicit natural language intent descriptions as input.

However, real-world software engineering goes far beyond this. Reverse engineering, binary analysis, legacy system maintenance—these scenarios demand the exact opposite capability: reconstructing original logic from existing artifacts.

ProgramBench is a novel evaluation framework targeting precisely this blind spot. Its core concept is highly disruptive: instead of providing the model with a natural language specification, it supplies a runnable binary program and requires the model to reverse-engineer equivalent source code or program logic by observing its behavior and input/output patterns. This ability to "infer process from results" is a dimension that current mainstream benchmarks have almost entirely failed to cover.

Why "Runnable Binaries" Are Key to Evaluation

The Core Pain Points of Traditional Reverse Engineering

Reverse engineering has always been one of the toughest challenges in software security, vulnerability discovery, and legacy system migration. Human engineers need tools like disassemblers, dynamic debuggers, and symbolic execution engines, spending enormous amounts of time to understand what a program without source code actually does. Specifically, disassemblers (such as IDA Pro, Ghidra) convert machine code into human-readable assembly language; dynamic debugging tools (such as GDB, x64dbg) allow step-by-step program execution while observing registers and memory states; symbolic execution is a more advanced program analysis technique that uses symbolic values instead of concrete inputs to systematically explore all possible execution paths. Even with these tools combined, an experienced reverse engineer may need days to weeks to analyze a moderately complex binary program. This process is highly experience-dependent and difficult to scale.

ProgramBench uses "Runnable Binaries" as its evaluation starting point because it simulates the most realistic information constraints in reverse engineering—you can run it and observe it, but you cannot see its design intent. The model must act like a real reverse engineer, inferring the program's internal structure and algorithmic logic through repeated interaction and experimentation.

The Capability Leap from Code Generation to Code Understanding

Reverse-engineering logic from a runnable program tests deeper reasoning capabilities in models:

• Behavior modeling: Given that specific inputs produce specific outputs, the model must inductively determine the underlying mapping rules;
• Boundary reasoning: By constructing edge cases and anomalous inputs, the model probes the program's branching and exception-handling logic;
• Equivalence verification: The reconstructed code must be functionally equivalent to the original program, not merely superficially similar.

This is fundamentally different from "writing code based on a requirements document." The former is divergent, exploratory reasoning, while the latter is more of convergent pattern matching. The value of ProgramBench lies in isolating and quantitatively evaluating this long-overlooked capability.

The Vetted Review Mechanism: Design Considerations for Evaluation Quality

The word "Vetted" in the title hints at the benchmark's meticulous attention to data quality. Evaluation of reverse engineering tasks is extremely prone to "false passes"—a model might get some examples right through superficial pattern matching without truly understanding the program logic.

A rigorously vetted benchmark needs to address several key issues:

Preventing Training Data Contamination

If the source code corresponding to a binary has appeared in training data, the evaluation becomes meaningless. Data contamination is one of the central challenges in LLM evaluation. Research has shown that training data for models like GPT-4 may contain test problems from benchmarks such as HumanEval, leading to inflated scores. For reverse engineering tasks, this problem is even more thorny: source code and compiled artifacts from vast numbers of open-source projects on GitHub may have been crawled into training corpora.

The vetted process must ensure that test samples are truly "unseen" by the model. Solutions include using new code written after the training data cutoff date, manually writing entirely new programs, or applying sufficiently deep mutations to existing programs to move them beyond memory-matching range. This is particularly important for reverse engineering tasks, as many open-source programs have long been included in training corpora.

Clear Correctness Criteria

Unlike generation tasks that can be judged by unit tests, the definition of "correct" in reverse engineering tasks is more nuanced. In computer science, program equivalence has different levels of definition: syntactic equivalence requires identical code structure; semantic equivalence requires identical output for all possible inputs—which, according to Rice's theorem, is an undecidable problem. Should line-by-line equivalence be required, or functional equivalence? Are different but equally correct implementations acceptable?

ProgramBench uses runtime behavior comparison (same input produces same output) for functional equivalence determination. This is similar to equivalence class partitioning and boundary value analysis strategies in software testing—a relatively objective and automatable engineering compromise.

Implications of ProgramBench for AI Programming Capability Assessment

Completing the AI Programming Evaluation Landscape

As model programming capabilities rapidly improve, scores on benchmarks like SWE-bench are gradually saturating, and the industry urgently needs more discriminating evaluation dimensions. ProgramBench approaches the problem from the entirely new angle of "reverse engineering" and is likely to become an important yardstick for distinguishing genuine reasoning capabilities in the next phase. Being able to write code doesn't mean being able to understand code, much less being able to reconstruct code under information-deficient conditions.

Targeting Real-World Software Engineering Scenarios

In actual development, when engineers face undocumented legacy systems, third-party closed-source components, or binary files requiring security audits, this "reverse-engineering logic from artifacts" capability is most valuable. It's estimated that over 70% of critical business systems in enterprises worldwide contain difficult-to-maintain legacy code, with a large portion of components having lost their original developers and design documentation. If AI can reach a usable level in this dimension, it will dramatically transform how software maintenance, security analysis, and technical debt management are conducted.

A Litmus Test for AI Agent Autonomous Exploration

Interestingly, reverse-engineering logic from runnable binaries is inherently a multi-step interactive task—the model needs to actively run the program, construct test inputs, observe feedback, and revise hypotheses. This precisely aligns with the core capabilities in current AI Agent research. In the programming domain, representative Agent frameworks include Devin (Cognition Lab), OpenHands (formerly OpenDevin), and others that can autonomously operate terminals, edit files, and run tests.

The process of reverse-engineering logic from runnable binaries naturally fits the Agent paradigm: it requires formulating probing strategies, executing tests, analyzing results, forming hypotheses, and verifying hypotheses—forming a complete scientific inquiry cycle (hypothesize-test-refine loop). This capability is also closely related to theoretical frameworks of Active Learning and Exploratory Testing. ProgramBench can therefore also be viewed as a litmus test for measuring the autonomous exploration capabilities of programming Agents.

Conclusion

Although community discussion around ProgramBench is still gaining momentum, the evaluation philosophy it represents deserves close attention. While the industry is broadly obsessed with "getting AI to write more code," the reverse challenge of "getting AI to understand existing code" is equally important—and perhaps closer to the essential difficulties of software engineering. Reverse-engineering program logic from runnable binaries serves both as a rigorous test of a model's deep reasoning capabilities and as a fresh perspective for re-examining whether "AI truly understands code."

Key Takeaways