Skeleton Coding: Taming AI Coding Agents with Program Structure

Providing AI coding agents with test cases and input-output examples is the current mainstream interaction paradigm. But a fundamental question is often overlooked: how much specification do developers actually need to provide for the agent to produce a program that meets expectations? A recent study called "Skeleton Coding" offers a counterintuitive answer—it depends on the implementation structure, and structure is precisely what the agent decides on its own.

When Behavioral Specifications Meet Structural Uncertainty

Traditionally, we constrain agents through end-to-end behavioral specifications (tests, examples). But the paper points out that the agent first commits to a structure—decomposing the program into data types, modules, and functions—which effectively fixes a class of candidate programs. Behavioral specifications then select a specific implementation within that class. The problem is that tests sufficient for one structure may be far from sufficient for another, requiring different edge cases or even exponentially more examples.

"Because the structure is chosen by the agent, developers cannot ensure specification completeness unless they reverse-engineer the structure from thousands of lines of generated code."

This explains why sometimes even a large number of tests still fails to prevent the agent from deviating from intent: with the wrong structure, no amount of behavioral constraint can make up for it.

Behavioral specification refers to describing what a program "should do" through input-output pairs, test cases, or assertions, without specifying "how to implement it." In software engineering, it belongs to black-box specification: as long as the program produces correct outputs for all test inputs, it is considered to conform to the specification. However, the completeness of behavioral specifications depends on coverage of the input space, and for complex programs, exhaustively enumerating all boundary conditions is often infeasible. When the agent selects the program structure itself, the same behavioral specification can correspond to vastly different test sufficiency under different structures—this is precisely the root of the "structural uncertainty" problem discussed in the paper.

Program structure here refers to the way functionality is decomposed into data types, modules, functions, and their call relationships. Structure determines the space of candidate programs: for example, whether to use recursion or iteration, a state machine or a pipeline, a single function or multi-module collaboration, directly changes the set of boundary conditions that need to be verified. "Fixes a class of candidate programs" in the text means that the structural choice is equivalent to carving out a subset of the infinite program space, and behavioral specifications can only filter within that subset. Therefore, a structural error means that even a large number of tests cannot rule out implementations that do not meet the intent.

Skeleton Coding: Letting Developers and Agents Jointly Specify Structure

The core idea of Skeleton Coding is straightforward: developers, while providing behavioral specifications, also specify the program structure (i.e., the "skeleton") in a high-level, code-like format. In this way, structure and behavioral specifications work together to dramatically shrink the candidate program space.

Theoretically, the paper models AI-assisted programming as a control problem and proves that skeletons can exponentially reduce the number of tests developers need to provide, thereby guiding the agent toward acceptable programs.

Empirically, across four models and three benchmarks, the best-performing skeleton specification reduced average test error by 48.4% compared to the best end-to-end behavioral specification. Even when the agent could query the full reference implementation, adding skeletons still reduced average test error by up to 27.0% relatively.

Skeleton in a programming context usually refers to a code framework containing only type signatures, module declarations, function headers, and key data structures, without concrete implementation logic. It is analogous to the structural beams and columns in an architectural blueprint: it specifies "what components exist and how they connect," but does not decide "how to lay bricks inside each component." The skeleton in Skeleton Coding is presented in a "high-level, code-like format"—formal enough for the agent to parse, yet not over-constraining implementation details. After the developer provides the skeleton, the agent only needs to fill in function bodies, thereby narrowing the search space from "any program" to "programs conforming to the given structure."

Additional Benefits of Skeletons: Narrowing the Model Gap and Reducing Cost

Beyond reducing test error, skeletons bring two notable effects.

First, they significantly narrow the average gap between small and large models, by about 52.4%. This means that with structural priors, less capable models can also produce code closer in quality to that of large models.

Second, implementation cost is reduced by 15–26×. This may be because, once the structure is clear, the agent no longer needs to repeatedly probe among multiple possible structures, reducing generation and verification overhead. In addition, skeletons reduce code erosion by an average of 24.7%, indicating better long-term maintainability of the generated code.

Code erosion refers to the phenomenon in which, during iterative development or repeated agent generation and modification, code gradually deviates from the original design and maintainability declines, manifesting as increased duplicated logic, blurred module boundaries, inconsistent naming, and so on. As an explicit structural contract, the skeleton provides a stable reference frame for each subsequent generation, making it less likely for the agent to introduce structural degradation when filling in implementations. Therefore, skeletons not only improve the quality of a single generation but also slow the rate of codebase degradation in long-term collaboration.

Implications for Developers

This research sends a clear signal: rather than spending all your effort writing more detailed tests, it is better to first take the time to sketch out the desired program structure. Skeletons are not meant to replace tests but to complement them—structure determines the candidate space, and tests make selections within it.

For developers who use AI coding assistants daily, this implies an actionable practice: before letting the agent write code, first describe your desired architecture using pseudocode or module partitioning. This may become a new habit for collaborating with AI.

Limitations and Outlook

The paper's empirical coverage spans four models and three benchmarks, and its conclusions are informative, but writing skeletons itself requires cognitive investment from developers. How to design skeletons that are neither over-constraining nor insufficiently guiding, and the generality of skeletons across different programming paradigms, remain worthwhile directions for exploration.

One-line Summary

Skeleton Coding research shows that letting developers specify program skeletons rather than relying solely on tests can exponentially reduce the required tests and significantly improve AI programming quality.

Paragraph Summary

The article introduces a study called Skeleton Coding, pointing out that the output quality of AI coding agents depends not only on behavioral specifications (tests, examples) but also on the program structure the agent chooses on its own. Because structure determines the candidate program space, behavioral specifications alone cannot guarantee specification completeness. Skeleton Coding lets developers specify program skeletons in a code-like format, complementing behavioral specifications. Theoretically, skeletons can exponentially reduce the number of required tests; empirically, the best skeleton reduces average test error by 48.4% compared to the best end-to-end behavioral specification, narrows the gap between small and large models by about 52.4%, reduces implementation cost by 15–26×, and reduces code erosion by 24.7%. The article recommends that developers sketch out the structure before letting the agent code.