Parental Inheritance: The Hidden Engine of LLM Optimizers

Iterative language model optimizers refine prompts and programs by evaluating candidates, showing failure samples to the proposing model, and retaining effective revisions. Recent research has attributed the limitations of this loop to the narrowness of single-round reflection, and has therefore introduced full error clustering, targeted mini-batches, or larger proposing models so that the optimizer sees more failure modes at each step. However, existing evaluations are conducted within searches that preserve lineage—each revision edits a surviving parent—which conflates the contribution of parental inheritance with the breadth of per-round feedback. A study covering nine prompt and program synthesis benchmarks and 1,750 pre-registered runs separates parental inheritance from feedback disclosure under matched evaluation budgets, yielding a counterintuitive conclusion.

Lineage as Memory: The Dominant Role of Greedy Inheritance

The study finds that greedy parental inheritance drives the vast majority of progress in the optimization process. Even when seed-reset resampling receives clustered summaries of all failures at every round, or uses a larger proposing model, it still underperforms greedy inheritance. This means that the optimizer's continuous editing of a surviving parent is itself a process of accumulating and exploiting memory—resolved error patterns are retained in the candidate and filtered out of subsequent failure lists. Lineage is not merely the trajectory of the search; it is an implicit memory mechanism.

Greedy parent inheritance refers to the optimizer always starting from the currently best-performing surviving candidate (the parent) when generating a new candidate at each round, rather than restarting from the seed or a random candidate. The opposite is seed-reset resampling: regenerating from the initial seed at every round without retaining revisions from the previous round. The key difference between the two is whether modifications accumulate across rounds. Greedy inheritance allows already-fixed error patterns to settle in the parent, thereby shortening subsequent failure lists; seed reset faces the complete original failure set at every round. Comparing the two under matched evaluation budgets in the study is precisely intended to disentangle the two confounders of "memory accumulation from inheritance" and "breadth of per-round feedback."

The Acceleration and Limits of Feedback Disclosure

Full error clustering and diverse failure selection do improve single-step rewrites starting from the seed and accelerate the early rounds of search. But the study shows that this advantage rapidly disappears as sequential parental inheritance progresses. By retaining resolved error patterns, parental inheritance naturally shrinks subsequent failure lists, thereby quickly closing the initial gap created by feedback breadth. The gains from feedback disclosure are local and temporary, whereas the gains from inheritance are global and cumulative.

Error clustering is the merging of failure samples produced by candidates during evaluation into several clusters by semantics or pattern, rather than listing them one by one. Full error clustering lets the proposing model see all failure modes within a single round, avoiding overfitting to individual samples; targeted mini-batching instead selects a small number of representative samples from the failure pool to display, in order to control context length and cost. Both are "feedback disclosure" strategies—that is, they determine how much and which failure information is shown to the proposing model each round. They improve the quality of single-step rewrites but do not change the structure by which candidates accumulate modifications across rounds, so their benefits are overtaken by the inheritance mechanism as parental inheritance progresses.

Reconstructing Computational Cost: Reflection Is Not the Bottleneck

The study also breaks down token expenditure in GEPA and finds that proposing-model reflection accounts for only a small fraction of total cost, while validation and mini-batch gating consume nearly the entire budget. This indicates that greedy parental inheritance based on raw failures can capture the benefits of complex reflection loops at extremely low computational cost. The optimizer's efficiency bottleneck lies not in the depth of reflection but in the breadth of evaluation and the continuity of inheritance.

GEPA (Genetic-Pareto) is the iterative prompt optimizer analyzed in this article, combining genetic-style candidate retention with Pareto-frontier selection and generating revisions through proposing-model reflection on failure samples. Its token expenditure can be broken down into three parts: the proposing model's reflection (reading failures and writing revisions), validation (running candidates on the evaluation set to score them), and mini-batch gating (using a small batch of samples to decide whether a candidate advances to the next round). The latter two require repeated calls to the model under test to execute tasks, with cost growing linearly with the number of candidates and evaluation samples; reflection calls the proposing model only once, so its share is small. This decomposition shows that the optimizer's compute is spent mostly on "judging" rather than "thinking."

Implications for Optimizer Design

These findings challenge the current trend of improving optimizers by enhancing single-round feedback. If parental inheritance is the primary driver, then design efforts should shift toward how to better maintain and exploit lineage memory, rather than simply expanding the breadth of per-round feedback. For resource-constrained settings, simple greedy inheritance combined with raw failure feedback may be more efficient than complex reflection loops. Future optimizers may need to explicitly model the inheritance process, finding a new balance between lineage retention and feedback diversity.

One-line Summary

Research reveals that greedy parental inheritance, not feedback breadth, is the primary driver of progress in LLM optimizers.

Paragraph Summary

A study covering nine benchmarks and 1,750 pre-registered runs separates parental inheritance from feedback disclosure under matched evaluation budgets, yielding a counterintuitive conclusion: greedy parental inheritance drives the vast majority of progress in LLM optimizers, and even seed-reset resampling with full error clustering or a larger proposing model still underperforms inheritance. Full error clustering and diverse failure selection only accelerate the early stages of search, with the advantage rapidly disappearing as sequential inheritance progresses. Token decomposition of GEPA shows that proposing-model reflection accounts for only a small fraction of cost, while validation and mini-batch gating consume nearly the entire budget. Lineage is therefore an implicit memory mechanism, and optimizer design should shift toward maintaining and exploiting lineage memory rather than simply expanding per-round feedback breadth.