PROOF-Gen: Repairing Failed Trajectories So Distillation Data Doesn't Go to Waste

The Hidden Cost of Tool-Calling Distillation

Distilling the tool-calling capabilities of frontier teacher models into deployable smaller models has become standard industry practice. Knowledge Distillation was originally proposed by Hinton et al. in 2015, with the core idea of having a small "student model" learn the output distribution from a large "teacher model." In the era of large language models, distillation has expanded from its original focus on logits matching to broader behavioral imitation — the student model performs supervised fine-tuning (SFT) directly on text sequences generated by the teacher model, approximating the teacher's behavioral distribution by minimizing cross-entropy loss. The first stage of this pipeline almost invariably follows this approach: SFT on teacher-generated trajectories. The process seems simple and straightforward, but once it enters a real production environment, hidden costs begin to surface.

A "trajectory" here refers to the complete interaction sequence of an agent accomplishing a task, including every reasoning output, tool call request, tool return result, and final task completion status. Tool Calling (or Function Calling) itself is the foundational building block for AI agents — agents need to perceive their environment across multiple interaction steps, invoke external APIs or databases, receive feedback, and adjust subsequent actions. A single trajectory may contain dozens of such interactions, and its quality directly determines the effectiveness of distillation training.

The post-training pipeline powering production tool-calling agents often needs to re-run the distillation stage on a daily or weekly basis. Each iteration means paying the steep cost of calling frontier teacher models all over again. More critically, the underlying mechanism remains a simple "generate-and-filter" approach: keep the trajectories where the teacher model succeeds, discard everything else. This generate-and-filter paradigm is the dominant industrial practice for LLM data synthesis, widely used in code generation, mathematical reasoning, and tool calling. Its advantages are simplicity and quality control, but the pass rate directly determines cost efficiency — when frontier model API pricing remains high, a low pass rate translates to enormous direct financial losses.

This approach creates a structural flaw — each cycle leaves the same difficult scenarios unconquered. Because failed trajectories are discarded outright, they provide no valuable learning signal to the model. In other words, on exactly the hard problems where the model most needs improvement, it receives zero training data.

Wasted Failure Data: 57% of Inference Costs Down the Drain

On τ2-bench (a tool-calling capability evaluation benchmark), 57% of teacher model trials are failures. τ2-bench is a recently proposed benchmark specifically designed to evaluate LLM tool-calling capabilities. It simulates the complex multi-step tool-calling scenarios that agents encounter in the real world — unlike simple single-turn function calling evaluations, τ2-bench requires models to perform a series of interrelated tool operations across multiple dialogue turns, such as querying orders, modifying information, and processing refunds in a customer service scenario. This evaluation design more closely mirrors the true complexity of production environments, and precisely because of this, even when single-step accuracy is high, the overall success rate after chaining multiple steps drops dramatically. A 57% failure rate means that more than half of the inference costs produce trajectories that, under the traditional generate-and-filter paradigm, are thrown straight into the trash.

What's even more noteworthy is the composition of these failures: roughly two-thirds qualify as "near-misses." A near-miss means the model got the vast majority of tool calls right but stumbled at one critical juncture — for example, a particular action failed to complete or was inadvertently reverted.

Why Near-Miss Data Is So Valuable

From a learning signal perspective, near-miss failure trajectories contain extremely high information density. Viewed through the lens of information theory, the value of near-miss samples aligns with the theoretical foundations of Curriculum Learning and Hard Example Mining. In machine learning, models gain the richest gradient information from samples that lie near their decision boundaries. A trajectory that is 90% correct has its error point precisely marking the model's current policy failure mode — this localization precision far exceeds that of completely random failures or fully successful trajectories. This is similar to the concept of Hindsight Experience Replay (HER) in reinforcement learning — even when the final goal isn't achieved, partial successes along the way can still be relabeled and reused.

A trajectory that's almost entirely correct, falling short by just one final step, clearly delineates where the model's capabilities end — it reveals at which step, and in what manner, the model's reasoning chain broke down.

By contrast, successful trajectories tend to cluster around "easy scenarios" the model has already mastered, offering lower marginal learning value. Yet traditional distillation pipelines make precisely the opposite choice: keeping easy successes while discarding difficult near-successes. This is the fundamental reason why each iteration cycle fails to conquer the same hard scenarios.

PROOF-Gen's Core Idea: Data Optimization First

PROOF-Gen's name itself states its thesis: From Optimized Data to Better Distillation. Its focus isn't on swapping in a stronger teacher model, nor on improving the fine-tuning algorithm itself, but on systematically optimizing how training data is generated and utilized.

The core insight is this: rather than repeatedly paying frontier teacher costs only to discard more than half the output each time, why not convert those failures — especially near-miss failures — into usable training signal?

From Discarding to Repairing: The Key Operation for Turning Waste into Value

For near-miss trajectories, the most natural approach is to "repair" rather than "discard." Trajectory repair is PROOF-Gen's core technical operation, and its essence is performing minimal edits on failed trajectories to convert them into successful ones. Specifically, the repair process needs to address three key sub-problems: first, fault localization — precisely identifying the critical step in a long sequence that caused the task to fail; second, correction generation — generating the correct alternative action for the erroneous step; and third, downstream consistency — ensuring the corrected step is logically consistent with subsequent trajectory steps, or regenerating all steps after the correction point. This is analogous to the "hotfix" concept in software engineering — instead of rewriting the entire program, you locate and fix the minimal error point.

Since most tool calls are already correct, you only need to locate and fix the critical erroneous step to transform a failed trajectory into a high-quality successful one. This means:

• Inference costs are no longer wasted: The output from expensive teacher model calls is fully utilized;
• Data covers difficult scenarios: Training data reaches boundary regions that the generate-and-filter paradigm could never cover;
• Models gain targeted improvement: Precise learning signals are acquired right at the model's capability boundaries.

This "data optimization first" paradigm essentially extracts higher-quality, more targeted distillation data at equal or even lower teacher calling costs.

Practical Implications for Industrial-Grade Agent Pipelines

For production systems that require frequent retraining, the approach PROOF-Gen represents carries direct economic value. When a pipeline needs to rerun the distillation stage daily or weekly, teacher model calling costs accumulate linearly over time. Converting 57% of failed trajectories from "pure sunk costs" into "effective training assets" means significantly improved data output efficiency per unit cost.

Post-training Engineering is becoming a technical direction in LLM development that rivals pre-training in importance. As pre-trained models become increasingly homogeneous, data quality and training strategy during the post-training phase become the key to competitive differentiation. The industry has already shifted from the early "data scale first" mindset to "data quality first" — for example, Meta's Llama 3 series extensively uses carefully curated synthetic data during post-training rather than simply amassing data volume. The data optimization approach PROOF-Gen represents pushes this trend further toward "data efficiency first" — maximizing the extraction efficiency of training signals under a fixed teacher calling budget. This carries significant engineering-economic implications at a time when inference costs account for a major share of operational expenses.

Looking further ahead, this also changes the dynamics of distillation iteration. In traditional pipelines, difficult scenarios act like a wall that remains insurmountable no matter how many iterations you run; under the data optimization paradigm, each iteration pushes the capability boundary outward a step further, enabling the model to gradually conquer hard scenarios it previously couldn't break through.

Redefining "High-Quality Data"

PROOF-Gen reveals a deeper issue: in LLM distillation and the broader field of data synthesis, our definition of "high-quality data" may be too narrow. For a long time, the industry has defaulted to the assumption that only "completely correct" samples have value, but samples that fail at the capability boundary are often the best fuel for driving model improvement. This cognitive shift echoes multiple research findings in machine learning: from the importance of "hard negatives" in contrastive learning, to preference pair construction strategies in RLHF, to the alignment quality effects of "chosen-rejected" sample pairs in DPO training — a growing body of evidence suggests that model capability improvements are often not driven by "correct answers" but by "fine-grained contrasts between correct and incorrect."

As inference costs increasingly become a core operational expense for AI systems, how to extract maximum value from every expensive teacher call will become a critical question in post-training engineering. PROOF-Gen offers a clear and pragmatic answer: Don't discard failures — repair them.

Note: This article is based on a single RSS source. For PROOF-Gen's specific technical implementation details and complete evaluation data, readers are encouraged to consult the original research materials for a more comprehensive understanding.