MLExpert: A 35B Agentic Model That Makes Machine Learning Engineering Autonomous

Large language models are moving from assisting scientific research and engineering to autonomously developing and optimizing AI systems. This trend is called AI for AI (AI4AI), and it is also a possible path toward recursive self-improvement (RSI). Workflow orchestration in machine learning engineering (MLE) is precisely a realistic entry point for validating this path: it requires managing long-horizon experiments and dynamically adjusting optimization strategies based on execution feedback.

Recently, a study titled "MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering" proposed a post-training method for autonomous long-horizon MLE optimization, and trained a 35B mixture-of-experts (MoE) agentic model called MLExpert. On MLE-Bench Lite, it raised the base model's average medal rate from 37.9% to 72.7%, approaching larger frontier models such as Kimi-K3 and GLM-5.3.

Long-Horizon MLE: Why It's Hard to Train

The core challenge of MLE systems lies in the delay and coupling of feedback. The result of each experiment is actually a cumulative reflection of a whole chain of prior decisions; and a complete MLE run often contains many such high-cost experiments.

This raises a problem: when training a model, you cannot simply treat the final result as the only reward signal. The choices in intermediate steps, tool calls, and code modifications may all affect final performance. Traditional methods either ignore intermediate signals or are slowed in convergence by sparse rewards.

MLExpert's entry point is this: break multi-turn interactions into learnable windows, rather than fixating only on the endpoint.

Sparse rewards and the credit assignment problem are key background for understanding the difficulties of long-horizon MLE training. In reinforcement learning, if a reward signal is only given at the end of a task, while there are many intermediate steps and the process is long, this forms a typical sparse reward scenario. At this point, it is very difficult for the model to determine which step in a sequence of decisions led to success or failure, which is the credit assignment problem. Traditional approaches include introducing intermediate rewards, reward shaping, or post-hoc relabeling (such as HER), but these methods either rely on manual design or struggle to adapt to highly heterogeneous scenarios in MLE where code execution feedback is complex. MLExpert's windowed design is essentially a compromise between sparse rewards and stepwise dense rewards: it segments trajectories at the natural stage-level nodes of experimental results, thereby alleviating credit assignment difficulties while preserving outcome orientation.

Kaggle-MLE Factory and Hierarchical SFT

The research team designed a system called Kaggle-MLE Factory to construct executable tasks and provide an interactive environment for model-led experimentation. In other words, it does not just give the model a pile of problems, but lets it actually run experiments, modify code, and inspect results.

In the first stage of training, they adopted Hierarchical Agentic SFT. The goal of this step is not to directly teach the model to do MLE, but first to transfer general tool-use and coding abilities. The supervision signal comes from outcome-oriented final outputs—which trajectories actually solved the problem and which did not.

This is equivalent to first letting the model learn "how to act," and then letting it learn "how to optimize."

WIRL: Treating Execution Windows as Reward Units

The real key innovation is Windowed Interaction RL (WIRL).

WIRL works by treating the multiple rounds of decisions between two experimental results as an "execution window," and then assigning this window a shared, outcome-based reward. In this way, even if the final result has not yet appeared, the model can still learn from intermediate outcomes.

At the same time, they also added asynchronous collection and policy correction mechanisms, allowing policy updates to occur before a trajectory ends. This is especially important for long-horizon tasks—if one must wait for an entire trajectory to finish before updating, training efficiency becomes unacceptably low.

According to the paper, it is precisely this windowed reward design that enables MLExpert to extract useful supervision signals from long interaction trajectories, rather than being flooded by a large number of irrelevant steps.

Asynchronous reinforcement learning and the policy lag problem are technical background that cannot be ignored in WIRL's design. In traditional on-policy RL, policy updates must wait for an entire trajectory to be sampled, which creates a serious throughput bottleneck under long-horizon tasks. Asynchronous collection mechanisms allow different trajectories to progress at different times, with sampling and training proceeding in parallel, but the cost is that the data used for updates may come from an old policy, leading to policy lag. Common mitigation methods include importance sampling correction, V-trace, and trust-region methods that limit update magnitude. The policy correction mechanism mentioned by MLExpert is precisely intended to strike a balance between the efficiency gains brought by asynchrony and the bias caused by stale policies. This combination is particularly important in MLE scenarios with long trajectories and expensive experiments, and it is also a key engineering support for its improved training efficiency.

Experimental Results: From 37.9% to 72.7%

On MLE-Bench Lite, MLExpert raised the base model's average medal rate from 37.9% to 72.7%. This number means that a 35B MoE model can already compete on specific MLE tasks with larger frontier models such as Kimi-K3 and GLM-5.3.

More noteworthy is generalization. The paper also reports improvements on NatureBench Lite, agent scaffold optimization, and algorithm optimization. This shows that what the model learned is not rote memorization of a particular task, but a transferable optimization capability.

If this capability can truly transfer across tasks, then its significance for AI4AI is not merely benchmark chasing, but giving the model the beginnings of self-improvement.

MLE-Bench and medal rate are commonly used benchmarks and metrics for evaluating autonomous machine learning engineering capability. MLE-Bench was proposed by OpenAI and is collected from real Kaggle competitions, requiring agents to complete the full pipeline of data loading, feature engineering, model training, and prediction submission within a limited time. The medal rate refers to the proportion of agent submissions that reach the bronze medal threshold or above for that competition, directly benchmarking relative performance against human competitors, and therefore reflecting end-to-end engineering capability better than simple accuracy or loss. MLE-Bench Lite is a streamlined subset used to reduce evaluation cost. It is critical to understand the relativity of this metric: 72.7% means reaching human medal level on about three-quarters of tasks, but it is not equivalent to achieving the same performance in all real ML projects, and its generalization boundary still needs to be tested by other benchmarks.

Implications for the AI4AI Path

The value of MLExpert lies in solving two problems at once: first, the selection of supervision signals in long-horizon MLE; second, training efficiency. The combination of WIRL and asynchronous policy correction provides a feasible approach for training agents in expensive experimental environments.

Of course, the current results are still concentrated on competition-level ML tasks. Real-world machine learning engineering often involves fuzzier goals, longer feedback cycles, and stronger domain knowledge. Whether MLExpert can maintain its advantage in those scenarios still needs further verification.

But at least it points in a direction: letting models learn in experiments how to optimize the experimentation process itself may be closer to the path of recursive self-improvement than simply scaling parameters.

One-line Summary

MLExpert uses windowed reinforcement learning to train a 35B MoE agent, raising long-horizon MLE medal rate from 37.9% to 72.7%.

Paragraph Summary

The article introduces a study on MLExpert, an autonomous machine learning engineering agent. Facing challenges in long-horizon MLE such as delayed feedback, difficult credit assignment, and sparse rewards slowing convergence, the research team built the Kaggle-MLE Factory interactive environment, first transferring general tool-use and coding abilities through hierarchical agentic SFT, then using Windowed Interaction RL (WIRL) to divide the multiple rounds of decisions between two experimental results into an execution window and assign a shared outcome reward, combined with asynchronous collection and policy correction to update the policy before a trajectory ends. The 35B MoE model trained with this method raised the base model's average medal rate on MLE-Bench Lite from 37.9% to 72.7%, approaching larger frontier models such as Kimi-K3 and GLM-5.3, and demonstrated transferable optimization capability on NatureBench Lite, agent scaffolds, and algorithm optimization tasks, providing a post-training approach for the AI4AI and recursive self-improvement path.