AutoFyn Framework: Expert Iteration Optimization for Frozen Models

A recent study from arXiv introduces the AutoFyn framework, an agent architecture inspired by expert iteration algorithms. Unlike traditional approaches, AutoFyn doesn't update model weights. Instead, it iteratively improves the performance of frozen models through persistent state and verification reward signals.

Expert Iteration (ExIt) is a reinforcement learning paradigm originally inspired by the AlphaGo series of work. Its core idea involves alternating between two steps: first, generating "expert-level" action trajectories that outperform the current policy through search or planning, and then using these high-quality trajectories to improve the policy itself. In traditional implementations, policy improvement typically means updating the neural network's parameter weights—for example, through supervised learning to imitate expert trajectories or gradient updates via reinforcement learning signals. This paradigm has achieved remarkable success in domains such as Go, theorem proving, and code generation. AutoFyn's innovation lies in preserving the "explore-evaluate-improve" loop structure of expert iteration while shifting the vehicle of improvement from model parameters to external persistent state.

Core Mechanism: Non-Parametric Expert Iteration

AutoFyn's innovation lies in its non-parametric design. Each iteration begins with a fresh model session, and persistent information is reintroduced only through explicit interfaces, including persistent memory files, reports, and code repository state. This design keeps the model itself frozen while policy improvement relies entirely on updates to external state.

The term "frozen model" refers to keeping all weight parameters unchanged during inference, with no form of fine-tuning, adapter training (such as LoRA), or online learning. This stands in stark contrast to the current mainstream model improvement approaches—whether RLHF (Reinforcement Learning from Human Feedback), DPO (Direct Preference Optimization), or various post-training techniques, all of which require modifying model weights. The non-parametric design means the system's "knowledge accumulation" occurs entirely outside the model, achieved through structured files, databases, or context engineering. The advantages of this approach include eliminating the need for GPU clusters for training, enabling any API-accessible model to be used immediately, and avoiding side effects of parameter updates such as catastrophic forgetting.

Within a single iteration, an orchestrator collaborates with specialized agents to explore, plan, and construct multiple candidate solutions. The orchestrator plays the role of a central coordinator—a classic design pattern in Multi-Agent Systems. In this architecture, the orchestrator handles task decomposition, agent scheduling, and result integration, while specialized agents each take on specific functions—such as code writing, solution evaluation, and information retrieval. This division of labor resembles the microservices architecture in software engineering, where each agent focuses on the subtask it excels at. When the orchestrator collaborates with multiple agents to explore "multiple candidate solutions," it is effectively performing a form of tree search or beam search, increasing the probability of finding an optimal solution by exploring multiple paths in parallel.

A task-oriented verifier is responsible for examining work outputs and providing objective reward signals to measure progress. These rewards are then distilled back into the persistent state, updating the effective policy for the next iteration.

The persistent state mechanism in AutoFyn can be understood as a carefully designed external memory system. In cognitive science and AI research, such mechanisms are referred to as "externalized cognition" or "scaffolded memory." Specifically, persistent memory files record lessons learned, failed paths, and successful strategies accumulated from prior iterations; reports provide structured summaries of task progress; and code repository state preserves actual work outputs. When each new iteration begins, this external state is reinjected into the model through context injection, essentially rebuilding working memory for an "amnesic" model. This design shares similarities with RAG (Retrieval-Augmented Generation) but places greater emphasis on iterative state evolution rather than static knowledge retrieval.

This architecture cleverly shifts the learning process from model parameters to external state management, enabling the same frozen model to achieve continuous improvement through iteration.

Real-World Validation Across Three Domains

The research team validated AutoFyn's effectiveness across three challenging domains: Olympic mathematics competitions, data science, and cybersecurity.

Mathematics Competition Breakthroughs

On six new International Mathematical Olympiad problems, all models with room for improvement scored higher under the AutoFyn framework than their providers' own coding agents. This result demonstrates that even without changing the model itself, effective state management and verification loops can significantly enhance problem-solving capabilities.

Notably, International Mathematical Olympiad (IMO) problems typically involve deep reasoning in number theory, combinatorics, geometry, and algebra. Each problem is worth a maximum of 7 points and requires rigorous formal proofs. The challenge these problems pose to AI systems lies in their demand for multi-step reasoning, creative construction, and airtight logical chains—precisely the scenarios where AutoFyn's iterative verification mechanism can shine, as each iteration can correct and deepen the reasoning from the previous round.

Leading in Data Science

Agents built with AutoFyn achieved the #1 ranking on the Spider 2.0 dbt benchmark. Spider 2.0 is an upgraded version of the data science benchmark developed by Yale University and other institutions. The original Spider benchmark focused on text-to-SQL conversion tasks, while Spider 2.0 significantly expanded the evaluation scope to cover more complex data engineering scenarios. The dbt (data build tool) subset is particularly challenging—dbt is a widely used data transformation tool in the modern data stack, requiring models to understand data modeling concepts, SQL dialect differences, dependency management, and incremental computation logic. Achieving first place on this benchmark means AutoFyn can handle real-world multi-step data pipeline construction tasks, not just generate simple SQL queries. This achievement demonstrates the framework's advantages in handling complex data engineering tasks.

Cybersecurity Contributions

In the cybersecurity domain, AutoFyn demonstrated practical real-world value. It discovered and reported 16 maintainer-confirmed vulnerabilities across several well-known open-source projects, including Next.js, MetaMask, pnpm, Warp, LiteLLM, Langflow, and Open WebUI.

This list of projects carries significant weight. Next.js is one of the world's most popular React full-stack frameworks, widely used by companies like Vercel and Netflix; MetaMask is the most widely used crypto wallet in the Ethereum ecosystem, custodying billions of dollars in digital assets; pnpm is the fastest-growing JavaScript package manager; LiteLLM is a proxy tool for unified access to various LLM APIs; and Langflow and Open WebUI are popular open-source projects for LLM application development and interaction. Discovering maintainer-confirmed security vulnerabilities in these high-value targets not only proves AutoFyn's technical capability but also indicates that AI-assisted security auditing is becoming a viable practice. Traditional security auditing relies heavily on experienced human experts, is expensive, and is difficult to scale—AutoFyn demonstrates the enormous potential of automated security scanning.

All of these vulnerability advisories were formally confirmed by project maintainers, reflecting the framework's practical utility in real-world security scenarios.

Technical Significance and Implications

AutoFyn represents an important directional shift in agent design. Traditional model improvement relies on parameter fine-tuning or training from scratch, which requires substantial computational resources and data. AutoFyn, through persistent state and verification loops, achieves a more lightweight and flexible improvement pathway.

This non-parametric approach is particularly well-suited for long-horizon tasks, as it allows agents to accumulate experience across multiple interaction rounds without needing to retrain the model each time. Persistent state serves as a form of external memory, enabling frozen models to operate as if they were systems with learning capabilities. From a broader perspective, this approach mirrors how humans use notebooks, documents, and knowledge bases to extend their cognitive abilities—the brain's biological neural network doesn't undergo structural changes in the short term, but through external tools and records, humans can handle complex tasks far exceeding their working memory capacity.

This research offers a fresh perspective for AI agent development: rather than constantly pursuing larger models, focus on how to more effectively organize and leverage the capabilities of existing models. Through carefully designed state management and verification mechanisms, even frozen models can achieve continuous progress on specific tasks. This idea also echoes the recent rise of the "inference-time compute" research direction—improving AI system performance by investing more computational resources during inference (such as more search steps, longer chains of thought, and more iteration rounds) rather than scaling up model size during training. AutoFyn can be seen as a concrete implementation of this trend at the agent architecture level.

Key Takeaways