BunshinRL: Efficient Agentic Reinforcement Learning via Workspace Partitioning

BunshinRL uses shared memory, tool caching, and speculative execution to cut agentic RL rollout memory by 51% and latency by 23%.
BunshinRL is a system-level solution for the memory and latency bottlenecks of agentic RL rollouts. It uses copy-on-write for inter-environment memory sharing, avoids repeated execution via tool prefix caching, and hides call latency through speculative tool execution with rollback on failure. Its foundation is BunshinFork—a live workspace fork mechanism for containers that rebuilds the process tree, shares memory pages, and preserves the filesystem, communication channels, and GUI state. The implementation relies on Linux kernel modules and modifications to CRIU, achieving measured peak memory reductions of up to 43%, total memory consumption reductions of up to 51%, and end-to-end rollout latency reductions of 23%. This work shifts the scalability problem from stacking hardware to system optimization, but the kernel-level changes also bring a high deployment barrier.
Introduction: The Memory Pain of Agentic RL
Having large models alternate between reasoning and tool calls within persistent workspaces has become the mainstream paradigm for training agents. But rarely discussed is the heavy memory overhead that scaling such agentic RL rollouts entails: each concurrent environment must maintain independent workspace state, and slow tool calls drag down the entire batch. BunshinRL is a system-level solution targeting these two bottlenecks—by sharing environment state and caching repeated tool calls, it reduces peak memory by up to 43%, total memory consumption by up to 51%, and end-to-end rollout latency by 23%.

Agentic RL rollout refers to the process in reinforcement learning training where a language model agent performs multi-step interactions (reasoning, tool calls, observing results) in an environment to generate trajectory data. Unlike single-turn generation, each concurrent environment must maintain an independent persistent workspace—including the filesystem, process state, API sessions, etc.—causing memory to grow linearly with concurrency. Meanwhile, tool call latency (such as network requests, file reads/writes) is typically far higher than model inference, making the batch completion time of a rollout dragged out by the slowest tool call. BunshinRL targets precisely this structural bottleneck of "many states, slow calls."
Three Key Techniques: Sharing, Caching, and Speculation
The core idea of BunshinRL is not complicated: since many environments are highly similar in their initial state and tool calls, there is no need to repeatedly pay memory and time costs for each environment. It approaches this from three levels:
- Inter-environment memory sharing: Using copy-on-write mechanisms, multiple environments share the same memory pages, and new memory is only actually allocated when a write occurs.
- Tool prefix caching: Tool calls often share identical prefixes (such as the same API call or file read), and caching the results of these prefixes avoids repeated execution.
- Speculative tool execution: Before the model explicitly requests it, tools that are likely to be called are executed in advance, thereby hiding tool call latency; if the guess is wrong, the state is restored via rollback.
These techniques are feasible thanks to a new underlying mechanism: BunshinFork.
Copy-on-Write (COW) is a memory management optimization strategy: multiple processes or execution units initially share the same physical memory pages, and only when one party attempts to write does the kernel copy a private copy for that writer. In the BunshinRL scenario, multiple environments share the initial workspace state, and the vast majority of pages are read-only early in the rollout, so physical memory usage is far lower than allocating independently for each environment. The cost of COW lies in page faults and copy overhead on writes, but for agentic workloads dominated by reads and reasoning with relatively sparse writes, this trade-off is very worthwhile.
Tool prefix caching draws on the idea of prefix caching in inference systems: when the request prefixes of multiple tool calls highly overlap, the execution results or intermediate states corresponding to the prefix are cached, and subsequent calls can hit directly without re-execution. This requires the system to recognize prefix equivalence and ensure that cached results remain valid when the environment state has not undergone conflicting changes.
BunshinFork: Live Workspace Fork for Containers
BunshinFork is the foundation of BunshinRL—a "live full-workspace fork" mechanism for containers. Traditional fork typically only copies processes, whereas BunshinFork can rebuild the process tree, share memory pages (copy-on-write), and fully preserve the filesystem, communication channels, and GUI state. This means a running workspace can be instantly "split" into multiple copies, with the copies sharing unmodified memory until one party writes.
It is precisely this capability that makes cache state reuse and rollback after failed speculation possible. When speculative execution guesses wrong, the system can quickly roll back to the fork point without rebuilding the environment from scratch.
CRIU (Checkpoint/Restore In Userspace) is an open-source tool on Linux for checkpointing and restoring running processes, capable of freezing the process tree, saving its memory, file descriptors, signal state, etc., and restoring them afterward. BunshinRL's modification to CRIU lies in extending it from a "cold checkpoint" to an efficient fork supporting live workspaces: it not only saves process state but also handles COW mapping of shared memory pages, filesystem views, and container-level state such as communication channels. Traditional CRIU checkpoints are often time-consuming and require pausing the process, whereas BunshinFork, with the cooperation of kernel modules, compresses the overhead of creating a fork point to a level usable for high-frequency speculative rollback.
Implementation and Performance: Kernel Modules and CRIU Modifications
BunshinRL and BunshinFork are not purely user-space solutions; they are implemented through Linux kernel modules and modifications to CRIU (Checkpoint/Restore In Userspace). This deep dive into the system's lower layers yields significant performance gains:
- Peak memory reduced by up to 43%
- Total memory consumption reduced by up to 51%
- End-to-end rollout latency reduced by 23%
These figures come from the paper abstract and reflect the direct impact of system-level optimization on resource efficiency in agentic RL scenarios.
Significance and Outlook
The value of BunshinRL lies in pulling the scalability problem of agentic RL back from "stacking hardware" to "changing the system." By sharing workspace state and caching tool calls, it allows the same hardware to support more concurrent environments while shortening the training iteration cycle. For agent training that requires large-scale parallel rollouts, this kind of system-level innovation may have more leverage than model-level fine-tuning. Of course, the kernel module and CRIU modifications also mean a non-trivial deployment barrier, and whether it can be integrated into mainstream RL frameworks in the future remains to be seen.
One-line Summary
BunshinRL uses shared memory, tool caching, and speculative execution to cut agentic RL rollout memory by 51% and latency by 23%.
Paragraph Summary
BunshinRL is a system-level solution targeting the memory and latency bottlenecks of agentic RL rollouts. It uses copy-on-write for inter-environment memory sharing, avoids repeated execution via tool prefix caching, and hides call latency through speculative tool execution with rollback on failure. Its foundation is BunshinFork—a live workspace fork mechanism for containers that can rebuild the process tree, share memory pages, and preserve the filesystem, communication channels, and GUI state. The implementation relies on Linux kernel modules and modifications to CRIU, with measured peak memory reductions of up to 43%, total memory consumption reductions of up to 51%, and end-to-end rollout latency reductions of 23%. This work shifts the scalability problem from stacking hardware to system optimization, but the kernel-level changes also bring a high deployment barrier.
Related articles

Abnormal Error Rate Spike in Claude Mythos 5.2 and Fable 5.2
Claude Mythos 5.2 and Claude Fable 5.2 show an abnormal spike in request error rates, marked as investigating on the official status page. This article covers the incident overview, practical impact on developers, and recommended responses.

ReAct Runtime Context Trimming: Why Claude Code Acts at 70%
A deep dive into ReAct runtime context trimming: why Claude Code compresses at ~70% usage, why tool calls dominate, why messages must be dropped in pairs, and why codebases don't need RAG.

AI Agent vs LLM: How Cursor and Claude Code Actually Work
An in-depth analysis of the essential difference between AI Agents and LLMs: LLMs generate code suggestions, while Agents modify files and run tests via tool calls. Using a Spring Boot bug fix as an example, it breaks down the workflows, permission control, and verification mechanisms of Cursor and Claude Code.