OpenAI Reveals Agent Sandbox Cloud: Design Philosophy from microVMs to Global Orchestration

Why Agents Need Sandboxes

The story begins with the evolution of model capabilities. Early pretrained large models could correctly answer "what is 3+3" but would often stumble on "how many R's are in 'strawberry'" — because the former has been written countless times on the internet, while the latter lacks sufficient training data. The real breakthrough came from giving models the ability to execute code and make tool calls. Once a model can write and run code, it can perform hill-climbing searches in domains with verifiable rewards — like math and programming — and its performance improves dramatically.

Underlying this breakthrough is the evolution from RLHF (Reinforcement Learning from Human Feedback) to RLEF (Reinforcement Learning from Execution Feedback). When a model can execute code and receive deterministic execution results as reward signals, the training process no longer depends on human annotation — whether the code runs successfully is itself a natural verifier. This paradigm gave rise to OpenAI's o-series reasoning models, and is also a core training mechanism behind models like DeepSeek-R1. During training, the training loop assigns tasks to the model, the model generates responses, the framework executes the code, a scorer judges correctness, and gradients propagate back to update weights. This closed loop teaches the model two things: when to call tools, and how to ensure the code it generates actually solves the problem. On the product side, whether it's locally running Codex or cloud-based CodexWeb and ChatGPT, a secure environment is needed to run model-generated code.

The core challenge is that model-generated code may contain malicious or unintentionally harmful operations. It might attempt to gain root access, exploit kernel vulnerabilities, or even steal other users' data. As the presenter colorfully put it, the model might be "overly enthusiastic about helping you" and try to escalate privileges. Sandboxes exist precisely to resolve this tension — they provide an environment where tool calls and code execution can run safely on behalf of the model, getting the job done without crossing boundaries.

Diverging Needs: Research vs. Product

The talk explicitly distinguished between sandbox requirements on the research side versus the product side. For research, the core metric is throughput: running massive numbers of training loops and Rollouts (individual task attempts) in parallel, where parallelism directly determines training efficiency. For products, latency is the lifeline — if the sandbox takes too long to start or execute code, users will abandon the experience.

Reliability and security are non-negotiable requirements on both sides. Frequent failures waste precious GPU compute on either end ("GPUs are as valuable as gold right now"); on the security front, a misaligned model gaining Rollout access on the research side could attack OpenAI's infrastructure or leak model weights, while on the product side it could expose user data. The entire architecture is built around three pillars: runtime security, persistence, and orchestration.

From Fork to microVMs: The Evolution of Security Isolation

This is the most technically dense section of the talk. The presenter traced a clear evolutionary path showing how the attack surface was progressively narrowed.

Fork and Exec: Best Performance, Highest Risk

The simplest approach is to stand up an API server and fork a process for each tool call. Its only advantage is native performance. But the drawbacks are fatal: forked processes can communicate directly with the kernel and attempt kernel exploits; and continuously forking processes in a loop creates a "noisy neighbor" problem that can degrade the entire node.

Containers: Resource Isolation, Shared Kernel

Containers rely on two core Linux mechanisms: Namespaces (resource isolation) and Cgroups (resource quota enforcement). Namespaces provide isolated views across seven resource types: PID (process IDs), Network (network stack), Mount (filesystem mount points), UTS (hostname), IPC (inter-process communication), User (user permission mapping), and Cgroup (control group views). Cgroups enforce hard limits on CPU time slices, memory caps, and disk I/O bandwidth. Combined with Seccomp to filter dangerous syscalls, containers offer substantial protection — Docker, Kubernetes, and the entire container ecosystem are built on these three mechanisms.

But the fundamental problem remains: processes inside a container are still native processes on the host, sharing the same host kernel. Breaking through the kernel boundary to gain root means being able to attack other sandboxes. Seccomp filters are difficult to configure with a complete syscall whitelist, and the debugging experience is quite painful.

gVisor: A User-Space Kernel

gVisor was open-sourced by Google in 2018. Its core idea is to move the kernel implementation from C in Ring0 privileged space into a user-space Go program — its Sentry component acts as an "application kernel," written in Go and implementing a large number of syscalls, with Gofer handling filesystem access. Attack code runs in Ring3 rather than Ring0, so vulnerabilities are confined to user space. gVisor compresses the attack surface from millions of lines of kernel code down to the relatively contained Sentry codebase. Services like Google Cloud Run position it as a middle ground between performance and security.

However, Sentry and Gofer themselves still run on top of the host kernel, so attackers can execute a "two-step chain" — compromise Sentry/Gofer first, then attack the kernel. The Go runtime itself also has some attack surface, and syscall dependencies on the host kernel cannot be fully eliminated. The bar for attack is higher, but the path still exists.

microVMs: Hardware-Level Security Isolation

The ultimate answer is virtualization. It provides abstraction at the CPU hardware level, leveraging VMX extensions introduced by Intel VT-x (and AMD's AMD-V/SVM on AMD): the guest kernel runs in Ring0 under the "VMX non-root" context, while the host kernel and Hypervisor run in Ring0 under "VMX root." There is a hardware-level isolation barrier between these two Ring0 environments — privileged instructions from the Guest OS trigger a VM Exit that traps into the Hypervisor for handling, rather than directly operating on physical hardware. This means even if you gain root inside the guest and execute Ring0 code, the host remains protected — attackers can do whatever they want inside the guest VM but cannot touch the host. The equivalent mechanism on ARM is EL2 (Exception Level 2) virtualization extensions.

A new generation of Rust-based VMMs has emerged: CrossVM (developed by the Google team for Chromebooks) was the pioneer, followed by AWS's Firecracker and the more general-purpose Cloud Hypervisor. Firecracker was originally built for the Lambda and Fargate serverless products, with the goal of launching secure microVMs in under 100ms and exposing only a minimal device model (virtio-net, virtio-block, etc.), compressing the TCB (Trusted Computing Base) to roughly 50,000 lines of code. Cloud Hypervisor was initiated by Intel and is now a Linux Foundation project, supporting a broader range of device types. Compared to QEMU — with its heavy legacy and nearly 2 million lines of code — these VMMs have smaller memory footprints and faster startup times, which is exactly where the name "microVM" comes from. Rust's memory safety properties are critically important here — memory safety vulnerabilities have historically accounted for a significant proportion of VMM CVEs, and Rust substantially reduces this risk at the language level.

The presenter summarized what he called the "seven stages of sandbox grief": nearly every startup eventually circles back to microVMs — after trying containers and gVisor, they realize what they actually needed was a complete, secure Linux machine. "Start with microVMs from day one" — engineering tricks can paper over performance issues, but they can't paper over security vulnerabilities, and once trust is lost it's very hard to recover.

Persistent Storage: The Key to Unlocking Long-Running Agents

If compute was the first breakthrough, persistent storage may be the next one. The presenter used an analogy: imagine a computer with no disk, where closing the lid makes all data disappear — that's exactly the situation cloud-based Agents face today. People are building presentations and creating entire GitHub repositories inside sandboxes; when a node goes down, vast amounts of GPU compute and user work are lost.

Persistence unlocks three major use cases:

• Reliability and scalability: create periodic checkpoints and recover precisely on another node after a failure
• Long-running tasks: Codex's Go mode can already run continuously for three days
• Exploratory search: snapshot sandbox state to enable Monte Carlo-style backtracking and branch exploration

This "Monte Carlo-style exploration" connects to an important research direction in AI reasoning: applying MCTS (Monte Carlo Tree Search) to language model inference — snapshotting sandbox state at key decision points, spawning multiple execution paths in parallel, scoring based on code execution results and selecting the best path, and rolling back to previous snapshots and retrying as needed. This paradigm directly converts storage capability into reasoning capability, and is an important infrastructure underpinning for "deep thinking" models like OpenAI's o-series and Google Gemini.

The presenter even expressed hope that if the infrastructure is robust enough for models to run continuously for days, it could help tackle diseases and accelerate drug discovery.

At the implementation level, the snapshot solution needs to satisfy a few properties: incremental snapshots (only packaging the diff between two snapshots, since storing several GB every time would be cost-prohibitive), and fast snapshot and restore APIs. The talk described a Copy-on-Write (COW)-based design: create a zero-copy writable layer on top of a base image, only copying a data block to the writable layer and recording the change when that block is modified; use fiemap (the Linux kernel FIEMAP ioctl interface, which allows userspace programs to precisely query a file's physical block mapping) to identify changed data blocks, then package, compress, and upload to the cloud. This is fundamentally the same as Docker's image layering mechanism and ZFS/Btrfs snapshot principles. There's a "well-intentioned lie" here — a snapshot can quickly return a confirmation while asynchronously uploading in the background, without waiting for completion; this optimistic acknowledgment strategy further hides network latency from the user experience.

For long-lived persistence, the architecture uses a layered block cache: the sandbox sees a block device (via NBD — a Linux kernel-supported protocol that maps a remote server's block device over the network as a local block device, completely transparent to the sandbox's operating system), backed by a cluster-local cache (local NVMe SSDs), then cluster-shared block storage, and finally object storage (like S3), forming a globally tiered structure. Compared to filesystem-level solutions like NFS, block-level caching bypasses the complexity of POSIX filesystem semantics, enables finer-grained cache control, and aligns naturally with the snapshot system's "dirty block tracking" mechanism. The presenter specifically noted that models perform well with POSIX-compatible, standard transactional behavior, making block-level caching (rather than NFS-style solutions) the more appropriate choice.

Global Orchestration: The Core Logic of Cross-Region Scheduling

A single node will eventually go down. To truly scale, you need to run across many nodes globally. The presenter sketched the architecture from first principles: nodes are grouped into clusters, clusters are distributed across regions; a top-level control plane selects clusters based on regional load, and intra-cluster schedulers select nodes based on load. "Proximity" is always the guiding principle — ideally scheduling sandboxes close to the ChatGPT cluster to enable low-latency access.

For low-latency creation, there are three approaches: pre-warming a pool of sandboxes, millisecond-level startup from memory snapshots of microVMs, and a hybrid of both (maintaining a warm pool and using memory snapshots for scale-out, at the cost of idle CPU and memory consumption). Even more interesting is the deep integration between snapshots and orchestration: because snapshots contain multi-layer lineage, the scheduler can intelligently route requests to the node that "needs to download the least data," enabling faster creation and more reliable orchestration.

Conclusion: Sandboxes Are the Underrated AI Infrastructure

The greatest value of this talk is bringing an often-overlooked "unsung hero" to the forefront. While everyone focuses on model capabilities, Abhishek reminds us: without a secure, reliable sandbox, a model's code execution capabilities are meaningless. The security evolution from Fork to microVMs, the engineering details spanning block-level snapshots to global orchestration — together these form the infrastructure foundation of the Agent era.

His parting thought for the audience is equally worth reflecting on: compute was the first breakthrough, storage may be the next. When building sandboxes, we should think about what we can quickly snapshot and restore — thereby offering a new paradigm for training frameworks, enabling Agents to recover from failures, explore branches, and perform Monte Carlo-style searches. This may well be the critical step toward long-running, persistent, autonomous agents.