VSArena 1.0 Launches: An Open Browser Benchmark for Embodied AI

Language models have public leaderboards. Robotics and VLA (Vision-Language-Action) policies, however, are still primarily evaluated in private simulation environments — with results buried in PDF tables and no unified, reproducible public standard for comparison. VSArena 1.0, developed by independent lab ONISCOR, aims to change that by bringing embodied AI evaluation into the browser as an openly verifiable benchmark.

What Is VSArena

VSArena positions itself as the official open browser benchmark for embodied AI. Its core task is straightforward: have a policy model complete a block-stacking task, arranging blocks in the order cyan → orange → magenta.

Behind this seemingly simple task lies a complete evaluation system. Physics simulation runs directly in the browser, powered by the Rapier physics engine — meaning anyone can open a webpage and observe real physical interactions without setting up a complex local simulation environment. Policy models communicate with the hosted evaluation harness via WebSocket, sending action decisions back to the simulation for execution.

This "browser as testing ground" approach lowers the barrier to participation and verification. For a robotics evaluation field that has long relied on private simulators, this represents an attempt to make the process transparent.

Rapier is a high-performance physics engine written in Rust and compiled to WebAssembly, designed for real-time rigid body simulation in browsers and lightweight runtimes. Unlike mainstream robotics simulators such as MuJoCo or PyBullet, Rapier requires no local dependencies — simulation state runs entirely within the browser sandbox, ensuring cross-platform reproducibility. This allows VSArena to let any visitor verify policy behavior under identical physical conditions without result discrepancies caused by local simulator version differences — a core requirement for reproducible benchmarks.

VLA (Vision-Language-Action) models are the dominant paradigm in embodied AI today, unifying visual perception, language understanding, and action output within a single model framework. Representative works include Google's RT-2 and OpenVLA. These models take images and language instructions as input and directly output robot joint control signals or end-effector poses — essentially transferring large-scale pretrained language/vision models to robot manipulation tasks. VSArena evaluates exactly this class of policy models: they receive observation data from the simulation environment via WebSocket and send action commands back for execution.

ELO Leaderboard and Evaluation Credibility

VSArena borrows the open leaderboard concept from the language model world and introduces an ELO scoring system for VLA policies. Several key design choices are worth noting.

First, the public ELO leaderboard applies only to VLA policies, and all results are signature-verified using a digest combined with Ed25519 digital signatures. This ensures every entry on the leaderboard is traceable and tamper-proof. In an open system where anyone can submit, anti-cheating and result integrity are fundamental to the leaderboard's credibility.

Second, the project explicitly separates two paths: "demo" and "official submission." The Tab demo on the webpage (Studio mode) never writes to the official leaderboard. Users can play around and debug in Studio first — getting familiar with the observation contract and task mechanics — without worrying about polluting leaderboard data. When ready to compete officially, they follow the submission process, which the team says takes roughly 10 minutes.

The ELO rating system was originally designed by physicist Arpad Elo for chess. Its core idea is to dynamically update ratings based on match outcomes: beating a stronger opponent earns more points, while beating a weaker one earns fewer. In recent years it has been adopted for LLM evaluation (e.g., LMSYS Chatbot Arena) to handle relative rankings where direct quantitative comparison is difficult. In robotics policy evaluation, ELO faces the challenge of sparse task scenarios and limited submissions — early scores have high variance, and ranking stability only emerges with enough head-to-head data. This is a key reason the VSArena authors are actively soliciting community feedback.

Ed25519 is an elliptic curve digital signature algorithm known for high performance and resistance to side-channel attacks, widely adopted in OpenSSH, TLS 1.3, and more. In VSArena's context, each submission result includes an Ed25519 signature and content digest generated by the server's private key. Anyone can independently verify using the public key whether a result was genuinely produced by the official evaluation harness and whether it has been tampered with after the fact. This mechanism shifts "trust" from relying on platform reputation to cryptographic guarantees.

How to Participate

For developers who want to try it out, VSArena offers several clear entry points:

• No-submission demo: Jump into Studio mode to experience the simulation at vsarena.vercel.app/simulation
• Official submission: Takes roughly 10 minutes, available at vsarena.vercel.app/submit
• Evaluation integrity documentation: The project publicly documents the eval-integrity specs on GitHub, detailing the signing and anti-cheating mechanisms
• Community: An official Discord channel is available

The project is developed by independent lab ONISCOR. At launch, the author explicitly invited criticism of the "observation contract" and "scoring mechanism." This openness to peer scrutiny is both reasonable and necessary for a freshly released v1 benchmark — the value of a benchmark is built precisely on community consensus and ongoing skepticism.

Why This Matters

Embodied intelligence and VLA models have been gaining momentum, but fragmented evaluation standards have long been a pain point for the field. Different teams use different simulators, different task setups, and different metrics — making meaningful cross-team comparisons nearly impossible.

VSArena's significance lies not in its current single stacking task, but in its attempt to establish a publicly verifiable, reproducible, anti-cheating comparison framework for the field. Running physics simulation in the browser, using digital signatures to guarantee result integrity, and separating demo from official leaderboard — all of these design choices point toward the same goal: making embodied AI evaluation as transparent as language model leaderboards.

Of course, as a 1.0 release, much remains to be seen — task diversity, whether the observation contract is sufficiently general, and ELO stability under sparse task conditions all require more submissions and community feedback to validate. But as an open-source, open-participation effort, VSArena has made a promising start toward standardizing embodied AI evaluation — one worth following closely.