Stanford Open-Sources HybridAgent: Letting AI Think in Vector Space, Saving 54% of Tokens

The "thinking process" of AI Agents has always been an open waste: it writes every step of intermediate reasoning as complete English sentences, spewing them out token by token, before finally reaching the step that actually matters—calling tools, executing code, giving answers. A paper and open-source project jointly released by Stanford, UIUC, and Together AI questions this: these "self-talk" monologues were never necessary.

The project is called HybridAgent, and its idea is so simple it's almost offensive: let the model think in raw vector space, and only switch to text when the external world needs to read it. Days after its announcement, it hit the front page of Hacker News, sparking widespread discussion.

Core Mechanism: One Generation, Two Modes

Most "efficient reasoning" research picks a side—either latent-space reasoning (fast, but opaque and bad at using tools) or text-space reasoning (slow, token-hungry, but readable). HybridAgent refuses to choose, switching back and forth within the same generation process.

The key is a small adapter called HybridLink, which maps the model's final-layer hidden state directly back into its own input embedding space. This bypasses the usual bottleneck—decoding tokens, re-embedding, feeding them back—allowing the model to "think" in pure vector form for dozens of steps before deciding whether to commit to a word.

Two special tokens, <latent> and </latent>, mark the "on-ramp" and "off-ramp" to and from latent space. Outside the tags, it's an ordinary language model, calling tools, reading search results, writing answers; inside the tags, it compresses the entire chain of thought into something closer to "holding your breath" than "speaking sentences."

What reassures skeptics is that this isn't a black box by default. An "interpretable latent mode" renders each latent step back into pseudo-text, letting you watch HybridAgent's internal monologue like an HTML report—fragmented and abbreviated, but readable. In a public trace, a 42-step latent thought was decoded into something like "the the speed of speed in air at 20° is about 343... should verify... let me search"—clearly the residue of real thinking, not noise.

Latent space refers to the high-dimensional vector space of the model's internal representation, where each token is encoded as a dense vector. Conventional language models must decode from this space to the vocabulary for every generated token, then re-embed back into vector space, creating a "decode-re-embed" round-trip overhead. HybridAgent's HybridLink adapter maps the model's final-layer hidden state directly back into the input embedding space, allowing continuous "thinking" at the vector level without going through the vocabulary. This is similar to how humans quickly organize thoughts in their heads without speaking each step aloud, significantly reducing the computational cost of token generation and context concatenation.

Training Approach: First Learn to Switch, Then Learn When to Switch

Training is two-stage. HybridSFT first teaches the base model (based on Qwen3, at 8B and 14B scales) to collaborate with HybridLink, mastering the basics of mode switching. Then HybridRL takes over, learning "when" using latent mode is worthwhile, with reward signals including answer correctness, output format, and tool call success. The model was not explicitly told to minimize tokens; it discovered on its own that thinking quietly is often cheaper and equally accurate.

Reinforcement learning (RL) is used here to optimize the policy: the model adjusts its behavior based on reward signals. In the HybridRL stage, rewards include answer correctness, output format compliance, and tool call success rate, and the model learns through trial and error when entering latent space is more worthwhile. Since latent-space thinking doesn't directly produce tokens, conventional token-level supervision signals are missing, making RL a natural choice—it encourages the model to use latent mode when it improves final results, rather than forcing token minimization. This design lets the model autonomously discover the cost-effectiveness of "quiet thinking" rather than relying on manually set token budgets.

Empirical Data: Fewer Tokens and More Accurate

Across 11 benchmarks covering math, science, medicine, search, and Agent tasks, across three model scales, HybridAgent averages 10.4% higher than frontier pure-text Agents and pure-latent Agents, while using 35.5%–54.7% fewer tokens, running 1.8–2.6× faster, and costing 2.7–3.7× less.

One ablation study is more noteworthy than these headline numbers: replacing the model's latent thinking segments with segments from unrelated questions drops accuracy by 45%–51%. This shows latent thinking isn't filler—it's doing real work. That said, tool call execution rate remains at a reliable 99.7%, so the efficiency gains don't come at the expense of interacting with the external world.

Ablation study is a method in machine learning for verifying the necessity of components: remove or replace a module and observe performance changes. Here, replacing latent thinking segments with latent segments from unrelated questions causes accuracy to plummet 45%–51%, directly proving that latent-space steps carry substantive reasoning rather than random noise. This controlled-variable design is more convincing than overall metrics because it isolates the contribution of latent thinking and rules out interference from other factors (such as model scale or training data).

Hands-On Experience and Niche

The code is open-sourced on GitHub (togethercomputer/HybridAgent, MIT license), providing runnable checkpoints, not just a paper:

git clone https://github.com/togethercomputer/HybridAgent.git && cd HybridAgent
pip install -r requirements.txt && pip install -e .
cp .env.example .env   # add TAVILY_API_KEY to use the search tool
python -m hybrid_inference.run_batch --model togethercomputer/HybridAgent-8B-Agentic

Running requires a CUDA GPU, and the Qwen3 tokenizer has anchor tokens built in.

HybridAgent is not alone. Another team's LatentMAS applies a similar idea to multi-agent systems, passing latent thoughts between agents rather than within a single agent. What makes HybridAgent unique is that it's single-agent, fully inspectable, and directly releases usable weights rather than just a method description. That's probably why it's getting attention on HN right now—you can git clone it and run it on a Tuesday night, instead of reproducing it from scratch from the methods section.

The real news isn't that Agents got smarter, but that they finally stopped assuming "smarter" necessarily means "more words." For anyone running Agent pipelines at real scale, this is exactly the metric that has been quietly burning money.

One-line Summary

HybridAgent lets AI think in latent space, saving tokens and improving accuracy, open-source and runnable.

Paragraph Summary

Stanford, UIUC, and Together AI jointly released HybridAgent, challenging the convention that AI Agents must perform intermediate reasoning in full text. Its core is the HybridLink adapter, which allows the model to switch between text and latent-space modes within the same generation: special tokens mark latent thinking intervals, the model reasons continuously in vector form, and outputs text only when external interaction is needed. Training has two stages: HybridSFT (learning to switch) and HybridRL (learning when to switch). Across 11 benchmarks, average accuracy improves 10.4%, token usage drops 35.5%–54.7%, speed increases 1.8–2.6×, and tool call reliability is 99.7%. Ablation experiments show latent thinking plays a substantive role. The project is MIT open-source and provides runnable 8B and 14B checkpoints based on Qwen3.