Cache-to-Cache: Letting LLMs Communicate Through KV Cache Instead of Text

When LLMs Stop Talking in Words

In multi-agent systems, collaboration between large language models has long relied on a straightforward approach: convert one model's output into natural language text, then feed it as a prompt to the next model. This pipeline is intuitive and human-readable, but it hides two long-overlooked costs — information loss and latency inflation.

Cache-to-Cache (C2C) is a research proposal with a genuinely disruptive premise: let models bypass text as an intermediary and communicate directly at the KV Cache (key-value cache) level. In other words, models no longer need to translate their internal "thoughts" into human language — they pass the underlying semantic representations directly to each other.

This work received 42 points and notable discussion on Hacker News, reflecting a growing community interest in the low-level communication mechanisms that underpin multi-model collaboration.

The Hidden Costs of Text-Based Communication

To appreciate what C2C offers, it helps to first understand what's broken about the current approach.

When model A generates text and passes it to model B, the rich, high-dimensional semantic representations inside model A get compressed into a discrete token sequence. This "compress-then-decompress" process is inherently lossy: fine-grained semantic information, uncertainty distributions, and contextual associations all get flattened in the conversion to text. Model B then has to re-encode everything from scratch, reconstructing its own internal representations from those words.

This creates two layers of waste. The first is semantic bandwidth waste — natural language's expressive capacity falls far short of the information density achievable in a model's continuous internal vector space. The second is computational waste — model B must redo the full prefill-stage forward pass over content that model A has already "understood", essentially duplicating the same comprehension work.

In agent collaboration scenarios that require frequent back-and-forth, these costs compound with each round, ultimately slowing response times and accumulating distortion throughout the information relay chain.

The prefill phase is the first step in LLM inference: the model performs a complete forward pass over all input tokens in parallel, generating KV Cache for every layer to prepare for subsequent token-by-token generation (the decode phase). Prefill computation scales roughly linearly to quadratically with input length, making it one of the primary sources of inference latency. In a multi-agent pipeline, if model A outputs a lengthy analysis, model B must re-execute a full prefill over that text — even though its semantic content has already been "digested" by model A. This redundant computation becomes a significant source of accumulated latency when context windows are long or agent call chains run deep, and it is precisely the bottleneck that C2C aims to address.

The Core Idea Behind Cache-to-Cache

C2C's key insight is this: KV Cache is itself the model's semantic "memory" of what it has already processed. If that's the case, why not pass that memory directly to another model?

In Transformer architectures, KV Cache stores the Key and Value vectors corresponding to each token at every attention layer — it is the intermediate state that crystallizes after the model processes its context. C2C's approach is to establish a mapping mechanism that transforms and aligns the source model's KV Cache into a form that the target model can use directly, allowing the target model to "pick up where the source left off" in terms of semantic understanding.

This yields two immediate benefits:

• Richer semantic fidelity: By bypassing the lossy text channel, transferring continuous representations can preserve far more information.
• Eliminated redundant computation: The target model no longer needs to re-run prefill over already-understood content, which can significantly reduce latency.

The hard part is that different models do not have naturally aligned representation spaces — they may differ in architecture, dimensionality, and training data. The core technical challenge C2C must solve is how to build an effective semantic bridge for KV Cache across heterogeneous models.

KV Cache is a critical inference acceleration technique in Transformer architectures. In the self-attention mechanism, each token at each layer produces a corresponding Key vector and Value vector. Without caching, generating each new token would require recomputing the K/V representations of all prior tokens, causing computation to grow quadratically with sequence length. KV Cache solves this by storing already-computed K/V vectors for reuse by subsequent tokens, reducing the generation phase from O(n²) to O(n). As a result, KV Cache is not a throwaway scratch buffer — it is the model's complete retained understanding of the input context. It inherently carries the full intermediate state from having "read" a piece of content, which is precisely why C2C treats it as a viable medium for cross-model semantic transfer.

What This Could Mean

If approaches like C2C can reach maturity, their implications for multi-agent systems could be structural.

Today's mainstream agent frameworks — whether for task decomposition, tool use, or multi-role debate — are almost universally built on the assumption that "text is the protocol." Text's advantages are real: it's universal, auditable, and easy to debug. But the cost is exactly the loss and latency described above. C2C offers an alternative path: in scenarios where performance and semantic fidelity are paramount, direct semantic communication can replace text-based round-trips.

Of course, this path comes with clear trade-offs. Cache-level communication sacrifices the readability and interpretability of text — humans cannot directly "read" what is being transmitted, making debugging and security auditing considerably harder. Whether cross-model representation alignment is robust, and whether it generalizes to highly heterogeneous model pairs, still requires substantial validation.

From a longer-term perspective, C2C represents a trend worth watching: as multi-model collaboration becomes the norm, the "communication protocol" between models is itself emerging as an independent research frontier. Text may be only the first-generation language for inter-model communication — not its final form.

Summary

Cache-to-Cache shifts the focus of LLM collaboration from "what to say" to "how to transmit." It challenges the default assumption that text is the only valid communication medium between models, and attempts to trade it for higher semantic fidelity and lower latency through direct KV Cache transfer. Whether this approach can scale to real-world deployment remains to be seen, but the question it raises — how should models communicate efficiently with each other — is important enough to stand on its own.