KITE Framework: Efficient Multi-Agent Collaboration Through Key-Layer KV Communication

The Communication Bottleneck in Multi-Agent Collaboration

Multi-agent systems built on large language models — where multiple AI models divide tasks and collaborate to tackle complex problems — have become an important path toward improving reasoning capabilities. But collaboration requires effective information exchange between agents, and the traditional approach relies on natural language: one agent writes its reasoning as text, and another reads it in. This is intuitive, yet it introduces substantial inference costs: every round of communication requires encoding and decoding large numbers of tokens from scratch.

To sidestep the overhead of natural language, researchers have proposed latent communication: transmitting a model's internal hidden states directly, so agents communicate in the model's "native language." One mainstream approach is KV cache-based communication — passing the key-value states from the sender's attention mechanism directly to the receiver.

The problem is that most existing KV latent communication methods optimize for sender-side state fidelity — the goal being to faithfully reconstruct the sender's internal state. This sounds reasonable but carries a hidden cost: transmitting KV states from all layers incurs enormous communication and compute overhead, and much of that information is redundant noise that doesn't actually help the receiver complete its task.

The KV cache is a core data structure in the attention mechanism of the Transformer architecture. In self-attention, each layer generates Key and Value matrices for the input sequence, used to compute attention weights between tokens. During inference, the model caches the K/V states of already-generated tokens to avoid redundant computation — this is the KV cache. In multi-agent communication, passing the sender's KV states to the receiver is essentially letting the receiver "remember" all the context the sender has processed, without needing to re-input and re-process the original text. This approach bypasses the token encoding/decoding overhead of natural language, but the trade-off is that KV cache size scales with the number of model layers, attention heads, and sequence length — for a model with dozens of layers and tens of billions of parameters, transmitting the full KV cache is considerable.

Shifting from "Faithfulness" to "Good Enough"

The core contribution of this arXiv paper is a redefinition of the goal of latent communication. The authors argue that the right metric isn't "how completely the sender's state is reconstructed," but rather "whether the receiver has enough to complete the task" — shifting from sender-side state fidelity to receiver-side task sufficiency.

This shift in perspective is significant. It transforms what was an engineering problem of maximizing information completeness into a task-oriented information selection problem. Since all the receiver truly needs is the information critical to completing its task, there's no need to transmit KV states from every layer — just identify and send the parts that matter most.

Building on this idea, the authors propose the KITE framework (task-oriented key-layer KV communication). Its defining feature is that it is training-free — no additional training is required, and it can be applied directly to existing models, dramatically lowering the barrier to deployment.

How KITE Works

KITE's mechanism can be broken down into three key steps.

Identifying the Key Layer

A large language model is composed of many stacked Transformer layers, each with its own KV states. Rather than treating all layers equally, KITE uses a receiver trajectory distortion criterion to identify the single most task-critical layer — the "key layer." This criterion measures how much the receiver's reasoning trajectory would deviate if a given layer's information were absent. The greater the deviation, the more critical the layer.

Different layers in a Transformer model serve distinct functions. Research has shown that shallower layers (closer to the input) tend to handle low-level lexical and syntactic features, middle layers handle semantic composition and relational reasoning, and deeper layers (closer to the output) are more associated with task decision-making and output generation. This functional specialization across layers means that for any given downstream task, not every layer's KV state carries equal importance. The receiver trajectory distortion criterion is designed with this in mind: it uses a data-driven approach to locate the highest-value layer by measuring each layer's impact on the receiver's reasoning path, rather than relying on hand-crafted priors. This also explains why the identified key layer may differ across tasks and models.

Transmitting Only the Key Layer's Working Memory

Once the key layer is identified, KITE transmits only the latent working memory associated with that layer — not the KV states from all layers. This is where the dramatic reduction in communication volume comes from: only the genuinely useful portion is transferred.

Using the Key Layer as the Entry Point for Latent Reasoning

More elegantly, KITE uses the same key layer as the entry point for autoregressive latent reasoning. The receiver begins its subsequent reasoning process from this layer, leveraging the transmitted information while maintaining coherent, uninterrupted inference.

Experimental Results: Faster and More Accurate

KITE's effectiveness was validated across seven benchmarks spanning two model families and three model scales. Compared to transmitting all-layer KV states, the results are striking:

• 28–36× reduction in communication volume: The amount of data exchanged between agents is compressed to a small fraction of the original;
• Up to 3× improvement in end-to-end inference speed: Less data transfer and computation directly translates to faster responses;
• Up to 23.3 percentage point improvement in accuracy: This last result is especially noteworthy — compressing the information not only avoided harming performance, it actually improved it.

That final result validates the paper's central insight: the redundant information in full-layer KV transmission doesn't just waste resources — it actively interferes with the receiver's reasoning. Filtering out the noise and retaining only task-relevant information leads to more precise collaboration. "Less is more" finds empirical support here.

Why This Work Deserves Attention

As multi-agent systems see wider deployment on complex tasks, the efficiency of inter-agent communication is becoming a key constraint on system scalability. KITE offers an important reframing: rather than striving for complete information transfer, design communication around what the receiver actually needs for the task at hand.

The training-free nature of the framework amplifies its practical value — developers can integrate this approach into existing systems without retraining any models, and gain simultaneous benefits in communication efficiency, speed, and accuracy. For teams looking to deploy multi-agent collaboration but constrained by inference costs, this is a method worth watching closely.

Of course, as a newly published paper, KITE's real-world generalization ability and performance across larger scales and more diverse tasks still await broader validation. But the task-oriented latent communication paradigm it represents offers a clear and compelling direction for designing efficient multi-agent systems.