The Communication Bottleneck in Language Models: A Round-Trip Study on Tree Expression Serialization

What Gets Lost When Reasoning Is Compressed Into Natural Language?

Whenever large language models engage in Chain-of-Thought (CoT) reasoning or exchange intermediate results as free text, they are fundamentally doing one thing: serializing structured information into natural language. This transformation may seem seamless, but it hides a critical question — when compositionally structured content is packed into a prose description, how much information can actually be preserved without loss?

This research zeroes in on a specific, well-defined scenario: tree-structured expressions. Arithmetic expressions naturally possess the hierarchical and compositional properties of trees. An expression like (3 + 5) × (7 - 2), with its operator precedence and nesting relationships, forms a parse tree. When a model "translates" such a tree into a word problem, and another model then tries to recover the original expression from that problem, does the natural language in between serve as a bridge for information — or a funnel that narrows it?

Chain-of-Thought (CoT), introduced by Wei et al. in 2022, is a prompting technique that guides models to output step-by-step reasoning before delivering a final answer. It significantly improves performance on math, logic, and commonsense reasoning tasks. Its core assumption is that making implicit reasoning explicit as natural language steps helps models better organize and leverage intermediate information. However, this assumption rests on an under-examined premise — can natural language steps carry structured information without loss? When reasoning involves nested logic, hierarchical dependencies, or compositional relationships, there is a fundamental structural gap between linearized text sequences and the original tree- or graph-shaped information. Quantifying this tension is precisely what this study sets out to do.

The Round-Trip Protocol: Using Symbolic Equivalence as a Precise Judge

The researchers propose a "round-trip" experimental protocol that quantifies this communication bottleneck in a reproducible way. The pipeline consists of three stages:

• Generator: Converts a programmatically generated arithmetic expression into a natural language word problem;
• Extractor: Using only the word problem, attempts to reconstruct the original arithmetic expression;
• Symbolic Equivalence: Acts as a precise oracle, determining whether the reconstructed expression is mathematically equivalent to the original.

The elegance of this design lies in how it transforms a vague question about "semantic fidelity" into a problem with a precise, verifiable answer. Whether the reconstructed expression is correct doesn't depend on human scoring or approximate similarity — it's determined by strict symbolic equivalence. Either they're equivalent, or they're not. No gray areas.

The arithmetic expressions are generated programmatically at random, which means the complexity, nesting depth, and structural form can be systematically controlled. This allows researchers to observe how information loss patterns change across different difficulty levels.

The role of symbolic equivalence here deserves further explanation. Two arithmetic expressions are symbolically equivalent if they produce identical results for all valid inputs and can be derived from each other through algebraic transformations — for example, (3+5)×2 and 3×2+5×2 are numerically equal but structurally different as trees. Depending on the evaluation objective, the study can apply either "numerical equivalence" or "structural equivalence" as the criterion, with differing levels of strictness and differing definitions of information loss. The key advantage of using symbolic equivalence over numerical computation is that it distinguishes between "accidentally matching answers" and "correctly reconstructed structure" — capturing the true cost of conveying compositional structure through natural language, rather than being masked by numerical coincidence.

Pairwise Evaluation Across Sixteen Models

The study goes beyond single-model self-evaluation and assesses all pairwise combinations of sixteen models. This means the Generator can be model A while the Extractor is model B, covering a wide range of cross-model "encode–decode" pairings.

This pairwise design reveals issues that single-model evaluation cannot expose:

• A model may excel at encoding expressions into clear word problems (strong encoder) while struggling to reconstruct structure from someone else's problem (weak decoder), or vice versa;
• When the Generator and Extractor are different models, their implicit "encoding conventions" for natural language may not align, potentially amplifying communication loss;
• By comparing within-model round trips against cross-model round trips, it becomes possible to disentangle two dimensions: language expression ability and structural comprehension ability.

For researchers and engineers working on multi-agent collaboration and natural language exchange of intermediate results between models, this cross-model pairing perspective is especially important — real-world Agent pipelines are often composed of different models chained together.

Why This Research Matters

As Chain-of-Thought reasoning, multi-agent systems, and inter-model text collaboration become mainstream paradigms, a long-neglected assumption is coming into focus: Is natural language truly a reliable medium for conveying structured information?

This work provides a quantifiable entry point into that question. It reminds us that when a model has "reasoned something through" but must express it in words — or must interpret text written by another model — the structural information of compositional content may quietly evaporate in that exchange. This has direct implications for:

• Chain-of-Thought reasoning: If structural information in intermediate steps is lost during naturalization, the reliability of final answers is compromised;
• Multi-agent collaboration: When Agents pass tasks and results as free text, the communication bottleneck may become an invisible performance ceiling for the overall system;
• Model interoperability: When mixing different models, it's worth evaluating how robust their shared "language" for structured information actually is.

Tree-structured expressions are just a controlled probe, but the communication bottleneck they reveal carries much broader significance. As models increasingly "talk to each other in language," understanding and quantifying this loss will be a necessary prerequisite for building reliable AI systems.

In Multi-Agent Systems, different AI Agents typically collaborate on complex tasks through text messages — for example, one Agent handles planning, another handles execution, and a third handles verification. In this architecture, what passes between Agents is not just final answers, but complex information including task state, intermediate reasoning, and structured instructions. When this information must be encoded in natural language, whether the receiving Agent can accurately parse the sending Agent's intent depends heavily on whether both parties share a compatible "understanding pattern" for language. The cross-model communication loss revealed in this study accumulates and amplifies as the number of nodes in a multi-Agent pipeline increases — each "encode→decode" cycle may shed another layer of structural information, ultimately making it difficult to trace task execution errors back to the specific point of information loss.