LEGO Framework: Expert GraphRAG and Expert Chain-of-Thought Tackle Legal Reasoning Together

Two Structural Barriers to LLMs in Legal Domains

Large language models are increasingly being deployed in high-stakes domains like law and medicine, yet complex legal reasoning remains constrained by two structural problems. First, existing RAG and GraphRAG methods rely too heavily on lexical or semantic similarity, overlooking the normative relations between legal provisions. Second, standard Chain-of-Thought (CoT) prompting can generate seemingly plausible reasoning but fails to enforce the normative structure that legal reasoning demands.

In other words, retrieved provisions may be "textually similar yet normatively irrelevant," while the reasoning process may be "logically fluent yet legally unfounded." These two problems compound each other, limiting LLM performance on complex legal questions that require multi-hop reasoning. The LEGO framework, recently published on arXiv, directly targets both bottlenecks.

RAG (Retrieval-Augmented Generation) is an architecture that combines external knowledge bases with language models: the system first retrieves relevant document passages for a query, then feeds them as context to the model for answer generation, compensating for limitations in the model's parametric knowledge. GraphRAG extends RAG with a graph structure, organizing knowledge as a graph (nodes represent entities or documents, edges represent relationships) and performing multi-hop traversal along graph connections during retrieval — theoretically capturing more complex associations. However, both approaches fundamentally rely on semantic similarity signals — the distance between query and candidate text in vector space — which works well in general QA settings but has a fundamental flaw in legal contexts: two provisions that are semantically dissimilar may be tightly coupled through normative relationships like "lex specialis derogat legi generali" or exception clauses, while two textually similar provisions may be normatively unrelated. This is the core contradiction LEGO sets out to resolve.

LEGO's Dual-Module Design

LEGO is a dual-module framework built on the idea of pairing "expert-style legal retrieval" with "expert-style legal reasoning," corresponding to two components: ExpertGraphRAG and ExpertCoT.

ExpertGraphRAG: Graph Retrieval Based on Normative Relations

The first module, ExpertGraphRAG, uses an expert-annotated civil code graph that explicitly encodes normative relations between provisions rather than relying on semantic similarity alone. During retrieval, it applies a greedy "normative-coverage" algorithm to dynamically extract a case-specific provision subgraph for each query.

The value of this approach lies in the fact that legal provisions are interconnected through citations, qualifications, exceptions, and applicability conditions. Only by incorporating these normative relationships into retrieval can we ensure that the retrieved provisions are legally complete and relevant — not merely textually similar.

ExpertCoT: Structured Provision-Fact-Conclusion Reasoning

The second module, ExpertCoT, handles reasoning. It organizes retrieved provisions and case facts into a "Provision-Fact-Conclusion" structured reasoning chain — essentially a simulation of the legal syllogism: legal provisions serve as the major premise, case facts as the minor premise, and the conclusion is derived accordingly.

Compared to letting the model freely generate reasoning steps, this structural constraint effectively suppresses outputs that seem plausible but violate the normative structure of legal reasoning, aligning the process more closely with how a professional legal practitioner thinks.

The legal syllogism is the foundational logical framework for applying law in civil law systems. Its structure is: major premise (legal norm: if condition C is met, legal effect E follows), minor premise (case facts: the present case satisfies condition C), conclusion (legal effect E applies to this case). This formal logical structure ensures the predictability and reviewability of judicial decisions and is a cornerstone of legal professional training. Standard CoT prompting lets models expand reasoning freely, often producing fluent text, but does not force the model to follow the sequence of "identify the norm → match the facts → draw the conclusion." This easily leads to replacing normative citations with common-sense inferences, or presupposing a conclusion and working backward to justify it. ExpertCoT hard-codes the syllogistic structure as a reasoning template, compelling the model to explicitly cite provision sources at every step — making the reasoning process formally verifiable and logically traceable.

Experimental Results: Small Model Reaches Large Model Performance

The research team used Qwen3-8B as the backbone model for evaluation. On the LawExamQA_Civil dataset, LEGO achieved 40.53% exact-match (EM) accuracy, surpassing all RAG and CoT baselines in the evaluation and performing comparably to larger models tested.

This is a noteworthy signal: by incorporating domain expert knowledge (a normative graph) and structured reasoning constraints, an 8B-scale model can match the complex legal reasoning performance of much larger models. This is particularly attractive for real-world deployments seeking a balance between cost and capability.

Furthermore, LEGO maintained robustness on multi-hop questions — the type that requires chaining multiple provisions to reach a conclusion, and precisely where standard RAG tends to fail. On open-ended benchmarks, LEGO also achieved the best results among the evaluated baselines.

Exact Match (EM) is a strict evaluation metric for QA tasks requiring the model's output to exactly match the reference answer (typically compared character-by-character after normalization), with no partial credit. Compared to F1 scores or human evaluation, EM is particularly suitable for legal QA because legal conclusions are typically deterministic — citing the wrong provision number or omitting a key qualifier counts as an error. A 40.53% EM accuracy may seem low at first glance, but must be understood in the context of the difficulty of multi-hop legal reasoning tasks: these questions require the model to chain multiple provisions, accurately identify applicability conditions, and eliminate exceptions — a level where general-purpose model baselines typically fall significantly short. Achieving performance comparable to larger models with an 8B parameter model demonstrates that introducing domain knowledge structures can partially substitute for the performance gains typically associated with scale.

Ablation Studies Confirm Complementary Value of Both Modules

Ablation studies confirmed the independent contribution of each module and the complementary nature of the two together. In other words, ExpertGraphRAG and ExpertCoT are not simply additive — they mutually reinforce each other. Accurate retrieval provides a solid provision foundation for structured reasoning, while structured reasoning in turn makes full use of the normative relationships uncovered during retrieval.

This result validates the design philosophy of the framework: the two bottlenecks in legal reasoning are fundamentally coupled. Only by simultaneously solving "finding the right provisions" and "applying the right logic" can the complex legal reasoning capabilities of LLMs be genuinely improved.

Significance and Outlook

The value of LEGO lies not only in setting a new benchmark score, but in providing a paradigm for injecting domain normative knowledge throughout the entire retrieval and reasoning pipeline. In heavily regulated, rule-based domains such as law, healthcare, and finance, general-purpose RAG based purely on semantic similarity often falls short. The "expert-annotated graph + structured reasoning" approach demonstrated by LEGO may represent a practical path toward enabling general-purpose large models to handle high-stakes professional tasks.

The research team has open-sourced the code and dataset (GitHub: BLK-WHT/LEGO) for developers and researchers who wish to reproduce or build upon this work.