GLM-Legal Tops Harvey LAB: A New King of Legal AI Benchmarks

The legal AI race has an unexpected challenger. Z.ai's GLM-Legal model has surpassed Claude Fable 5 and Kimi K3 on Harvey's Legal Agent Benchmark (Harvey LAB) with a 19.2% all-pass rate, making it one of the best-performing legal-specific models on the leaderboard.

Why Harvey LAB Matters

Harvey is one of the most closely watched startups in legal AI, and its LAB (Legal Agent Benchmark) is not a simple multiple-choice test — it simulates real lawyer workflows through agent tasks. That means models need multi-step reasoning, document retrieval, clause comparison, and structured output capabilities, rather than merely "reciting legal statutes."

For the legal industry, this kind of benchmark matters far beyond academic scoring. When law firms and corporate legal departments choose AI tools, what they care about most is not whether a model can pass a bar exam, but whether it can reduce manual intervention in real scenarios like contract review, due diligence, and compliance checks. Harvey LAB's design is precisely aligned with that need.

The difference between agent benchmarks and static Q&A benchmarks: Traditional legal AI evaluations are mostly multiple-choice or short-answer formats, where a model only needs to produce an answer within a given context to score. Agent benchmarks, by contrast, require models to proactively call tools, retrieve documents, plan steps, and adjust strategies based on intermediate results — essentially placing the model in a simulated work environment to test end-to-end task completion. This means evaluation tests not only language understanding but also engineering capabilities like function calling, state management, and error recovery. Harvey LAB is considered more rigorous precisely because it doesn't score "whether a particular step was correct" but "whether the entire workflow ran through" — which aligns closely with what law firms actually care about when procuring AI tools.

What the 19.2% Figure Means

At first glance, a 19.2% all-pass rate doesn't seem high. But under Harvey LAB's evaluation standards, the weight of this score needs to be reinterpreted:

• This is an "all-pass" metric, meaning the model must be correct on every sub-step of the same task; any single error disqualifies it.
• Legal agent tasks are inherently difficult by design, involving cross-referencing across multiple documents and long-chain reasoning.
• GLM-Legal's result is nearly three times that of its base model GLM-5.3-Flash, showing that domain-specific fine-tuning for legal work delivers significant gains.

In other words, under a "strictly all-correct to pass" rule, any double-digit pass rate represents preliminary usability for the model in real legal tasks.

The rigor of the all-pass metric: In multi-step task evaluation, there are two common scoring methods — "partial credit," which scores sub-steps individually, and "all-pass," which requires every sub-step to be correct. The former produces relatively flattering numbers but easily masks problems where a model errs in the middle of a long reasoning chain; the latter judges any single misstep as overall failure. Legal work is characterized precisely by extremely low error tolerance — if a due diligence report misses one key clause, the usability of the entire report is greatly compromised. So while the all-pass metric may look low numerically, it is closer to the "deliverable = usable" standard in legal practice, which is why interpreting 19.2% cannot simply involve horizontal comparison with scores on other benchmarks.

GLM-Legal's Technical Foundation

GLM-Legal is not a new model trained from scratch, but a legal-specific version built on GLM-5.3-Flash. This "base + domain fine-tuning" approach has become the mainstream choice in vertical AI — it balances inheriting general capabilities with injecting domain knowledge.

A detail on cost data: each test case runs at approximately $0.15. For legal scenarios requiring repeated execution and batch processing, this price is far below the calling cost of most general flagship models. This also explains why Z.ai chose the Flash series as its base — while ensuring usable capabilities, it pushes inference costs down to a level where the legal industry can deploy at scale.

Currently, GLM-Legal is accessible by application only, rather than through an openly available API. This may be out of caution regarding compliance in legal scenarios, or it may be to collect real user feedback for model iteration.

The relationship between domain fine-tuning and base models: The so-called "base + domain fine-tuning" approach means first selecting a model already pretrained on large-scale general corpora as the foundation, then performing supervised fine-tuning or preference optimization with high-quality domain-specific data (such as legal contracts, case law, compliance documents, and corresponding task trajectories). The advantage is that the model doesn't need to learn language and reasoning from scratch — it only needs to "align" general capabilities to the expression conventions, terminology systems, and task flows of legal scenarios. Compared with training a specialized model from scratch, this route requires less data, iterates faster, and more easily inherits the base model's capabilities in long context, tool calling, and other areas. GLM-Legal's result being nearly three times that of its base model precisely demonstrates that domain knowledge injected through fine-tuning can yield returns on specific tasks several times greater than differences in parameter scale.

The Legal AI Competitive Landscape Is Shifting

GLM-Legal's result poses a direct challenge to the existing order. Claude Fable 5 and Kimi K3 were both previously standout models on legal agent tasks — the former backed by Anthropic's accumulated expertise in long text and safety alignment, the latter a major domestic player in legal vertical applications of large models.

GLM-Legal's lead points to two trends:

The cost-effectiveness route for domain-specific models is being validated. Using a smaller base with high-quality legal data fine-tuning can beat larger general models on specific tasks. This is good news for legal teams with limited budgets but clear needs.

The authority of evaluation benchmarks is rising. Harvey LAB is used as a competitive yardstick because it comes from within the industry rather than academic institutions, and its task design is closer to law firms' real workflows. When model vendors start proactively citing results from such benchmarks, it signals that the evaluation system for legal AI is shifting from "can it chat" to "can it work."

Safety alignment and compliance constraints in legal AI: Legal scenarios impose additional compliance requirements on model outputs, including disclaimers that no formal legal advice is being provided, confidential handling of client data, and accuracy constraints on legal citations. This is also one reason many legal-specific models choose application-based access rather than open APIs — vendors need to control usage scope, avoid outputs being misused as formal legal advice, and facilitate monitoring of hallucinations and citation errors in real-world use. Anthropic's accumulated expertise in safety alignment, as well as domestic vendors' caution around data compliance, reflect the same logic: the commercialization of legal AI is not just a capability question but a question of responsibility boundaries.

Questions That Remain

Leading on a single benchmark does not equal comprehensive superiority. Whether GLM-Legal's performance on Harvey LAB translates into stability in actual deployment remains to be verified. Legal tasks have extremely low error tolerance, and a 19.2% all-pass rate means that on most tasks, the model still requires a human lawyer as a backstop.

Moreover, the application-based access mode limits the possibility of independent third-party reproduction and horizontal comparison. As more teams gain access, the true gap between GLM-Legal and other legal models on the same task set will become clearer.

Competition in legal AI has only just entered the "usability" phase, and GLM-Legal's rise to the top is more of a signal: the small-and-specialized vertical route is becoming an important force challenging general large models.

One-line Summary

Z.ai's GLM-Legal tops Harvey LAB with a 19.2% all-pass rate, validating the cost-effectiveness route for small vertical legal models.

Paragraph Summary

Z.ai's legal-specific model GLM-Legal achieved a 19.2% all-pass rate on Harvey's Legal Agent Benchmark (Harvey LAB), surpassing Claude Fable 5 and Kimi K3 to become one of the best-performing legal-specific models on the leaderboard. Built on GLM-5.3-Flash, GLM-Legal's domain fine-tuning brings its results to nearly three times that of the base model, at a cost of about $0.15 per test run. Harvey LAB simulates real lawyer workflows with a strict all-pass metric, so a double-digit pass rate already represents preliminary usability. The article argues this result validates the cost-effectiveness route of "small base + high-quality legal data fine-tuning," and shows that the legal AI evaluation system is shifting from "can it chat" to "can it work." However, leading on a single benchmark does not equal comprehensive superiority, and application-based access limits independent reproduction, so the true gap remains to be seen.