Harvey LAB
An agent benchmark in the legal AI domain that simulates real lawyer workflows, requiring models to perform end-to-end tasks such as multi-step reasoning, document retrieval, clause comparison and structured output. It uses an all-pass scoring criterion.
Timeline (last 90 days)
The GLM-Legal model launched by Z.ai achieved a 19.2% all-pass rate on the Harvey LAB legal Agent benchmark, surpassing Claude Fable 5 and Kimi K3.
Harvey LAB (Legal Agent Benchmark) simulates Agent tasks from real lawyer workflows, requiring the model to have comprehensive capabilities such as multi-step reasoning, document retrieval, clause comparison, and structured output.
Harvey LAB adopts an all-pass criterion, where the model must be correct in all sub-steps of a task; any error in any step means it does not count.
The running cost of each test case on Harvey LAB is about $0.15.
GLM-Legal's 19.2% all-pass rate on Harvey LAB means that on most tasks the model still needs a human lawyer as a fallback.
Harvey LAB comes from within the industry rather than an academic institution, and its task design is closer to the real workflows of law firms.
All Facts (6)
The GLM-Legal model launched by Z.ai achieved a 19.2% all-pass rate on the Harvey LAB legal Agent benchmark, surpassing Claude Fable 5 and Kimi K3.
50%UnverifiedHarvey LAB (Legal Agent Benchmark) simulates Agent tasks from real lawyer workflows, requiring the model to have comprehensive capabilities such as multi-step reasoning, document retrieval, clause comparison, and structured output.
50%UnverifiedHarvey LAB adopts an all-pass criterion, where the model must be correct in all sub-steps of a task; any error in any step means it does not count.
50%UnverifiedThe running cost of each test case on Harvey LAB is about $0.15.
50%UnverifiedGLM-Legal's 19.2% all-pass rate on Harvey LAB means that on most tasks the model still needs a human lawyer as a fallback.
50%UnverifiedHarvey LAB comes from within the industry rather than an academic institution, and its task design is closer to the real workflows of law firms.
50%