Artificial Analysis launches Harvey LAB-AA to benchmark legal models

Artificial Analysis has rolled out Harvey LAB-AA, an independent benchmarking framework that tests AI language models on 120 private legal tasks spanning 24 practice areas. The top performer, Claude Fable 5, managed an all-pass rate of just 14.2%.

What Harvey LAB-AA actually measures

Harvey LAB-AA isn’t your typical chatbot leaderboard. It’s an independent implementation of Harvey AI’s Legal Agent Benchmark, which was originally open-sourced on May 6, 2026. The full benchmark includes more than 1,200 tasks evaluated against over 75,000 expert-defined rubric criteria.

The “all-pass” grading standard is deliberately brutal. A task only counts as successful if every single rubric criterion is met. Not most of them. Not the important ones. All of them.

Artificial Analysis runs its evaluations using a system called the Stirrup harness, applied to Harvey’s private task subset. The framework reports both task completion rates and specific criterion performance metrics.

The benchmark evaluates agentic legal work, meaning the models need to produce meaningful legal documents like memos and analyses, not just answer questions about the law.

The leaderboard and what it tells us

Claude Fable 5 sits at the top with its 14.2% all-pass rate. That’s nearly double the performance of its closest competitors, Claude Opus 4.8 and GLM-5.2, which each recorded a 7.5% all-pass rate.

Claude Fable 5, released on June 9, 2026, has been on a tear across multiple benchmarks since launch. On Harvey’s own internal LAB evaluation, it scored a 13.3% all-pass rate, slightly below the 14.2% it achieved on the Artificial Analysis implementation.

The bigger picture for legal AI

Harvey, the company behind the original LAB benchmark, is valued at $11 billion. The company has partnered with major model providers including Anthropic, OpenAI, and Nvidia on the benchmark.

The decision to open-source the benchmark and then have it independently evaluated by Artificial Analysis adds a layer of credibility that proprietary benchmarks often lack. The benchmark covers 24 practice areas and includes 75,000 expert-defined rubric criteria, making it a domain-specific evaluation that should expose uneven model capabilities across different areas of legal work.

Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our Editorial Policy.