Harbor-Index
A high-quality metadata collection provided by the Harbor framework, containing 82 carefully selected high-difficulty evaluation tasks spanning 29 benchmarks. Items are filtered for difficulty and reviewed through both AI and human verification, forming a high-discriminability evaluation set for measuring comprehensive AI agent capabilities.
Timeline (last 90 days)
The strongest configuration, GPT-5.5 paired with Codex, achieved a pass rate of only 28.0% on Harbor-Index
Harbor-Index is a curated set of 82 high-difficulty tasks spanning 29 benchmarks
The construction of Harbor-Index went through multiple rounds of filtering, including difficulty filtering, dual AI and human review, and review-fix iteration cycles
All evaluated model-harness configurations achieved pass rates of no more than 30% on Harbor-Index
The research team released all adapters, evaluation results, in-depth analysis reports, and Harbor-Index as open-source resources
Harbor-Index significantly reduces operational costs while maintaining evaluation breadth and challenge
All Facts (6)
The strongest configuration, GPT-5.5 paired with Codex, achieved a pass rate of only 28.0% on Harbor-Index
50%UnverifiedHarbor-Index is a curated set of 82 high-difficulty tasks spanning 29 benchmarks
50%UnverifiedThe construction of Harbor-Index went through multiple rounds of filtering, including difficulty filtering, dual AI and human review, and review-fix iteration cycles
50%UnverifiedAll evaluated model-harness configurations achieved pass rates of no more than 30% on Harbor-Index
50%UnverifiedHarbor-Index significantly reduces operational costs while maintaining evaluation breadth and challenge
50%UnverifiedThe research team released all adapters, evaluation results, in-depth analysis reports, and Harbor-Index as open-source resources
50%