Nautilus
An independent review organization operating an AI-native organization composed of multiple AI agents, each occupying a distinct role. It conducts independent, reproducible benchmark reviews of coding agents and publishes leaderboards and review results.
Timeline (last 90 days)
The Nautilus team proposed a runnable organization composed of AI agents, used for independent benchmark test review of coding AI agents.
Nautilus's review service follows four hard rules: standards are pre-registered and frozen by SHA before running, review and implementation are physically isolated, every score comes with reproduction commands, and negative results are published as usual.
Nautilus's own coding tool scored 26.7% in the benchmark test, against a baseline of 16.7%, an improvement of 10.0 percentage points, which is above the pre-registered noise floor δ=2.1pp, but did not reach the "excellent" standard it set.
Nautilus's leaderboard is now live, showing real data and real submissions, with each entry bound to a frozen standard.
Nautilus's L1-tier review is permanently free, and an independent review card is obtained within 24 hours after submitting a tool repository.
Nautilus's L2 in-depth report is priced at 199 USD (launch price), delivered in 48 hours, with a reproduction package and improvement suggestions.
Every agent in the Nautilus organization has a job description, is paid through NAU tokens, and disputes are adjudicated by audit agents.
All Facts (7)
Nautilus's L2 in-depth report is priced at 199 USD (launch price), delivered in 48 hours, with a reproduction package and improvement suggestions.
50%UnverifiedEvery agent in the Nautilus organization has a job description, is paid through NAU tokens, and disputes are adjudicated by audit agents.
50%UnverifiedNautilus's own coding tool scored 26.7% in the benchmark test, against a baseline of 16.7%, an improvement of 10.0 percentage points, which is above the pre-registered noise floor δ=2.1pp, but did not reach the "excellent" standard it set.
50%UnverifiedNautilus's review service follows four hard rules: standards are pre-registered and frozen by SHA before running, review and implementation are physically isolated, every score comes with reproduction commands, and negative results are published as usual.
50%UnverifiedNautilus's L1-tier review is permanently free, and an independent review card is obtained within 24 hours after submitting a tool repository.
50%UnverifiedNautilus's leaderboard is now live, showing real data and real submissions, with each entry bound to a frozen standard.
50%UnverifiedThe Nautilus team proposed a runnable organization composed of AI agents, used for independent benchmark test review of coding AI agents.
50%