Kimi K3.1 Pro Benchmark Results Revealed: First Place in 6 of 14 Tests, Coding Nearly Catches Opus

Moonshot AI released the full benchmark results for Kimi K3.1 Pro late at night, and the model is already available for direct use—this is not a teaser. It ranks first in 6 of 14 tests, with particularly impressive performance on coding and terminal tasks, closing the gap with Claude Opus to single digits or even less than 1 point.

Coding and Terminal: Closing In on Opus

The most closely watched field-task coding score is 51.7, surpassing Opus's 46.3 by 5.4 points. The terminal combat score is 65.7, just 0.7 points behind Opus while also overtaking Astra. For a model that built its reputation on long-context text, this level of improvement shows that Moonshot AI has made targeted optimizations in code generation and terminal interaction.

From a practical standpoint, coding and terminal are the two most core capabilities for agent deployment. K3.1 Pro approaching the top tier on these two fronts means its usability in scenarios such as automation scripts, code completion, and command-line tasks has improved significantly. However, the 0.7-point gap also reminds us that competition among leading models has entered a decimal-point tug-of-war.

Knowledge and Office: First Place on Tool-Assisted Exams

Knowledge and office scenarios also have highlights. The tool-assisted Humanity's Last Exam score reached 70, and office Q&A scored 64.4—both first in the table. This shows that K3.1 Pro's comprehensive ability to call external tools to complete complex Q&A is already competitive, rather than just excelling at isolated benchmarks.

3D modeling scored 92.7, entering the top tier; computer operation scored 79.6, staying neck-and-neck with the leaders. But Moonshot AI did not sweep every leaderboard—the scientific terminal task scored 38.1, while Astra scored 68.1, a still-obvious gap. This also shows that K3.1 Pro's capabilities are unevenly distributed, with clear shortcomings on terminal tasks requiring deep scientific reasoning.

The "Humanity's Last Exam" mentioned in the article refers to a class of high-difficulty knowledge Q&A benchmarks (such as Humanity's Last Exam), with questions written by domain experts covering mathematics, natural sciences, humanities, and other disciplines. The questions are deliberately designed to avoid answers that search engines can directly surface, making it a "ceiling" test for evaluating a model's true knowledge depth and reasoning ability. "Tool-assisted" means the model can call external capabilities such as retrieval, code execution, and calculators while answering, rather than relying purely on parametric memory. This setup is closer to real agent workflows, but it also means scores are affected by tool orchestration strategy, so comparisons need to be aligned with equivalent configurations for other models.

Hallucination Rate Drops Sharply: From 23.4% to 8.3%

More noteworthy than the benchmark scores is the improvement in hallucination rate. On internal high-difficulty tests, K3.1 Pro's hallucination rate dropped from K3's 23.4% to 8.3%, a decrease of more than 15 percentage points. The model's tendency to confidently fabricate information has clearly decreased, which may be more valuable for real business scenarios than any single benchmark score.

Hallucination rate is a foundational metric for model trustworthiness. In scenarios requiring accurate output—coding, office work, knowledge Q&A—a lower hallucination rate means less manual review cost. The fact that Moonshot AI highlighted this item separately also shows the team understands what enterprise users really care about.

Hallucination rate usually refers to the proportion of content in model outputs that appears plausible but is factually incorrect or unsupported. There are two common evaluation methods: first, the model is judged to have hallucinated if it still answers when it knows information is insufficient, testing its ability to "refuse/acknowledge not knowing"; second, a judge model or humans fact-check the output. The "internal high-difficulty test" in the article is a self-built evaluation set, so this number should not be directly compared horizontally with hallucination rates published by other vendors. However, the longitudinal comparison from K3 to K3.1 Pro on the same set of questions still has reference value. Reducing hallucinations is often accompanied by a higher refusal rate or more conservative answers, so in real deployment a trade-off between accuracy and coverage is still needed.

High-Speed Version and Open-Source Plan

In addition to the standard version, the official announcement also introduced a high-speed version, claiming 2-3x faster speed, support for million-token context, and a plan to release full weights. According to the official statement, the weights will be released before November 4. The million-token context continues the Kimi series' long-text tradition, while the high-speed version targets latency-sensitive production environments.

The plan to open full weights is worth noting. If implemented as planned, developers can deploy and fine-tune locally, which would do more for ecosystem building than simply topping leaderboards. However, there are still no further details on the specific license terms and hardware requirements for the weight release.

Million-token context (1M tokens) refers to the upper limit of input length the model can process in a single pass, roughly equivalent to hundreds of thousands of Chinese characters or hundreds of pages of documents. The key to long-context capability is not just "being able to stuff it in," but also the recall accuracy of information in the middle of long texts, commonly measured in the industry by "needle-in-a-haystack" tests. High-speed versions usually achieve 2-3x throughput and lower latency through methods such as distilling smaller models, speculative decoding, MoE sparse activation, or dedicated inference acceleration, often at the cost of a small drop in scores on complex reasoning tasks. As for "releasing full weights," what the industry calls open source mostly means downloadable weights; whether commercial use is allowed and whether training data and license restrictions are included must be determined by the final released license.

Uneven Capabilities, but a Clear Direction

Overall, K3.1 Pro offers a report card with trade-offs: coding and terminal have closed in on Opus, knowledge and office take first place, 3D modeling enters the top tier, but the scientific terminal task lags clearly behind Astra. It is not an across-the-board domination, but a focused breakthrough in key scenarios.

For developers, the most practical question is whether to use it for coding or for 3D. Based on the benchmark scores, coding and terminal tasks already have the strength to replace some leading models, while 3D modeling, although scoring well, still needs time to prove ecosystem and toolchain maturity. The model is already available; the final answer will have to come from real projects.

One-line Summary

Kimi K3.1 Pro benchmark results are out: coding and terminal approach Opus, hallucination rate drops to 8.3%

Paragraph Summary

Moonshot AI released the full benchmark results for Kimi K3.1 Pro late at night, and the model is already available for direct use. It ranks first in 6 of 14 tests: field-task coding scores 51.7, surpassing Claude Opus's 46.3; terminal combat scores 65.7, just 0.7 points behind Opus; tool-assisted Humanity's Last Exam scores 70 and office Q&A scores 64.4, both first in the table; 3D modeling scores 92.7, entering the top tier. But the scientific terminal task scores only 38.1, far behind Astra's 68.1, showing an uneven capability distribution. More notably, the internal high-difficulty hallucination rate dropped from K3's 23.4% to 8.3%. The official announcement also introduced a high-speed version that is 2-3x faster, supports million-token context, and plans to release full weights before November 4.