Kimi K3.1 Hands-On Test: Open-Source Model Nears Opus, Video Generation Steals the Show

Title

Kimi K3.1 Hands-On Test: Open-Source Model Nears Opus, Video Generation Steals the Show

Meta Description

Kimi K3.1 completed motion graphics, 3D rendering, mini-games, and video generation tasks in a live hands-on test. This article analyzes the real-world performance of its 2.8-trillion-parameter MoE architecture, how it feels compared to Opus/GPT, the video generation capability the official team overlooked, and the workflow shifts in an era of using multiple models.

Article Content

A livestream originally themed around AMA (Ask Me Anything) turned into an impromptu model evaluation session thanks to the surprise release of Kimi K3.1. Host Matt, along with guests Alex and Brian, answered viewer questions while running a series of extreme Kimi K3.1 tasks live on camera—from motion graphics and 3D rendering to complete mini-games and video generation. This unscripted hands-on test unexpectedly exposed the most genuine capability boundaries of current open-source models.

What Is Kimi K3.1: A Three-Trillion-Parameter Open-Source Beast

Shortly after the stream began, Matt installed the official Kimi app directly on his computer and ran Kimi K3.1 Pro High mode. He joked half-seriously that "installing Kimi on my own computer makes me a bit nervous."

From the information revealed during the stream, K3.1 uses a MoE architecture with approximately 2.8 trillion total parameters and 104 billion active parameters. This activation ratio is quite aggressive among comparable models, meaning inference costs stay manageable while retaining large-scale knowledge. It supports a million-token context, native multimodality, and can directly output code-driven game projects covering Unreal, Unity, and Godot.

According to official documentation shown during the stream, K3.1 shows clear improvements over its predecessor in coding and agentic tasks, with hallucination rates on financial tasks dropping by about 65%. Team member Alex said on air that he's a long-time Kimi user, for a straightforward reason: "We originally thought open source was six months behind closed source, but when Kimi 3 came out we found the gap was only two or three months."

According to concurrent reviews from multiple Bilibili creators and YouTube tech channels, K3.1 has indeed entered the top tier in code generation and long-context tasks, but still lags behind top closed-source models in some fine-grained visual tasks—which largely matches the impressions from this livestream.

MoE (Mixture of Experts) architecture is key to understanding K3.1's parameter scale. Traditional dense models activate all parameters on every inference, while MoE splits the model into multiple "expert" subnetworks, with a routing gate network deciding which experts process each token. So although K3.1 has up to 2.8 trillion total parameters, each inference only activates 104 billion, keeping compute costs close to a hundred-billion-scale dense model. This design lets the model "stack knowledge" without "stacking costs," but also brings engineering challenges like high VRAM usage, uneven expert load balancing, and difficult fine-tuning. A lower activation ratio (active parameters / total parameters) generally means cheaper inference, but demands higher routing quality and training stability.

Test 1: Motion Graphics and 3D Rendering—Coding Skills Pass, But Aesthetics Fall Short

The first test asked K3.1 to generate a tutorial video: using motion graphics to demonstrate how to tie a belaying knot for climbing. Matt specifically noted the compound nature of this task—it requires coding ability, video transcoding, spatial understanding, and either the model's own knowledge of knots or proactive web search.

The result came out in five minutes: a 28-second 1080p video.

Objective assessment: the code-driven animation logic is correct, the rope's interweaving relationships hold up at the code level, and it even thoughtfully added a disclaimer: "Please learn and verify with a qualified climbing instructor." But the visual quality is typical of "code-generated" output—solid color blocks, flat, lacking material texture.

The following comparison makes the point even clearer. The same prompt given to GPT-6.1 (referred to as "6.1 Sole Medium" in the stream) produced an animation with noticeably clearer over-under rope relationships and genuine 3D depth. Matt's assessment was "about 10% to 15% better," while Alex bluntly said "still not good enough."

When Matt asked K3.1 to redo it in Blender with braided rope textures and continuous camera movement, the result was still disappointing. The key flaw was shadows: the model applied a uniform shadow to the entire rope rather than calculating shadows separately at each over-under crossing, making the spatial relationships look flat.

This test reveals an industry-wide pattern: open-source models have approached or even matched top closed-source models at the "code correctness" level, but "visual taste" and rendering detail remain a hurdle.

Motion Graphics refers to animation forms primarily built from dynamic shapes, text, icons, and color blocks, as distinct from live-action video and photorealistic 3D rendering. It's typically driven by code or keyframes, emphasizing information delivery and visual rhythm. Asking a large model to generate motion graphics essentially requires it to do three things at once: write runnable animation code (such as Manim, Processing, or Blender Python), ensure correct timeline and spatial relationships, and possess graphic-design-level aesthetic judgment. The first two fall under "code correctness," while the latter is "visual taste"—which is precisely hard to measure through benchmarks and is the root cause of why current model outputs are often described as "too code-like."

Test 2: Mini-Games and Video Generation—Kimi Had an Unadvertised Card Up Its Sleeve

The game testing segment produced the most dramatic moment of the stream.

Matt had K3.1 build a Fall Guys-style obstacle course mini-game called "Bean Blitz." It turned out all the obstacles were non-functional—the character could roll straight from start to finish. "I got first place on my first try." The team called the work "charming"—rough, but at least honestly rough.

The real highlight came from another test thread: having the model make a parody video about Theo (YTP style). K3.1 called upon a capability almost no one knew about—Kimi's built-in video generation model.

The result stunned everyone present: a five-minute finished piece with complete narrative, character movement, camera cuts, and even an 8-bit electronic soundtrack (though the audio didn't play during the stream). Brian's comment was "if you didn't tell me, I couldn't tell this was made by AI," and Matt immediately said he wanted to post the video on X.

This information deserves special attention: Moonshot barely promoted its video generation capability when releasing K3.1, yet the API is already usable. According to logs read during the stream, the call path goes through a Python FFmpeg processing pipeline. This may make it one of the few models in the current open-source camp with end-to-end video generation capability.

YTP (YouTube Poop) is a parody editing culture originating from early YouTube, characterized by clip splicing, audio-visual mismatches, repetition and looping, and absurd narratives, with a deliberately rough, anti-logic style. Generating YTP-style videos with AI is a challenging test: it doesn't just require visual coherence, but also requires the model to understand this subculture's "intentionally chaotic" aesthetic and match it in narrative pacing, camera cuts, and music. The fact that K3.1 could produce a five-minute finished piece with complete narrative and 8-bit soundtrack during the stream shows its video generation pipeline already has some long-sequence consistency and style control—not just a few-second clip demo.

Test 3: The Mismatch Between Benchmark Scores and "Feel"

The stream showed a K3.1 benchmark comparison chart. On metrics like Terminal Bench, Humanity's Last Exam, and GPQA, K3.1 sits right next to Opus 5.5 and GPT-6 Astra Fable 5.1, even surpassing them in some categories.

But Matt poured cold water on it live: "Benchmark scores and actual feel are two different things. I've only run four tests, but I can already feel it's not at Opus's level." He also pointed out a suspicious detail—one comparison chart's x-axis labels were generic terms (coding, reasoning, mathematics, agentic) rather than specific benchmark names, and Kimi K3.1 was first in every category. "This looks like a chart they made themselves."

This skepticism is worth noting. There's inherent tension between official benchmarks and third-party reproduction, especially as open-source model competition heats up—benchmark credibility itself is becoming an issue.

In the same stream, the team also compared DeepSeek^Ad V4.1 Flash (version labeled 10.17). On Matt's local DGX Station, DeepSeek ran at 70-80 tokens/second with power consumption around 1600 watts. The subtext was clear: open-source model capabilities have caught up, but the hardware barrier for self-hosted inference remains a real cost.

The Real Workflow in the Multi-Model Era

Setting aside model evaluation, the discussion about daily toolchains was actually more information-dense.

Matt revealed his workflow has fragmented into several parts: GrokBot for personal assistant, Codex and Claude as primary coding tools, and automation tools for deep automation. Alex shared a key usage pattern—connecting GrokBot to Cursor, letting the former call the latter with additional context. According to him, SpaceX's engineering team widely adopts this approach "because GrokBot has context that Cursor alone doesn't."

The team also mentioned a subscription consolidation trend: xAI's Unified subscription bundles GrokBot, Cursor, and X Premium together, as personal AI subscriptions shift from "buying per tool" to "buying per platform."

One detail from the stream nicely illustrates the maturity of personal agents: when Matt manually sent a guest email invitation, he misspelled Alex's address, and minutes later a personal assistant proactively popped up a notification saying "you made a typo, suggest following up to correct it." This kind of proactive error correction reflects productization level better than any benchmark score.

Cursor is an AI coding editor based on VS Code, whose core capability is integrating codebase context, multi-file editing, and model calls into the IDE workflow. Connecting a general-purpose personal assistant like GrokBot "into Cursor" is typically done through MCP (Model Context Protocol) or custom tool calls: an assistant with broader context handles understanding task background and breaking down steps, then calls Cursor to complete specific code modifications. This layered "outer assistant + inner coder" approach reflects a reality—a single model struggles to simultaneously excel at long-term memory, cross-application coordination, and fine-grained code editing, and combined calls can compensate for each other's context blind spots.

Three Judgments from One Livestream

First, the gap between open and closed source is narrowing—but what's narrowing is the capability floor, not the ceiling. K3.1 can complete complex compound tasks, build its own games, and even produce videos—unimaginable two years ago. But detail polishing, visual aesthetics, and complex spatial reasoning—the "last mile"—remain the moat of top closed-source models.

Second, video generation is Kimi's underrated capability. With almost zero official promotion, hands-on testing reached "indistinguishable to the naked eye" quality. If this capability opens up as an API, its impact on content creation workflows would be quite direct.

Third, benchmark chart credibility is being overdrawn. When every metric in a chart favors the company itself and the x-axis uses generic terms, audience trust gets discounted. As model competition enters its second half, the value of evaluation transparency will only grow.

At the end of the stream, Matt said something honest: "I have to choose among all the tools—GrokBot for personal assistant, Codex and Claude for coding—I have to try everything." This is probably the real state of today's heavy AI users—no silver bullet, only assembly.