Kimi K3 Native Video Understanding Tested: A Head-to-Head with Claude on the Same Questions

Moonshot AI's Kimi K3 supports passing video files directly through the API, while Claude still has no video input channel to this day—to let Claude watch a video, you first have to extract frames into images. Even if Codex CLI is pointed at K3, there's still no video pathway. Developer kenimoto ran a real comparison test using two 40–55 second SSH tutorial hands-on videos, and the results were somewhat surprising.

Test Background: Automatically Generated Tutorial Videos

The test material came from handson-cast, a CLI tool the author is developing. You give it a JSON step file, it actually executes each command inside a Docker container, records the terminal screen with vhs, then adds Japanese narration generated by OpenAI TTS with burned-in subtitles. The underlying stack is Python and ffmpeg.

The two videos are: key authentication (55 seconds), demonstrating creating a key, installing the public key, verifying key-only login, and disabling password authentication; and jump host (47 seconds), demonstrating the same key being rejected by the jump host via ssh -J but succeeding via ~/.ssh/config. All terminal output was genuinely produced by OpenSSH 10.0p2 inside the container—nothing staged.

The problem was that the first version (v1) was completely unwatchable—in the very first second, the screen showed the author typing an internal wrapper script bash /lab/ssh-config.step-01.sh, and viewers had no idea what the video was about. The author himself admitted he couldn't stand watching it. He gave this v1 to both K3 and Claude for analysis.

How to Feed Video to K3

The K3 API is compatible with the OpenAI format. The process is to first upload the video with files.create, get the returned file ID, then reference it with the ms:// prefix. Public URLs are not accepted. The official documentation describes video as a set of keyframes, and the higher the resolution, the more tokens consumed.

Measured data for a 55-second 1080p video: about 40,659 input tokens (almost entirely video), 2,500 output tokens (1,912 of which were reasoning), a cost of about $0.16, and a runtime of 83.5 seconds. Pricing is based on $3 per million input tokens and $15 per million output tokens.

Assume It Can't Hear

The author made a version of the video without burned-in subtitles and asked K3 to transcribe the last line of the narration; K3 replied that it couldn't recognize audio. Not pretending to hear things it can't hear is exactly the attitude an evaluator should have. But it also means K3 cannot perceive narration content or subtitle audio-video offset—it can only see the visuals and the subtitles burned into the picture.

K3 performed well at reading the screen: it accurately listed the commands and outputs of four steps in order, including the 600/644 permissions, changes in the rejection message, and an on-screen annotation. The only thing it misread was the filename of the wrapper script.

When video understanding models process video, the mainstream approach is to decode the video into a sequence of keyframes sampled by time, then feed them as multi-frame images into a vision encoder. This approach inherently discards the audio track, because audio requires a separate speech recognition pipeline to convert it into text. The fact that K3's official documentation describes video as "a set of keyframes" confirms this: the model sees discrete frames rather than a continuous stream. Higher resolution consumes more tokens because each frame image is cut into fixed-size patches and encoded individually; frame count multiplied by patches per frame equals the total visual tokens. The measured ~40,000 input tokens for a 55-second 1080p video allows us to infer its frame sampling density and per-frame encoding cost. This also explains why burned-in subtitles can be read while narration cannot—subtitles have become part of the pixels, while narration is still in the audio track.

Give Enough Output Budget

K3 always reasons before answering. When max_tokens is set to 300, reasoning exhausts the budget and the answer comes back blank; when set to 8,000, the full video review gets truncated midway through the first question. For this kind of long answer, the author recommends using 32,000.

OpenAI-compatible format means K3's API interface aligns with OpenAI's Chat Completions specification in request body structure, field naming, and authentication. So SDKs or code originally written for OpenAI only need the base_url and model name swapped to migrate. files.create is the file upload endpoint under this system, and the returned file ID can be referenced multiple times in subsequent messages, avoiding repeated uploads of large files. ms:// is Moonshot's custom reference prefix, used to point to already-uploaded files within message content, replacing the file:// form used by OpenAI. Public URLs not being accepted means the model side won't proactively fetch external resources—all input must go through the upload process, which adds an extra control barrier for videos containing sensitive content.

Controlled Experiment Setup

K3 and Claude each reviewed the same two v1 videos under matched conditions: K3 consumed the file directly in one shot; Claude read images after frame extraction. On audio, K3 couldn't understand it, and transcription was disabled on Claude's side. Both were given the step JSON and asked five identically worded questions.

Interestingly, Claude's side ran as a sub-agent and never saw K3's answers, avoiding anchoring bias. Frame extraction used claude-video's /watch tool, with frame count and resolution left to Claude's own discretion; as a result it re-extracted 4 times, sampling frames at 0/0.5/1/2/3 seconds at the start, then one frame every 2 seconds, for a total of 34 frames.

Frame extraction strategy is the core variable in Claude's video analysis approach. Too low a frame rate misses brief on-screen changes; too high a frame rate quickly exhausts the context budget. claude-video's /watch tool delegates this decision to the model's own judgment, and the model dynamically chooses sampling intervals based on video duration and question type. In this experiment, Claude ultimately adopted a hybrid strategy of dense sampling at the start (0/0.5/1/2/3 seconds) and one frame every 2 seconds thereafter, for 34 frames total. This "dense at the start, sparse in the middle" approach fits the pattern that short-video hooks concentrate at the beginning. Running as a sub-agent means Claude completes the task in an independent context, and the main conversation doesn't carry K3's answers as reference, thereby eliminating anchoring bias—if a model first sees someone else's conclusions, it tends to agree rather than judge independently.

Result: Findings Are Almost Identical

The two models' answers overlapped heavily. The 0–2 second wrapper script typing causing an unclear topic, the first narration line introducing the website rather than a reason to watch, the strong hook Could not resolve hostname being buried in the jump host video, opening cold with the conclusion, cutting typing and waiting between steps, enlarging the font to highlight key output, making a vertical version, inserting a permissions diagram at ls -l, putting an OG image at the end, publishing the jump host video first—both sides hit all of these.

The differences were only in a few items: K3 additionally suggested adding a progress bar at the top; Claude additionally provided rewritten narration sentences and pointed out that the four ssh-copy-id INFO logs were noise.

These modification suggestions are themselves standard short-video common sense—open with the conclusion, cut the dead air, streamline subtitles, end with a call to action. This comparison did not find a single issue that only K3 could discover. The author had expected a model that could natively read video to see things Claude couldn't, but it didn't.

What Really Sets Them Apart Is Timestamp Precision

The biggest difference was in temporal localization. K3 could write sub-second judgments like "the first meaningful output appears at 3.7 seconds" and "the error at 00:03.2 in the jump host video is buried"; when reviewing the improved version it even gave a range: "between 30.5 and 36.1 seconds, the screen freezes after the output appears."

Claude's timestamps were estimated based on one frame every 2 seconds, and it itself admitted the range boundaries could be off by ±1 second. Sampling could be densified, but the cost is more images to read. Questions like "at exactly which second does the screen freeze" are hard to answer from a few still frames—this is precisely where the value of reading video directly lies.

Sub-second judgments come from the model's internal representation of frame sequence timestamps. When video is decoded into a frame sequence with time information, each frame carries a PTS (presentation timestamp) accurate to the millisecond, and the model can directly reference these values in reasoning. The frame extraction approach loses the temporal continuity between frames: if you take one frame every 2 seconds, any change occurring between two adjacent frames can only be localized to a 2-second-wide interval, and the model can only give a range estimate with self-reported error. Frozen-frame detection especially depends on temporal continuity, because freezing itself is the pattern of "multiple consecutive frames with identical content," and only frame-by-frame comparison can determine the start and end of the freeze. Simply increasing frame extraction density can shrink the error, but token cost grows linearly with frame count, with a point of diminishing returns.

Efficiency and Speed Each Have Their Wins

On K3's side, the author only had to hand over the file, taking 363 seconds and consuming 84,461 input / 10,863 output tokens, about $0.20 (some input billed at cached pricing). On Claude's side, it had to decide the frame extraction strategy (it redid it 4 times), taking 185 seconds and consuming 111,313 tokens and 40 tool calls. K3 required less effort; Claude was faster. Because K3 spends time reasoning, each review takes 6–8 minutes—enough time to brew a coffee and come back.

The Video Really Did Get Better

Following the review feedback, the author added six changes to handson-cast: an opening card "The same key, yet ssh -J is rejected. Why?"; cutting the wrapper script typing and step waits; typing commands character by character with a larger font; showing the article illustration full-screen for 4.5 seconds after the rejection occurs; slowly zooming into the top-left of the terminal after output completes; and putting the OG image and URL at the end.

The jump host video shrank from 46.8 seconds to 41.0 seconds. The author used a 10-item technical short-video checklist to have K3 score the improved version (covering a hook within 3 seconds, visual change every 1–3 seconds, a short call to action only at the end, etc.), and the result was 7 passes, 2 borderline, and 1 estimated pass; the two borderline items—the freeze segment after output and overly long subtitles—have now been fixed.

Incidentally, the author originally wanted to use K3's web search $web_search for checklist research, but on K3 the second call (the one returning search results) failed every time with 400 tokenization failed; kimi-k2.6 could get through the first search, but afterward stopped searching and only wrote text in the answer that looked like a search call. Documentation shows the feature is scheduled for retirement on 2026-10-20. The author gave up, switched to Claude's web search to complete the research, and then handed the results to K3.

Division of Labor

Based on this test, the author's conclusion is: use K3 when you need to know "at which second something happens in the video"—questions like frozen-frame intervals, the moment the screen gets cut off, and subtitle dwell time have answers that are a time span, and you don't need to worry about frame extraction strategy. For deciding "how to change it," either works; there's no difference in improvement ideas. Anything involving audio should not use K3, and narration content and synchronization issues require a separately prepared transcript for comparison. Also, videos get uploaded to Moonshot AI's servers, so internal screen recordings need separate handling.

The author also acknowledges this is a small-sample comparison: two videos, one set of questions, no statistical significance. But it gives a usable answer—reading video directly didn't change the judgment; what it changed was timestamp precision and effort invested. Next, he plans to deliberately plant flaws in videos and see how many each model can find.