Matrix-Game 4.0
A real-time interactive video large-model game world generation system from Kunlun Wanwei. It employs a dual-pipeline architecture in which a game engine maintains world state and a video generation model renders the visuals, and it can run in real time on a single consumer-grade graphics card.
Timeline (last 90 days)
Matrix-Game 4.0, released by Kunlun Wanwei, is the first video large-model game that runs on a single consumer-grade GPU, and it is open-sourced at the same time.
The core architectural decision of Matrix-Game 4.0 is to split the game engine (which maintains world state and interaction rules) and the video generation model (which is responsible for rendering frames in real time) into two parallel pipelines.
Matrix-Game 4.0 runs at 60 FPS at 480p resolution on a single NVIDIA RTX 5090, with end-to-end latency below 150 milliseconds.
Matrix-Game 4.0 achieves a steady-state generation speed of 83 FPS for single-step denoising, with a peak VRAM usage of 18.5 GiB.
Matrix-Game 4.0 has an end-to-end latency of 112 milliseconds, whereas Code World Model is about 10.2 seconds, StreamRender-H3 about 545 milliseconds, and AgentGarten about 878 milliseconds.
Matrix-Game 4.0 requires only a single consumer-grade GPU, whereas Code World Model requires 1× H200 141GB, StreamRender-H3 requires 8× GB200, and AgentGarten requires 1× H100 datacenter card.
Matrix-Game 4.0 employs a 30-layer causal DiT, in which self-attention reads the K/V cache and cross-attention reads the text K/V, thereby reusing historical computation during frame-by-frame generation.
Matrix-Game 4.0 is adapted from Wan2.2 5B into a video-to-video mode, in which the structural frames rendered by the engine and the video to be generated are concatenated by channel (148-channel input).
Matrix-Game 4.0's inference acceleration uses CUDA Graph to pre-record and replay the DiT backbone computation, writes fused kernels for RoPE/normalization/FP8 quantization, recomputes the text-conditioning KV only when the Caption updates, and switches the video history KV to a ring buffer.
Matrix-Game 4.0's data layer builds a structural-frame-to-fine-frame cross-domain alignment dataset based on Unreal Engine 5, covering four major scenes: the CitySample megacity, open desert, complex castle, and high-dynamic air combat.
3 more timeline events
All Facts (13)
Matrix-Game 4.0 is more open-sourced than Code World Model, StreamRender-H3 (already open) and AgentGarten (partially open).
50%UnverifiedThe core architectural decision of Matrix-Game 4.0 is to split the game engine (which maintains world state and interaction rules) and the video generation model (which is responsible for rendering frames in real time) into two parallel pipelines.
50%UnverifiedMatrix-Game 4.0 runs at 60 FPS at 480p resolution on a single NVIDIA RTX 5090, with end-to-end latency below 150 milliseconds.
50%UnverifiedMatrix-Game 4.0 achieves a steady-state generation speed of 83 FPS for single-step denoising, with a peak VRAM usage of 18.5 GiB.
50%UnverifiedMatrix-Game 4.0 has an end-to-end latency of 112 milliseconds, whereas Code World Model is about 10.2 seconds, StreamRender-H3 about 545 milliseconds, and AgentGarten about 878 milliseconds.
50%UnverifiedMatrix-Game 4.0 requires only a single consumer-grade GPU, whereas Code World Model requires 1× H200 141GB, StreamRender-H3 requires 8× GB200, and AgentGarten requires 1× H100 datacenter card.
50%UnverifiedMatrix-Game 4.0 employs a 30-layer causal DiT, in which self-attention reads the K/V cache and cross-attention reads the text K/V, thereby reusing historical computation during frame-by-frame generation.
50%UnverifiedMatrix-Game 4.0 is adapted from Wan2.2 5B into a video-to-video mode, in which the structural frames rendered by the engine and the video to be generated are concatenated by channel (148-channel input).
50%UnverifiedMatrix-Game 4.0's inference acceleration uses CUDA Graph to pre-record and replay the DiT backbone computation, writes fused kernels for RoPE/normalization/FP8 quantization, recomputes the text-conditioning KV only when the Caption updates, and switches the video history KV to a ring buffer.
50%UnverifiedMatrix-Game 4.0's data layer builds a structural-frame-to-fine-frame cross-domain alignment dataset based on Unreal Engine 5, covering four major scenes: the CitySample megacity, open desert, complex castle, and high-dynamic air combat.
50%UnverifiedWith the help of a Coding Agent, players of Matrix-Game 4.0 can customize the game in natural language on top of the existing game code, including adjusting terrain, placing scene objects, and modifying NPC behavior and interaction logic.
50%UnverifiedMatrix-Game 4.0 was open-sourced at the time of release, with the code, weights, and white models all made available, and the Huggingface repository address is Skywork/Matrix-Game-4.0.
50%UnverifiedMatrix-Game 4.0, released by Kunlun Wanwei, is the first video large-model game that runs on a single consumer-grade GPU, and it is open-sourced at the same time.
50%