Running Gemma LLM in Godot with GDScript and Vulkan: An Experimental Deep Dive
Running Gemma LLM in Godot with GDScri…
Running a Gemma LLM natively in Godot 4 with GDScript and Vulkan, zero external dependencies.
Developer asallay built a complete LLM inference pipeline inside Godot 4 using only GDScript and Vulkan compute shaders, running a quantized Gemma model with no llama.cpp, Python, or GDExtension required. Though roughly 10x slower than CUDA-based solutions, the project proves local LLM inference is feasible in a pure game engine environment and offers real value for zero-dependency deployment and learning.
A Bold Experiment: Can an LLM Run Natively Inside a Game Engine?
Recently, GitHub user asallay shared a fascinating experiment on Reddit: running a large language model entirely inside the Godot game engine using only GDScript and Vulkan compute shaders — no llama.cpp, no Python, no external servers, and no GDExtension.
The project targets Godot 4.7 and runs the gemma-4-E2B-it-Q4_K_M.gguf model — a 4-bit quantized variant from the Gemma model family. GGUF is a model weight storage format introduced by the llama.cpp project in 2023, bundling all inference-required data — weights, hyperparameters, vocabulary, and more — into a single file, dramatically simplifying model distribution and loading. Q4_K_M refers to a specific quantization scheme that compresses model weights from 32-bit floats down to 4-bit low-precision representations using a K-means grouping strategy at medium quality, shrinking model size by over 75% with acceptable accuracy loss and making it runnable on consumer hardware. The entire inference pipeline runs within the Godot ecosystem with zero external dependencies. While the author repeatedly emphasizes "this is just an experiment," the completeness and viability of the technical approach makes it worth a serious breakdown.
Technical Architecture: A Clear Division of Labor
What makes this project particularly interesting is how it cleanly distributes the two major categories of LLM inference work across Godot's two core mechanisms.
Vulkan Compute Shaders Handle the "Heavy Lifting"
At the heart of any large language model is a massive volume of matrix operations. The author offloads all of these compute-intensive tasks to Vulkan compute shaders.
Vulkan is a next-generation graphics and compute API released by the Khronos Group in 2016, designed to replace OpenGL with lower driver overhead and finer-grained hardware control. Unlike OpenGL, Vulkan was designed from the ground up to treat "graphics rendering" and "general-purpose computation" as parallel capabilities — compute shaders allow developers to execute arbitrary parallel computation tasks directly on the GPU, without having to disguise data as textures or route it through a graphics pipeline. This capability has fueled widespread GPGPU (General-Purpose GPU Computing) adoption. Godot 4 completed a major architectural migration from OpenGL to Vulkan, and exposes the underlying Vulkan capabilities to developers via the RenderingDevice API, making it possible to dispatch compute shaders directly from within the game engine.
This means the model's forward pass, matrix multiplications in the attention mechanism, and other operations are actually running on the GPU — orchestrated by the game engine's rendering backend. It's a clever act of repurposing: using GPU general-purpose compute power, originally prepared for graphics rendering, directly for deep learning inference.
GDScript Handles the "Logic"
Beyond pure computation, a conversational LLM requires a significant amount of "glue logic." The author implemented several key modules in GDScript:
- GGUF file loading: Parsing the model weight format common across the llama.cpp ecosystem;
- Tokenization: Converting input text into token sequences the model can understand. Modern LLMs don't process raw characters or words directly — they split text into subword units (typically generated by BPE algorithms), where each token maps to an integer ID in the vocabulary. Models like Gemma typically have vocabularies of tens of thousands of entries;
- Sampling: Selecting the next token from the model's output probability distribution, using common strategies like Temperature sampling and Top-P sampling;
- KV Cache: During autoregressive generation, the model predicts one token at a time but needs to "remember" the attention computations for all previous tokens. KV Cache stores the key-value pairs from the attention mechanism for past tokens, avoiding redundant recomputation of already-processed content — a critical optimization that multiplies generation speed;
- Chat UI: Building the conversational interface directly using Godot's UI system.
From reading the model file all the way to rendering text on screen, the entire pipeline is implemented in a scripting language that most developers assume is only suitable for writing game logic — which itself challenges a lot of assumptions about GDScript's capability ceiling.
Performance Reality: Roughly 10x Slower Than CUDA Solutions
The author is candid about performance: this approach runs at approximately 1/10th the speed of llama.cpp + CUDA.
This gap is expected, and has deep technical roots. llama.cpp is a C++ project deeply optimized for LLM inference: on the CPU path, it uses hand-written AVX2/AVX-512 SIMD instructions for matrix operations and dedicated dequantization kernels for different quantization schemes; on the GPU path, the CUDA backend leverages NVIDIA's official libraries like cuBLAS and CUTLASS, along with custom CUDA kernels tuned specifically for LLM workloads, fully utilizing hardware units like Tensor Cores for maximum throughput. By comparison, general-purpose Vulkan compute shaders are cross-platform but lack deep hardware-level optimization for matrix multiplication. GDScript itself is an interpreted dynamic language, so tokenization, sampling, and other logic tasks can't match the performance of compiled C++. That 10x gap ultimately reflects the unavoidable trade-off between generality and extreme optimization.
Additionally, the project currently only supports this one specific model, which severely limits its generality. From a "production-ready tool" standpoint, it's clearly not there yet. But as a proof of concept, it achieves its goal perfectly — demonstrating that running an LLM in a pure Godot environment is feasible.
So What's the Real Value of This Experiment?
If it's 10x slower than llama.cpp and only supports a single model, what does this experiment actually offer?
Zero Dependencies Unlock a New Deployment Imagination
The most immediate value is radical deployment simplicity. If a game or application is already built on Godot, embedding a local LLM would require no Python runtime, no bundled llama.cpp binaries, and no additional GDExtension compilation. The entire AI capability becomes part of the engine project itself, inheriting Godot's cross-platform support naturally.
For indie developers looking to integrate local intelligent NPCs, dialogue generation, or narrative assistance into their games, the appeal of "everything inside the engine" is very real.
Pushing the Boundaries of Godot's Compute Capabilities
This project is also an extreme stress test of Godot 4's underlying capabilities. It demonstrates that the RenderingDevice and compute shader interfaces are already powerful enough to support high-intensity general-purpose computing tasks like deep learning inference. This opens up broader possibilities for Godot beyond games — think scientific computing or AI tooling.
Educational and Research Value
Because the entire inference pipeline is implemented in relatively readable GDScript, the logic isn't buried under layers of C++ optimization. This makes the project an excellent learning resource for understanding LLM inference fundamentals. Developers who want to understand how GGUF files are parsed, how KV Cache works, or how sampling strategies are implemented can find a transparent implementation here that doesn't depend on any complex framework — every step from tokenization to autoregressive generation is clearly visible. It's a rare "open it up and look inside" case study for learning how large model inference actually works.
Closing Thoughts: Possibility Itself Is the Point
The last line the author wrote when sharing the project captures the spirit of this kind of experiment perfectly: "Still, I found it interesting that this was possible using only Godot."
Many of the technical community's most meaningful breakthroughs originate from this kind of pure curiosity about what's possible. This project may never become anyone's first choice for a production environment, but it expands the perceived boundary of what a game engine can do — and offers a quirky, thought-provoking reference point for the broader trend toward local AI on edge devices.
The project is open source on GitHub (asallay/godot-llm). Developers who are curious can explore this complete "pure Godot LLM" implementation for themselves.
Related articles
The Truth Behind Codex 'Build a Website in 5 Minutes': AI Isn't Creating Sites—It's Helping You Copy Them
Exposing the truth behind viral Codex 5-minute website videos: creators aren't building original sites with AI—they're copying shared prompts or scraping others' work. Learn AI coding tools' real limits.
Getting Started with AI Agent Development: A Complete Guide from Concept to Practice
A comprehensive guide to AI Agent architecture and development, covering automated marketing, intelligent customer service, and investment analysis scenarios with single and multi-agent collaboration.
The Truth Behind Codex 'Build a Website in 5 Minutes': AI Isn't Creating Sites — It's Helping You Copy Them
Exposing the truth behind viral Codex 5-minute website videos: creators aren't building original sites with AI — they're copying shared prompts or scraping others' work.