EmbeddingGemma 2 In-Browser Image-Text Retrieval: Local Inference with WebGPU

EmbeddingGemma 2 Puts Images and Text Into the Same Semantic Space

The core challenge of multimodal retrieval is making a photo and a sentence computationally comparable. EmbeddingGemma 2 addresses this by mapping both images and text into a shared 768-dimensional vector space. In practice, when you describe a photo in natural language, the model converts that description into a vector and then searches for the nearest image vector in the same space — this is the fundamental logic behind image-text retrieval.

This shared embedding space design isn't new. The CLIP family of models has already proven its effectiveness, but EmbeddingGemma 2 brings this capability into the Gemma ecosystem. It provides both a text tower and a vision tower that developers can call independently, making it flexible enough to compose into retrieval, classification, or semantic search applications.

CLIP (Contrastive Language–Image Pre-training), introduced by OpenAI in 2021, uses large-scale image-text pairs with contrastive learning: during training, matching image-text pairs are pulled closer together in vector space while mismatched pairs are pushed apart. This approach lets the model naturally learn cross-modal semantic alignment without needing task-specific labeled data. Subsequent variants like SigLIP and ALIGN further improved training efficiency and retrieval accuracy, and the shared embedding space has become the standard paradigm for multimodal retrieval. EmbeddingGemma 2 follows this same approach and integrates it into Google's Gemma open-source model family, meaning developers can access both language model capabilities and multimodal retrieval within the same ecosystem, reducing friction from cross-framework integration.

The Real Highlight: Running Entirely on the Browser GPU

The most noteworthy aspect of this project isn't the model itself — it's how it runs. The developer ported EmbeddingGemma 2's text tower and vision tower to ruNNtime, a TypeScript-based WebGPU inference library, and built a small photo gallery demo on top of it. The entire retrieval process runs completely on the user's local GPU, with no server involvement whatsoever.

This has several practical implications. Inference requires no backend, keeping deployment costs extremely low — a static webpage can host the full AI search functionality. User photo data never leaves the local device, providing inherent privacy advantages. It also lowers the barrier to entry: just open the webpage and try it, no environment setup or API key required.

WebGPU, as the next-generation browser graphics and compute API, offers computing capabilities far closer to native compared to earlier WebGL, and is gradually becoming a practical choice for running machine learning models on the frontend. Libraries like ruNNtime are filling the engineering gaps on the path toward running real models in the browser.

WebGPU is a W3C-standardized next-generation web graphics and general-purpose compute API, enabled by default in Chrome, Edge, and other major browsers since 2023. Compared to WebGL, which was previously widely used for in-browser machine learning, WebGPU exposes lower-level GPU compute primitives (Compute Shaders) that support more general parallel computation beyond the graphics rendering pipeline, delivering performance closer to native for core neural network operations like matrix multiplication. While existing libraries like ONNX Runtime Web and TensorFlow.js already partially support WebGPU backends, ruNNtime is specifically designed around WebGPU's compute capabilities, managing GPU resources natively in TypeScript without needing WASM or native plugins as intermediaries.

ruNNtime: A WebGPU Inference Library for the Browser

Beyond EmbeddingGemma 2, ruNNtime supports a variety of vision models and provides hands-on examples in its interactive documentation. The project lives at github.com/software-mansion/runntime.

For developers, ruNNtime's value lies in encapsulating the low-level WebGPU inference details behind TypeScript, the mainstream frontend language. Compared to deploying models in the cloud and calling them via API, a purely frontend approach offers distinct advantages in interaction latency, offline availability, and data privacy — making it especially well-suited for privacy-sensitive scenarios like image semantic search and local photo library management.

Understanding the Boundaries

It's worth being realistic: browser-side inference isn't a universal solution. Limited by device GPU performance, the network overhead of loading model weights, and memory constraints, this approach currently works best for small-to-medium embedding models and retrieval tasks — not large generative models with billions of parameters. EmbeddingGemma 2, as an embedding model, has a relatively manageable parameter count that falls squarely within what WebGPU can handle, which is part of why it makes for a suitable browser demo.

The memory bottleneck for browser-side inference typically comes from two sources: the size of the model weights themselves, and the activation cache during inference. Current mainstream browsers impose limits on the size of individual GPU buffers, and device VRAM is far smaller than server GPUs — making it still quite challenging to run generative models with tens of billions of parameters in the browser. Embedding models only need to map inputs to fixed-dimensional vectors without autoregressive decoding, so inference involves short sequence lengths and low memory requirements, making them one of the most practical model categories for browser-side AI deployment today. Additionally, model weights need to be downloaded over the network on first load, so sensible model quantization (such as INT8/FP16) and browser caching strategies have a significant impact on user experience.

What This Practice Reveals About Broader Trends

Bringing embedding models into the browser reflects a broader shift: on-device AI is moving from concept to practical reality. "Local inference" used to mean mobile apps or desktop programs, but WebGPU makes the webpage itself a viable inference host. For tasks that don't require massive compute, developers are increasingly inclined to push computation as close to the user as possible — saving on server costs while avoiding data transmission compliance risks.

EmbeddingGemma 2's multimodal embedding capability combined with ruNNtime's WebGPU wrapper forms a lightweight, reusable technical stack. For developers looking to explore browser-side AI applications, this is a compelling starting point: the model handles semantic understanding, the runtime handles moving inference to the GPU, and the end result is just a webpage that works the moment you open it.

If you're interested in image-text retrieval or on-device inference, head directly to the project repository and try searching photos yourself in the interactive documentation.