Google EmbeddingGemma 2: The First Native Multimodal On-Device Embedding Model Explained

One Vector Space for Five Modalities

Google has released EmbeddingGemma 2, an embedding model with 740M total parameters and Google's first native multimodal, on-device open-source embedding model. It maps text, code, images, audio, and video into a single shared vector space. Released under the Apache 2.0 license, the weights are available on Hugging Face and Kaggle and can be used commercially without restriction.

The previous generation EmbeddingGemma surpassed 20 million downloads. This upgrade expands capabilities from pure text to multimodal, targeting on-device scenarios — local retrieval tasks where data never leaves the device and everything runs fully offline.

Modular Design: Load Only What You Need

The 740M total parameters aren't a monolithic block — they're divided into four parts: a text component at 270M (130M backbone + 140M embedding layer), a vision encoder at 170M, and an audio encoder at 300M. Crucially, these three encoder types can be loaded on demand.

Text-only loads 270M; text plus image loads 440M; text plus audio loads 570M; loading everything together reaches 740M. Encoders you don't need simply aren't loaded, saving memory. This modular approach is extremely friendly for on-device deployment — the vast majority of applications don't need all five modalities simultaneously.

Architecturally, it's based on Gemma 3 (decoder architecture) and shares the text tokenizer and audio encoder with Gemma 3. The context window reaches 8K tokens — four times the previous generation — and the output vector dimension is 768.

Benchmark Performance: Code Retrieval Shows the Biggest Gains

Across benchmarks, EmbeddingGemma 2's improvements are uneven:

• Multilingual text MTEB: 61.36, up from 61.15 in the previous generation — essentially flat;
• Code retrieval MTEB Code: jumped from 68.68 to 70%, an improvement of roughly 1.9 points, the most notable gain. In the official charts, this score surpasses the 0.6B Qwen3 Embedding and sits within 2 points of 8B-class models;
• Image MIEB: 64.64, more than 4 points above Jina's small models;
• Audio MAEB: 49.39 — relatively weaker, trailing dedicated 3B–7B models by 4 to 6 points, but the highest in its class for models under 1B.

The official positioning is clear: it's the strongest multimodal embedding model under 1B parameters, and can outperform some dedicated models with more than twice its parameter count.

How Much Content Fits in an 8K Window

The model card specifies fixed token costs per modality: one image costs 280 tokens, one video frame costs 140, and one second of audio costs 25. That works out to a maximum of 5.5 minutes of audio, 29 images, or 58 video frames per window when used alone.

Note that all modalities share the same window — mixing them means they compete for space.

One interesting design feature is interleaved input: you can insert placeholders in text to mark the positions of images, videos, or audio, and the result is still encoded as a single vector. The official example is a product detail page where images and video are interspersed within the text — the entire page is represented as one vector, allowing a single text query to search across the full page.

Storage Optimization and Two Critical Pitfalls

On the storage side, the model uses Matryoshka Representation Learning: the 768-dimensional output can be truncated to 512, 256, or 128 dimensions, reducing vector store footprint by up to a factor of six. Quality-wise, dropping to 256 dimensions causes negligible loss; 128 dimensions loses significantly more, and the official guidance is to use that only for pure text scenarios.

The model card explicitly calls out two easy-to-hit pitfalls:

First, you must re-apply L2 normalization after truncation. Simply slicing the vector means it's no longer unit-length, and similarity rankings will silently go wrong — no error thrown.

Second, use BF16 or FP32 for inference — never FP16. The model's activation values exceed FP16's dynamic range, resulting in NaN outputs or significant quality degradation, again with no error raised.

What both pitfalls have in common is "silent failure" — wrong results, no error. This is especially worth watching out for in real development.

Matryoshka Representation Learning (MRL) takes its name from Russian nesting dolls. The core idea is to simultaneously optimize multiple prefix subsets of the vector during training, so that the first k dimensions themselves form a high-quality low-dimensional embedding. This differs from post-hoc PCA dimensionality reduction — PCA requires a separate principal component analysis step, whereas MRL vectors natively support truncation. In practice, choose the appropriate dimension based on storage and retrieval latency constraints: 768 dimensions offers the highest accuracy, 256 dimensions is negligible in quality difference for most scenarios, and 128 dimensions loses enough information that it's only recommended for text use cases. Vector store storage scales linearly with dimension — going from 768 to 128 theoretically yields a 6× reduction, meaningfully helping with Flash storage and RAM on edge devices.

On-Device Deployment: Sharing a Pipeline with Gemma 3

On a Pixel 11 Pro, after quantization, the text-only model occupies approximately 191MB, while the full multimodal configuration uses about 567MB. Because it shares a tokenizer and audio encoder with Gemma 3, both models can run in a single pipeline, with lower total memory than deploying them separately.

The canonical use case is local RAG: EmbeddingGemma 2 handles retrieval, Gemma 3 handles generation, the entire process runs offline, and data never leaves the device. Google included two demos in AI Edge Gallery — one that finds semantically similar content in your own media library using a text query or photo, and another that locates specific segments within a video using text or a voice recording.

RAG (Retrieval-Augmented Generation) is an inference paradigm that combines vector retrieval with language model generation: an embedding model encodes a document collection into a vector index offline; at query time, the question is encoded as a vector and used in an approximate nearest-neighbor search; the retrieved relevant passages are concatenated into the context and passed to a generative model to answer. The benefit of this architecture is that the generative model can access private knowledge bases without retraining, and retrieval results are traceable. In an offline on-device setting, EmbeddingGemma 2 handles indexing and retrieval while Gemma 3 handles generation. Sharing a tokenizer and audio encoder means those weights only need to be loaded once — combined deployment uses less memory than loading them independently, which is especially important on mobile devices that typically have only 8–12GB of RAM.

Developer Ecosystem

The tooling ecosystem is already fairly complete: MediaPipe includes ready-made embedding and retrieval tasks; in-browser use is supported via Transformers.js with WebGPU; local deployment supports Ollama, llama.cpp, MLX, and LM Studio; fine-tuning guidance is available through Unsloth's tutorials.

Weights are on Hugging Face and Kaggle under the Apache 2.0 license — no barriers to commercial use. For developers looking to build privacy-first multimodal retrieval on-device, this is a new option well worth exploring.