D Fuchs

Chat with video using EmbeddingGemma 2 (100% local): EmbeddingGemma 2 by @GoogleDeepMind maps text, images, video, and audio into a shared vector space. This makes it possible to search video frames with natural language. Devs have already built native Rust implementations for local inference. Unsloth also provides a 176 MB, 4-bit GGUF checkpoint for text embeddings, although that version doesn't include the full multimodal pipeline. I tested the video-search use case by building a chat interface that finds matching frames and jumps to their timestamps. On my Apple M3 Pro, using the CPU: - Indexing took 29s for a 60s video - Search took 25 ms on average across 9 queries - The saved frame vectors occupied 180 KiB. The video is indexed once. Each new search embeds the query and compares it with the saved frame vectors. Processing stays entirely on the laptop. The video below depicts the entire flow in action. That said, model quantization reduces the memory needed for the weights. The retrieval index has its own storage cost, which grows as you add videos. To dive deeper into reducing that cost, I wrote a full article on binary quantization. It explains how storing vectors with one bit per dimension reduces their raw storage by 32x compared with float32, and includes a working retrieval implementation. Read it below.