feat: add multimodal search and granular performance diagnostics

docs/SEMANTIC.md

@@ -38,9 +38,7 @@ on other operating systems remain independent release checks.
 | embeddinggemma-2-Q8_0.gguf | 309,855,456 | 2188ac1deca4b77dffefd603c2776a9d76d9d74ec01841392982ebb840b09135 |
 | mmproj-embeddinggemma-2-Q8_0.gguf | 554,821,024 | c4a8a52691ecef40618438928bdf9e68379b854e24166f292592353db0aab64f |
 
-Setup downloads both pinned files. **The current worker implements text**;
-image/audio projection loading and image/audio/video embeddings remain required
-work. Downloaded media weights do not establish functioning multimodal search.
+Setup downloads both pinned files. The native worker implements genuine text, image pixels, and mono 16 kHz audio embeddings through the verified Google projector in the same 768-dimensional query space. The shared service now persists unified document, image, video-frame and soundtrack/audio/music vectors with source locations and metadata. Scoped desktop media previews and an independent retrieval evaluation are implemented; precise localization, broader quality, native playback QA and releases remain gates.
 
 Weights use Apache-2.0; llama.cpp and DigUp use MIT; USearch and NumKong use
 Apache-2.0. Retain notices with distributions. Five DigUp text reference vectors
@@ -64,14 +62,15 @@ HybridSearch is the shared CLI/desktop service:
 HybridSearch::open(db, model_dir, worker_executable, threads)
 HybridSearch::embedding_status(db) // No model/runtime load or network
 service.warmup()                  // Real query inference + ANN cache
-service.index_embeddings()
+service.index_embeddings() // Explicit document-only generation
+service.index_embeddings_with_media(&tools, &MediaLimits::default()) // Unified generation
 service.search(query, limit, &SearchOptions)
 service.search_semantic(query, limit, &SearchOptions) // Diagnostics
 service.worker_info()             // Actual model/runtime/device/threads
 ~~~
 
 Threads are bounded to 1–8; VM tests use one. Document batches contain at most
-eight windows. Request/response budgets are 2/4 MiB, startup has a 120-second
+eight windows. Request/response budgets are 16/4 MiB, startup has a 120-second
 deadline, and each request has a 180-second deadline. Malformed or interrupted
 IPC and invalid embeddings fail explicitly.
 
@@ -177,3 +176,92 @@ The ignored ANN test deliberately requires release mode. Default tests require
 no model/network and cover protocol bounds, invalid vectors, UTF-8 coverage,
 fusion, lifecycle, checksums, pre-top-k filtering, and dirty-path caching.
 Linux/macOS and GPU checks are not inferred from this Windows CPU run.
+
+## Genuine image and audio adapter
+
+`GemmaEmbedder::embed_images(&[&Path])` uses a provisional 280-token image
+budget. `embed_images_with_budget(paths, budget)` supports 70 through 1120
+Google visual tokens, including full-resolution model input. Use
+`embed_images_with_metadata` for encoded dimensions and the chosen budget.
+The aspect-preserving dimensions are multiples of 48; Rust performs bounded
+PNG/JPEG/WebP/BMP decoding, applies EXIF orientation, and bicubic Catmull-Rom
+resizing before sending RGB pixels. No captions or filenames produce vectors.
+Bounds are eight paths per call, one image per inference operation, 64 MiB
+input files, 64 Mi-pixel JPEG sources (scaled decoding above 16 Mi pixels),
+16 Mi-pixel non-JPEG sources, and a 256 MiB decoding-buffer/output ceiling. Decoder dimensions
+and output size are checked before pixel allocation. Tiny/extreme images still
+receive a valid bounded grid. The same API embeds decoded video frame files.
+
+`GemmaEmbedder::embed_audio_pcm(samples)` accepts normalized finite mono 16 kHz
+PCM with 1–480,000 samples, at most 30 seconds, on a little-endian host. It
+encodes actual sound through Google MTMD and the shared mean-pooled text model.
+Callers must segment the entire recording/movie soundtrack and retain actual
+window timestamps; this API never silently truncates audio. It provides no
+transcript. Speech, music, and environmental sounds require independent quality
+evaluation even though all use this genuine encoder.
+
+The projector is size/SHA256 verified on first media use, then loaded lazily in
+the retained native worker. Model initialization and returned inference failures
+share the CPU fallback policy. Native video/FFmpeg spawning is disabled; the
+media adapter owns extraction. Media generations must include
+`GEMMA_MEDIA_FINGERPRINT` and image budget/window policy in their identity.
+Text fingerprints remain unchanged. GPU media inference is untested here.
+
+Actual Windows CPU parity with copied MIT DigUp fixtures passed for screenshot
+280, photo 140, frame 70, and speech: cosine respectively **0.99994194,
+0.99974583, 0.99921928, and 0.99983318**, all exceeding the 0.999 gate. The test
+completed in 58.79 seconds including fresh model/projector startup. This proves
+runtime parity on those inputs, not broad image/audio retrieval quality.
+Run with the model/worker environment above:
+
+~~~powershell
+cargo test -p searchi-semantic --test gemma_media_cpu -- --ignored --nocapture
+~~~
+The large camera JPEG policy, established decoder tradeoffs, EXIF/resource test,
+and memory-bound caveats are documented in [the semantic crate](../crates/searchi-semantic/README.md).
+
+## Unified media indexing and setup
+
+The desktop and default CLI `embed` command use `index_embeddings_with_media`.
+Document-only indexing is an explicit `--text-only` choice. `searchi-media`
+prepares actual image files, movie frames and mono 16 kHz audio windows with
+verified caller-supplied FFmpeg/FFprobe executables. There is no transcription,
+fabricated caption or model-derived filename substitution. Existing stream/file
+artist/title metadata enters guarded keyword indexing.
+
+Default media policy uses image 280/frame 70 visual tokens, 30-second primary
+audio windows with two-second overlap, up to 240 distributed frames plus 120
+scene candidates, up to 600 audio windows, a four-hour/16 GiB file ceiling and
+45-second subprocess deadlines. Coverage includes the audio tail or reports an
+explicit resource error; video sampling spans the movie without guaranteeing
+every transient scene. Media files have stable content identity, persisted
+vectors and sources (`image`, `timestamp_ms`); source changes/deletions suppress
+stale results. Generations include preprocessing/window/sampling fingerprints.
+Per-file decoding failures are reported and persisted rather than silently
+counted as successful media indexing. Other filename/text matches stay useful.
+
+Developer preparation from the repository root:
+
+~~~powershell
+python tools/embeddinggemma2/build_runtime.py --backend cpu --jobs 2
+python tools/embeddinggemma2/stage_runtime.py --backend cpu
+python tools/media-runtime/provision.py --stage
+$env:SEARCHI_FORCE_CPU = '1'
+$env:WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS = '--disable-gpu --disable-accelerated-video-decode'
+~~~
+
+The decoder script verifies a pinned FFmpeg 9.0.2 LGPL artifact and stages tools,
+notices and the manifest without global installation. Only Windows x86-64 has
+been executed; Linux/ARM64 entries require native checks, and macOS decoder
+packaging is still missing. End-user packages must include the worker/decoders;
+Google weights are provisioned automatically and verified locally. Search does
+not perform setup downloads. Headless CLI helper discovery and explicit paths
+are documented in [README](../README.md) and [BUILDING](BUILDING.md).
+
+[Independent media evaluation](MEDIA-QUALITY.md) preserves the frozen corpus,
+actual production pipeline, direct-harness failure, music confusions, speech
+point-localization failures and unsuppressed no-answer results. Strong file
+recall is insufficient for accurate spoken-topic navigation. A ten-second audio
+experiment reduces some distances but does not close that gate; it is not a
+new default. Quality policies must be validated on held-out data rather than
+inventing thresholds from the evaluation corpus.