feat: add multimodal search and granular performance diagnostics

apps/desktop/README.md

@@ -1,11 +1,24 @@
 # Searchi desktop
 
-This Tauri 2 shell calls the same Rust core as the CLI and the real Google EmbeddingGemma 2 service. It provides folder indexing, native folder selection, progressive filename/content/semantic search, folder/extension/size/date filters, keyboard navigation, escaped text previews, native open/reveal actions, persisted selected folders/resource settings, and a global shortcut. There are no mocked results or embeddings. Watchers and release installers remain roadmap work.
+This Tauri 2 shell calls the same Rust core as the CLI and the real Google EmbeddingGemma 2 service. It provides folder indexing, native folder selection, progressive filename/content/semantic search, folder/extension/size/date/media-duration filters, keyboard navigation, escaped text and scoped image/audio/video previews, native open/reveal actions, persisted selected folders/resource settings, and a global shortcut. There are no mocked results or embeddings. Watchers and release installers remain roadmap work.
 
 ## Develop
 
-Install Node.js LTS, Rust stable, and the platform prerequisites documented at https://v2.tauri.app/start/prerequisites/. Windows requires MSVC build tools and WebView2; Linux requires WebKitGTK and system development packages; macOS requires Xcode command line tools.
+Install Node.js 24 LTS, Rust stable, and the platform prerequisites documented at https://v2.tauri.app/start/prerequisites/. Windows requires MSVC build tools and WebView2; Linux requires WebKitGTK and system development packages; macOS requires Xcode command line tools.
 
+Before launching from a checkout, build and stage both native helpers from the
+repository root. These explicit setup commands access the network and verify
+pinned artifacts; they do not install tools globally:
+
+```sh
+python tools/embeddinggemma2/build_runtime.py --backend cpu --jobs 2
+python tools/embeddinggemma2/stage_runtime.py --backend cpu
+python tools/media-runtime/provision.py --stage
+```
+
+The decoder manifest currently has Windows/Linux x86-64 and ARM64 artifacts;
+a vetted macOS decoder package remains a release gate. Build `cpu` in the
+Windows VM. Production acceleration requires separate platform/hardware QA.
 ```sh
 cd apps/desktop
 npm ci
@@ -25,7 +38,7 @@ The desktop index is stored in the platform app-data directory under `dev.search
 
 On first launch, the native app downloads and verifies the pinned Google model artifacts (310 MB text model plus 555 MB media projector, approximately 865 MB total) into app-managed storage and prepares the packaged embedding worker. No model paths, runtime paths, or terminal setup are required. The UI shows native download bytes, verification/warming states, the actual worker device, and explicit errors with a retry control. The first download requires network access; file extraction, embeddings, and queries stay local.
 
-After a folder is indexed, semantic indexing starts automatically when the model is ready. Existing keyword results stay searchable during setup/indexing. Progress is indeterminate during embedding because the native command does not expose a chunk completion count. Failed automatic builds retry only after a changed index/status revision or an explicit rebuild, avoiding repeated failure loops. Status polls run once per second while setup/indexing/warming is active and stop when idle.
+After a folder is indexed, unified document/media semantic indexing starts automatically when the model and packaged media helpers are ready. Images use actual pixels; video frames and full primary-stream audio windows use the Google media projector. Audio/music files also retain artist/title metadata for keyword retrieval. The UI reports unavailable decoder/runtime support and per-file failures while keeping text search available. Existing keyword results stay searchable during setup/indexing. Progress is indeterminate during embedding because the native command does not expose a chunk completion count. Failed automatic builds retry only after a changed index/status revision or an explicit rebuild, avoiding repeated failure loops. Status polls run once per second while setup/indexing/warming is active and stop when idle.
 
 Each query captures one immutable filter snapshot. Keyword results appear first; real hybrid results follow after the query settles. `src/search-state.ts` bounds each lane to one active native request and one latest pending request, discards old revisions, and retries a busy request at most once. Escape clears both pending lanes and their timers; already-running native work finishes without changing cleared results. Hybrid updates preserve the selected file and keyboard focus where possible.
 
@@ -38,7 +51,7 @@ node --test tests/search-state.test.mjs
 For Windows VM UI testing, disable WebView2 GPU rendering as well, before starting Tauri in PowerShell:
 
 ```powershell
-$env:WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS = '--disable-gpu'
+$env:WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS = '--disable-gpu --disable-accelerated-video-decode'
 $env:SEARCHI_FORCE_CPU = '1'
 npm.cmd run tauri dev
 ```
@@ -56,3 +69,38 @@ Visual fidelity remains subject to native screenshot QA. Compilation and state t
 ## Verification and packaging
 
 `npm run build` checks strict TypeScript and creates production frontend assets. `npm run tauri build -- --no-bundle` compiles the native shell when platform prerequisites are installed. Installer bundling is intentionally disabled until release icons, signing and native platform checks are complete. Native Windows, macOS, and Linux checks must be recorded separately; a frontend build does not prove the native shell works.
+
+## Media preview and quality
+
+The result pane renders images and audio/video controls through Tauri's scoped
+asset protocol. Native preparation verifies the selected path belongs to a
+current indexed file before exposing it. There is no general filesystem access
+or local HTTP server. Players use metadata preload, never autoplay, and seek
+matched millisecond timestamps after metadata loads. Selection changes and
+Escape pause/unload playback; stale preparation responses cannot replace the
+current preview. Unsupported WebView codecs show an Open file fallback.
+
+Opening the default application does not guarantee exact-location navigation.
+Preview seeking uses the returned window/frame start, which may differ from the
+actual utterance. The [independent media quality report](../../docs/MEDIA-QUALITY.md)
+records real file recall, speech alignment failures, music hard negatives and
+unmet no-answer handling. Genuine multimodal support is implemented; production
+quality and native playback/visual interaction QA remain open gates.
+
+## Performance details
+
+Normal results show the actual native search duration and latest result-display
+latency from input. The **Performance details** checkbox in Settings is opt-in
+and captured per request. Its expandable panel separates input debounce,
+intentional semantic delay, pending-lane queue, failed attempts and bounded retry
+wait, invocation time, native stage hierarchy and synchronous result DOM updates.
+First response and hybrid-final latency are distinct; keyword-only results do not
+claim semantic completion. Native spans use a command-relative clock and nested
+or parallel spans must not be summed. Invocation minus native total is explicitly
+an estimated transport/serialization/scheduling residual, not an exact IPC cost.
+DOM update timings do not claim actual screen paint or deferred media loading.
+
+The local clipboard report excludes queries, paths, filenames, snippets and error
+strings. It captures available model/device/thread/runtime context without
+guessing missing values. Histories are bounded and dropped counts remain visible.
+Clipboard failure is explicit; no diagnostic telemetry is uploaded.