feat: add multimodal search and granular performance diagnostics
Core CI / core (ubuntu-latest) (push) Failing after 3m31s
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
Core CI / desktop (ubuntu-latest) (push) Successful in 5m39s
Core CI / desktop (macos-latest) (push) Canceled after 0s
Core CI / desktop (windows-latest) (push) Canceled after 0s

This commit is contained in:
karl committed 2026-10-12 19:16:23 +02:00
1 parent d3bb8c7618
commit b8400c4fbf
110 files changed
+79395 -250

No files matched your search

+43
View File
@@ -0,0 +1,43 @@
# Shared hybrid retrieval
`HybridSearch` combines genuine lexical and Google multimodal retrieval. The
CLI and desktop use the same service; it has no desktop-shell dependency.
`search`, `search_semantic` and `search_semantic_with_origin` preserve grouped
parent files. ANN candidates expand until enough unique eligible parents exist
or the complete bounded eligible chunk set is requested. Filename/type/folder,
size/date/duration and explicit source filters apply before ANN top-k. Media
candidate filesystem metadata is checked once per parent per query. Ninety-six
near-identical video vectors cannot hide a second eligible parent (regression
tested). HNSW remains approximate retrieval, not an exact recall guarantee.
Explicit spoken/audio wording prefers actual audio-source vectors, preserving
exact lexical filename priority. Conservative English/German token cues detect
requests such as “someone mentions”, “spoken”, “audio”, “jemand erwähnt” and
“Gespräch”. Visual/document cues veto automatic routing. This finite parser is
not language understanding; paraphrases, negation and mixed intents can remain
ambiguous. If the preferred lane has no eligible results, ordinary retrieval
is used. `SemanticOrigin::All` bypasses the hint; `Audio`, `Image`, `Video` and
`Text` select an explicit channel without fallback or reembedding.
Persisted custom sampling limits can be reopened without generating embeddings
again only when the complete model and preprocessing fingerprint matches.
The custom suffix must contain the eight valid numeric sampling limits. Model
or preprocessing changes never silently reuse incompatible vectors.
`search_measured(query,limit,options,debug)` returns `SearchResponse` with results
and monotonic total milliseconds/result count. Basic mode stores no trace.
`search_traced(...,&mut TraceCollector,parent)` shares the native request clock
so queue/lock time and actual engine stages remain comparable. Debug mode adds
bounded nested spans for lexical SQL, query embedding, worker diagnostics,
filtered eligibility and its cache, ANN read/checksum/deserialization/search
passes, parent materialization and validation, and rank fusion. Static stage
names and numerical counters exclude query/source text and paths. Worker-local
microsecond counters have a separate clock and are not aligned request spans.
Measured/source APIs retain existing result behavior except the explicit
spoken/audio preference. Debug instrumentation does not change vector policy.
Audio timestamps represent segment starts, not exact word locations. Smaller
windows alone have not demonstrated accurate spoken navigation. No-answer
calibration, music discrimination, exact spoken positions, process isolation,
unchanged-vector reuse and production platform checks remain release gates.