Files
searchi/crates/searchi-service/README.md
T

49 lines
3.2 KiB
Markdown
Raw Normal View History

# Shared hybrid retrieval
`HybridSearch` combines genuine lexical and Google multimodal retrieval. The
CLI and desktop use the same service; it has no desktop-shell dependency.
`search`, `search_semantic` and `search_semantic_with_origin` preserve grouped
parent files. ANN candidates expand until enough unique eligible parents exist
or the complete bounded eligible chunk set is requested. Filename/type/folder,
size/date/duration and explicit source filters apply before ANN top-k. Media
candidate filesystem metadata is checked once per parent per query. Ninety-six
near-identical video vectors cannot hide a second eligible parent (regression
tested). HNSW remains approximate retrieval, not an exact recall guarantee.
Explicit spoken/audio wording prefers actual audio-source vectors, preserving
exact lexical filename priority. Conservative English/German token cues detect
requests such as “someone mentions”, “spoken”, “audio”, “jemand erwähnt” and
“Gespräch”. Visual/document cues veto automatic routing. This finite parser is
not language understanding; paraphrases, negation and mixed intents can remain
ambiguous. If the preferred lane has no eligible results, ordinary retrieval
is used. `SemanticOrigin::All` bypasses the hint; `Audio`, `Image`, `Video` and
`Text` select an explicit channel without fallback or reembedding.
Persisted custom sampling limits can be reopened without generating embeddings
again only when the complete model and preprocessing fingerprint matches.
The custom suffix must contain the eight valid numeric sampling limits. Model
or preprocessing changes never silently reuse incompatible vectors.
`search_measured(query,limit,options,debug)` returns `SearchResponse` with results
and monotonic total milliseconds/result count. Basic mode stores no trace.
`search_traced(...,&mut TraceCollector,parent)` shares the native request clock
so queue/lock time and actual engine stages remain comparable. Debug mode adds
bounded nested spans for lexical SQL, query embedding, worker diagnostics,
filtered eligibility and its cache, ANN read/checksum/deserialization/search
passes, parent materialization and validation, and rank fusion. Static stage
names and numerical counters exclude query/source text and paths. Worker-local
microsecond counters have a separate clock and are not aligned request spans.
Measured/source APIs retain existing result behavior except the explicit
spoken/audio preference. Debug instrumentation does not change vector policy.
Audio timestamps represent segment starts, not exact word locations. Smaller
windows alone have not demonstrated accurate spoken navigation. No-answer
calibration, music discrimination, exact spoken positions, process isolation,
unchanged-vector reuse and production platform checks remain release gates.
Exact artist/title constraints use the core's normalized structured probe keys
before ANN limits and in the eligibility cache key. Metadata-only writes advance
the corpus revision and dirty path so warmed readers cannot retain old eligibility.
Empty metadata-only hybrid requests use core results without model inference.
These explicit constraints do not establish abstention for free-form topic queries.