2026-10-13 08:26:01 +02:00
|
|
|
# Mixed query routing audit
|
|
|
|
|
|
|
|
|
|
Read-only implementation audit, 2026-10-13, source revision `5d9cfc4`.
|
2026-10-13 17:49:10 +02:00
|
|
|
These opening findings describe that pre-fix snapshot; they are not a retrieval-quality
|
2026-10-13 08:26:01 +02:00
|
|
|
measurement or an implemented ranking change.
|
|
|
|
|
|
2026-10-13 17:49:10 +02:00
|
|
|
## Pre-fix behavior at 5d9cfc4
|
2026-10-13 08:26:01 +02:00
|
|
|
|
|
|
|
|
`HybridSearch::query_vector` selects Google's CodeRetrieval task only when
|
|
|
|
|
`SearchOptions::code_only` is true. All-files searches use the general task,
|
|
|
|
|
including natural-language queries asking for an implementation. The bounded
|
|
|
|
|
query-vector LRU distinguishes tasks, so vectors cannot be reused across them
|
|
|
|
|
accidentally. See `crates/searchi-service/src/lib.rs`.
|
|
|
|
|
|
|
|
|
|
The audio preference in `crates/searchi-service/src/routing.rs` recognizes words
|
|
|
|
|
such as speech, sound and music. Document and image cues veto that preference;
|
2026-10-13 17:49:10 +02:00
|
|
|
code cues did not at this snapshot. If the preferred audio search returns any qualifying
|
2026-10-13 08:26:01 +02:00
|
|
|
candidate, it becomes the exclusive semantic lane. Consequently a query such as
|
|
|
|
|
"code that detects speech" can exclude semantic code candidates in All files.
|
|
|
|
|
Lexical candidates still participate in hybrid fusion, and explicit Code scope
|
|
|
|
|
bypasses this audio preference. This is a risk to evaluate, not proof that every
|
|
|
|
|
such query fails.
|
|
|
|
|
|
|
|
|
|
## Cost of adding lanes
|
|
|
|
|
|
|
|
|
|
`crates/searchi-service/src/store.rs` retains one eligibility-cache entry.
|
|
|
|
|
Its key includes the generation, user filters, source type and code scope.
|
|
|
|
|
Changing a filter replaces the entry and scans/hashes matching passage metadata
|
|
|
|
|
again. Alternating general and code lanes would therefore perform repeated
|
|
|
|
|
eligibility work even when query vectors and the ANN graph are warm. The existing
|
|
|
|
|
empty-audio fallback can also alternate audio and general filters.
|
|
|
|
|
|
|
|
|
|
Incremental dirty-path updates apply when the cached filter matches and the
|
|
|
|
|
revision is still within the dirty-history floor. A replacement cache design
|
|
|
|
|
must preserve those freshness checks, generation invalidation, strict filtering
|
|
|
|
|
before candidate limits, and the total allocation budget.
|
|
|
|
|
|
|
|
|
|
## Next experiment
|
|
|
|
|
|
|
|
|
|
First complete the unchanged frozen mixed baseline described in
|
|
|
|
|
[MIXED-QUALITY.md](MIXED-QUALITY.md). Keep its labels and source bytes unchanged.
|
|
|
|
|
Then compare soft audio preference and task-specific rank fusion against that
|
|
|
|
|
baseline, retaining exact filename/symbol priority. Different task scores must
|
|
|
|
|
not be treated as calibrated probabilities or compared blindly.
|
|
|
|
|
|
|
|
|
|
Before introducing multiple lanes, measure a bounded multi-entry eligibility
|
|
|
|
|
cache against a shared validated snapshot with derived code/origin views.
|
|
|
|
|
Bound memory across all entries, rather than giving each the full chunk budget.
|
|
|
|
|
Reuse the existing worker and prepared ANN graph.
|
|
|
|
|
|
|
|
|
|
Report All-files and Code scope separately: top-1, recall@5, hard negatives and
|
|
|
|
|
no-answer behavior, plus query-vector cache misses, inference time, eligibility
|
|
|
|
|
scans/hash calls, memory and warm p95. Include mutation/deletion regressions.
|
|
|
|
|
The existing `semantic.eligibility` and
|
|
|
|
|
`semantic.eligibility_filtered_scan_and_hash` traces expose eligibility cost.
|
|
|
|
|
Use an independently reviewed held-out corpus for acceptance; the frozen
|
|
|
|
|
diagnostic alone must not become the optimization target.
|
2026-10-13 08:47:41 +02:00
|
|
|
|
|
|
|
|
## Narrow suppression fix and actual regression
|
|
|
|
|
|
|
|
|
|
After the frozen baseline confirmed the English speech-related code failure,
|
|
|
|
|
explicit code/function/implementation cues now veto automatic audio preference,
|
|
|
|
|
alongside the existing document/image cues. English and German cues are covered
|
|
|
|
|
by the routing unit test. This changes candidate eligibility through routing;
|
|
|
|
|
it does not add CodeRetrieval to All files or implement soft multi-lane fusion.
|
|
|
|
|
|
|
|
|
|
The ignored integration `routing_google_cpu` passed on Windows CPU in **38.40 s**
|
|
|
|
|
using the verified normal worker and Google model, unchanged frozen routing
|
|
|
|
|
source bytes and the licensed real roaring-creature sound. The same remembered
|
|
|
|
|
code query produced qualifying audio candidates through the explicit Audio
|
|
|
|
|
diagnostic lane. Updated default semantic routing also returned the source
|
|
|
|
|
implementation, proving the exclusive suppression is removed in this fixture.
|
|
|
|
|
The test asserts candidate preservation, not top-1 superiority or broad quality.
|
|
|
|
|
The full mixed baseline above remains the unchanged pre-fix measurement.
|
2026-10-13 17:49:10 +02:00
|
|
|
The completed [0.1.4 full regression](MIXED-QUALITY-0.1.4.md) now confirms that
|
|
|
|
|
this known failed query returns the implementation first. Code intents in All
|
|
|
|
|
files improved from 10/14 to 11/14 top-1 and 13/14 to 14/14 recall@5, without
|
|
|
|
|
scored regression elsewhere. Independent relevance and no-answer gates stay open.
|
2026-10-13 08:47:41 +02:00
|
|
|
|
|
|
|
|
With this fix and the verified-asset callback, the Windows CPU workspace suite
|
|
|
|
|
passed 130 tests (23 explicitly ignored integration/resource checks), desktop
|
|
|
|
|
Rust passed 28, and both strict all-target Clippy checks and formatting passed.
|
|
|
|
|
The ignored actual routing regression was executed separately as described above.
|