# Mixed retrieval quality checkpoint The frozen [mixed diagnostic](../fixtures/mixed/README.md) was executed on 2026-10-13 with the genuine Google EmbeddingGemma 2 Q8_0 model, the normal CPU/Vulkan worker forced to CPU, and the shared CLI/MCP engine. Windows 11 VM; one inference thread. The CLI was a debug build. The corpus contains 32 actual files: eight implementations (six searchi files and two pinned llama.cpp files), four technical documents, and twenty licensed media files. It is a small single-annotator diagnostic, with reused previously evaluated media labels; **these results do not establish production quality or untouched held-out quality**. ## Completed execution The evaluator finished successfully with 210 rows: 104 executed requests per mode plus one unsupported temporal-filter query per mode. All 32 files were indexed; document extraction reported zero failures. Semantic indexing took **1,060,840 ms (17 minutes 40.84 seconds)** and published generation one with **182 chunks: 64 text and 118 media**, derived from 62 document passages and 20 media files. The final report records zero media failures and an empty failure list. It made 62 document, 99 image/frame and 19 audio inference calls; 82 source tasks were encoded and none reused. Image/frame calls plus audio calls account for the 118 media chunks. Indexing costs include decoder work and model initialization; they are not pure native inference measurements. | Positive queries | Hybrid All files top1 | Hybrid All files recall@5 | | --- | --- | --- | | Implementation intent | 10/14 | 13/14 | | Technical documents | 7/8 | 8/8 | | Images | 24/24 | 24/24 | | Videos, file identity only | 22/22 | 22/22 | | Music | 2/4 | 4/4 | | Sounds | 4/4 | 4/4 | | Exact media metadata/filter | 4/4 | 4/4 | | **All positive queries** | **73/80** | **79/80** | Lexical-only All files found the expected file first for **3/80** positives, with **3/80** recall@5. In explicit Code scope, hybrid found **14/14** code queries first; lexical-only found **0/14**. This covers two upstream files and six self-repository implementations, not fourteen independent projects. All **10/10 no-answer queries returned results** in hybrid All files; lexical-only returned no results for those ten. This exposes uncalibrated irrelevant neighbors, rather than proving an answer exists. Do not interpret the displayed hybrid scores as confidence or tune a rejection threshold on these same frozen labels. One query's requested temporal-range filter remains unsupported and was explicitly skipped in each mode. ## Concrete failures - English `code that chooses audio search when a query mentions speech` returned audio matches and omitted the expected routing implementation entirely from twenty results. Explicit Code scope placed that implementation first. The German paired request found it first in both scopes. This confirms a real All-files audio-intent suppression failure; the separate routing audit explains the current rule. No production ranking change was made for this run. - Both CPU-budget implementation queries ranked the performance documentation above the implementation (rank two). The English rendering/queue timing implementation query also placed the documentation first. Code scope resolved these small cases, but requiring users to switch scope does not establish the requested All-files experience. - The German media-change limitations query ranked incremental-indexing documentation above the intended automatic-indexing document (rank two). Its desired source remained present in the first five. - The English and German jazzy-violin queries ranked piano music first and the correct violin music second. The frozen music hard negative remains a genuine top-result failure. Video/speech rows above measure **correct file identity**, not a useful scene or spoken passage. They do not overturn the poor strict speech timestamp results in [MEDIA-QUALITY.md](MEDIA-QUALITY.md). Returned source timestamps are retained in the raw rows for further inspection. A correctly retrieved video can still open at the wrong moment. ## Timing boundaries and evidence Across 90 executed All-files requests in their fixed fixture order, observed hybrid MCP roundtrip median/p95 were **101.98/136.38 ms**; native basic search median/p95 were **94.22/125.90 ms**. Lexical roundtrip median/p95 were **2.33/3.24 ms**, with native **1.31/2.14 ms**. The first hybrid request took 117.73 ms roundtrip and 107.04 ms natively. These are ordered tiny-corpus observations, excluding process initialization/model loading; they are neither first user-input timing nor randomized production-scale warm p95. Code-scope requests alternate with All-files code queries, so task/filter caches and prior requests are part of this run's conditions. The complete raw report remains in ignored `.searchi/qa/mixed-v1.json` (636,101 bytes), SHA-256 `c6619d4a66332f0378d42b9ec4f108349ce095aeb2dbcccd535de6095438b37a`. It retains every ordered hit, query, expected label, source location, native metric, index stdout/stderr and final semantic report. Runtime stderr retained Google token-type warnings and the native runtime's experimental audio warning; these were not hidden as failed assets or omitted from evidence. | Provenance | SHA-256 | | --- | --- | | Frozen labels | `f83d27227582f614159ff153654d5bb5a47bf74998141e389557f3950ea0b924` | | Actual debug CLI | `23afd9986d987bc45db9e08aa50f1070f3ad2229d275793c1f2a41ee0fcea545` | | Actual normal worker | `c01cb3ec13cfca722ee768e8350e1399a8b9a4d14a24a35adc3473a5cfc82a73` | | Pinned Google model manifest | `fad82f171f4976d8eeb89c11d6a79fc46d92afb73c10a34f140d5b56e377397f` | The worker/model pair was SHA-verified before inference. The active fingerprint records query context 512 and document context 2048. The owned evaluator session finished with exit zero and cleaned its temporary corpus/index and worker; no restart or substituted model was used. Reproduce with the [evaluator instructions](../tools/mixed-eval/README.md). ## Open quality gates - Validate the subsequent explicit-code routing fix on broader mixed queries without hiding legitimate media results. A tiny real-model candidate-preservation regression passed; see [routing evidence](RETRIEVAL-ROUTING.md). The frozen baseline recorded here remains the pre-fix measurement. - Evaluate general/code query task fusion on a new independently labelled set, with bounded retrieval and measured cache/filter costs. - Improve music discrimination and precise speech timestamps. - Establish no-answer behavior on a separate calibration/held-out split while preserving positive recall. - Add unrelated repositories, customer documents, longer recordings, human listening and a second blinded annotator before broad relevance claims.