# Mixed retrieval quality gate

A frozen mixed diagnostic is prepared in [fixtures/mixed](../fixtures/mixed/README.md).
It addresses the missing comparison between natural-language All-files queries
and explicit Code scope while images, music, speech, videos and actual documents
compete. It contains 32 real files and 91 English/German queries, including two
pinned upstream llama.cpp implementations. The existing media labels are reused
with their licenses and attribution. They are not an untouched held-out set.

**No actual mixed inference results are recorded yet.** The frozen manifest
SHA-256 is `f83d27227582f614159ff153654d5bb5a47bf74998141e389557f3950ea0b924`.
Asset/hash verification and six meaningful scoring/provenance tests passed.
These prove harness behavior only; they establish no retrieval accuracy.

The [evaluator](../tools/mixed-eval/README.md) calls the shared production engine
through CLI indexing and MCP search. It compares lexical/hybrid retrieval and
All-files/Code scope without changing ranking or tuning a cutoff against these
labels. File-level recall and no-answer returned-result rates remain separate
from spoken timestamp localization. Raw evidence must preserve actual failures,
encoding errors and unsupported filters.

Remaining work:

- Run the frozen corpus with the verified Google model and CPU runtime in a
  coordinated slot; record exact CLI/worker hashes and actual raw results.
- Inspect failures by modality, language, requested artifact type and scope;
  distinguish correct-file retrieval from a useful passage/timestamp.
- Author a larger independently reviewed held-out corpus before ranking changes;
  frozen diagnostics must not become the sole optimization target.
- Establish recall and no-answer acceptance targets on representative user
  libraries, unrelated repositories and longer recordings. Tiny diagnostics
  cannot close the production relevance gate.