Freeze mixed-file retrieval diagnostic and shared-engine evaluator

docs/MIXED-QUALITY.md

@@ -0,0 +1,32 @@
+# Mixed retrieval quality gate
+
+A frozen mixed diagnostic is prepared in [fixtures/mixed](../fixtures/mixed/README.md).
+It addresses the missing comparison between natural-language All-files queries
+and explicit Code scope while images, music, speech, videos and actual documents
+compete. It contains 32 real files and 91 English/German queries, including two
+pinned upstream llama.cpp implementations. The existing media labels are reused
+with their licenses and attribution. They are not an untouched held-out set.
+
+**No actual mixed inference results are recorded yet.** The frozen manifest
+SHA-256 is `f83d27227582f614159ff153654d5bb5a47bf74998141e389557f3950ea0b924`.
+Asset/hash verification and six meaningful scoring/provenance tests passed.
+These prove harness behavior only; they establish no retrieval accuracy.
+
+The [evaluator](../tools/mixed-eval/README.md) calls the shared production engine
+through CLI indexing and MCP search. It compares lexical/hybrid retrieval and
+All-files/Code scope without changing ranking or tuning a cutoff against these
+labels. File-level recall and no-answer returned-result rates remain separate
+from spoken timestamp localization. Raw evidence must preserve actual failures,
+encoding errors and unsupported filters.
+
+Remaining work:
+
+- Run the frozen corpus with the verified Google model and CPU runtime in a
+  coordinated slot; record exact CLI/worker hashes and actual raw results.
+- Inspect failures by modality, language, requested artifact type and scope;
+  distinguish correct-file retrieval from a useful passage/timestamp.
+- Author a larger independently reviewed held-out corpus before ranking changes;
+  frozen diagnostics must not become the sole optimization target.
+- Establish recall and no-answer acceptance targets on representative user
+  libraries, unrelated repositories and longer recordings. Tiny diagnostics
+  cannot close the production relevance gate.