These 36 queries were frozen before their model results: 16 answerable, 16 close-but-absent, and four broad/partial requests. Each intent has an English and German version. The fixture SHA-256 is in fixture.lock.json.

This is a new query holdout over an existing source corpus, not a new corpus holdout. No historical absence query or model output was read to author these queries. The author inspected document/code bytes and the original independent media descriptions. One AI annotator supplied the labels; independent human review has not happened. In particular, audio/music labels rely on the original publisher descriptions and tags, not fresh human listening. Report these categories separately rather than implying universal ground truth.

The 36 declared inputs are all 14 relevance-v2 source files, all 20 licensed media files, and two owned speech PCM fixtures containing Article 1 of the Universal Declaration of Human Rights. See each original source manifest and README for license, attribution and derivation. Existing licenses remain in force. Speech WAVs remain ignored local test assets; their exact hashes are frozen here, without adding another binary copy to Git.

Before inference:

python fixtures/absence-v3/verify.py

Index a dedicated copy of only the declared bytes, preserving basenames. Document/code/photo/audio/video extraction failures and missing semantic channels must be reported independently of retrieval. An empty result after failure is not successful abstention. Record asset coverage before interpreting the strict absent metric. The four partial requests are not strict absence tests; keeping relevant partial evidence is acceptable.

The old relevance runner deliberately accepts only the frozen v2 fixture. This fixture requires a separate runner or explicit schema adaptation. Do not silently bypass its frozen-policy gate or overwrite historical reports. build_fixture.py records authorship and refuses to overwrite frozen labels. Do not rerun it to revise labels after scores become available.

Frozen-policy CPU results

evaluation-summary.json records the completed report's hash, runtime/source hashes, coverage, timings and per-language/category scores. All 36 assets had active semantic channels, with no extraction, semantic or speech failures. The 36 queries were evaluated without changing either fixture or policy.

Primary neighbors appeared for 4/16 absent requests, down from 16/16 under the previous leading-cluster rule. The remaining errors ask for a customs export declaration when only transit insurance exists, and an HTTP registry fetcher when only a local version parser exists. Those related sources exceed the display threshold despite lacking the requested facts. Similarity is not entailment.

All 16 positive requests retained their expected sources within five accessible results (mean recall@5 1.0). Primary mean recall@5 was 0.625: 11/16 queries retained at least one expected primary source; 10/16 had the correct primary first result. Five positive queries required opening related suggestions. Both performer/composition music requests placed the wrong music file first in raw retrieval, while retaining the correct file within five. This is a real tradeoff, not a claim that all unsupported requests disappear.

Initial full text/media/speech indexing took 894.9 seconds on the CPU VM. Query-role startup took 1.017 seconds; the ordered warm raw query median was 94.4 ms, maximum 122.9 ms. These are fixture timings, not a general desktop performance promise or independent cold-input measurements. The persistent profile retains the real index so unchanged files need not be encoded again.