Files
searchi/fixtures/relevance-v2

Text and code relevance diagnostic

This frozen fixture contains 14 actual files and 48 English/German requests. It is a small, agent-authored diagnostic, not a human-labelled production quality gate. No inference has been performed against these labels.

inventory.json records exact source bytes, upstream immutable commits and complete licence files. Twelve documents describe fictional organisations and procedures; no customer information is used. Two complete Rust files come from unrelated upstream projects. Local .gitattributes prevents Git newline conversion of source and licence files.

Calibration uses bicycle warranty, bakery labels, apartment maintenance and aho-corasick. Held-out uses museum loans, irrigation, ferry operations and semver. Topic families and upstream repositories are disjoint. Each bilingual intent pair stays within one split. Positive requests include contrasting nearby procedures; absent requests require a specific missing artifact or implementation. A related topic alone cannot fulfil such a request.

The independent agent reviewer should read only blind-review-input.json and its referenced source files, then return candidate file IDs, an empty set for an absent artifact, or an explicit ambiguity. Do not read author-labels.json until that review is saved. No vector, search ranking or threshold is available to either annotator. Disagreements must be resolved with written reasons before a final immutable label lock is created.

The independent AI review is now saved and agreed with all expected file sets. fixture.json combines the reviewed queries and labels; fixture.lock.json records their hashes and the UTC freeze time before any inference. Review history preserves encoding repairs and wording clarifications before that freeze. Do not evaluate held-out queries until a policy or threshold has been fixed using calibration only. Do not adjust that policy from held-out results.

No media is included: this corpus cannot calibrate music, images, video or spoken-passage abstention. Existing media benchmarks remain separate. A held-out split made by the same author is weaker than independent human annotation and larger real customer libraries.

Run python fixtures/relevance-v2/verify.py for source integrity and split structure. That check proves consistency, not retrieval quality. Run python fixtures/relevance-v2/test_verify.py for isolated corruption, duplicate, split-leak and lock regressions, including verifier execution with Python optimization enabled.