Freeze independently reviewed relevance calibration corpus

fixtures/relevance-v2/README.md

@@ -0,0 +1,43 @@
+# Text and code relevance diagnostic
+
+This frozen fixture contains 14 actual files and 48 English/German requests. It is a
+small, agent-authored diagnostic, not a human-labelled production quality gate.
+No inference has been performed against these labels.
+
+`inventory.json` records exact source bytes, upstream immutable commits and
+complete licence files. Twelve documents describe fictional organisations and
+procedures; no customer information is used. Two complete Rust files come from
+unrelated upstream projects. Local `.gitattributes` prevents Git newline
+conversion of source and licence files.
+
+Calibration uses bicycle warranty, bakery labels, apartment maintenance and
+aho-corasick. Held-out uses museum loans, irrigation, ferry operations and
+semver. Topic families and upstream repositories are disjoint. Each bilingual
+intent pair stays within one split. Positive requests include contrasting
+nearby procedures; absent requests require a specific missing artifact or
+implementation. A related topic alone cannot fulfil such a request.
+
+The independent agent reviewer should read only `blind-review-input.json` and
+its referenced source files, then return candidate file IDs, an empty set for
+an absent artifact, or an explicit ambiguity. Do not read `author-labels.json`
+until that review is saved. No vector, search ranking or threshold is available
+to either annotator. Disagreements must be resolved with written reasons before
+a final immutable label lock is created.
+
+The independent AI review is now saved and agreed with all expected file sets.
+`fixture.json` combines the reviewed queries and labels; `fixture.lock.json`
+records their hashes and the UTC freeze time before any inference. Review history
+preserves encoding repairs and wording clarifications before that freeze.
+**Do not evaluate held-out queries until a policy or threshold has been fixed
+using calibration only.** Do not adjust that policy from held-out results.
+
+No media is included: this corpus cannot calibrate music, images, video or
+spoken-passage abstention. Existing media benchmarks remain separate. A held-out
+split made by the same author is weaker than independent human annotation and
+larger real customer libraries.
+
+Run `python fixtures/relevance-v2/verify.py` for source integrity and split
+structure. That check proves consistency, not retrieval quality.
+Run `python fixtures/relevance-v2/test_verify.py` for isolated corruption,
+duplicate, split-leak and lock regressions, including verifier execution with
+Python optimization enabled.