5.4 KiB
Unseen music metadata stress validation v1
This small source pool and its 18 queries were frozen before any model inference. It is independent of the two music files used to motivate the metadata-field hypothesis. Labels come from publisher descriptions, not model outputs. There is one assistant annotator; human listening and a blinded second annotator have not been executed. These instrument demos and short excerpts do not represent a large music collection or full-song recognition.
| Source | Publisher / attribution | License | Download bytes |
|---|---|---|---|
| Acoustic guitar sample | RyGuy / Ryan | PD-self | 889069 |
| Cello strings | Georg Feitscher | CC BY-SA 3.0 | 296474 |
| Drum cadence A | United States Navy Band | PD-USGov | 1574547 |
| Bucket mute trumpet demo | Ovinus Real | CC BY-SA 4.0 | 228972 |
| After You've Gone, tenor saxophone excerpt | Julien Grandgagnage / Beachcomber | CC BY-SA 3.0 | 300001 |
| Ain't She Sweet, soprano saxophone excerpt | Julien Grandgagnage / Beachcomber | CC BY-SA 3.0 | 250846 |
Total acquisition: 3539909 bytes, approximately 3.38 MiB. License URLs and
publisher attribution are preserved in fixture.json; source SHA-1 comes from
the official Commons imageinfo API, acquired bytes are also SHA-256 locked in
inputs.lock.json. Do not substitute similarly named files. Raw audio and
generated containers stay under ignored .searchi/qa/music-validation-v1.
Preparation and stress variants
Preparation takes the first at most 30 seconds, resamples to mono 16 kHz and encodes lossless FLAC. It produces actual containers with original whitelisted tags, missing tags and cyclically transferred actual tags from the next source. The exact decoded PCM SHA-256 must match across each source's three variants before audio vectors can be reused. These are controlled tag-corruption cases, not naturally mislabeled customer files. Generated excerpts/variants retain the original source's license and attribution, with these modifications disclosed.
The drum source actually has no descriptive tags. The guitar has only artist
Ryan. Cello supplies cello strings; trumpet supplies Bucket mute; the sax
titles identify tenor/soprano. Empty tag input creates no metadata vector. A
cyclic transfer from the tagless drum remains tagless; this fact is retained,
not replaced with an invented instrument label. Comments and encoder strings
are not semantic metadata input.
Two existing photo and two sound-effect files are fixed mixed-media competitors:
image01, image02, sound01, sound02 from the previously frozen media corpus.
They are distractors, not new validation positives. All competitors stay in each
lane. This does not cover the entire 32-file mixed regression or spoken videos.
Twelve instrument queries have English/German pairs. Broad saxophone queries credit both sax recordings; they are not incorrectly declared absent. Four absence queries ask for bagpipes, an absent singer/title, and an absent spoken topic. They remain separate from positive recall; no threshold has been selected. Specific tenor/soprano labels follow primary descriptions, with listening still needed to independently verify perceptual specificity. Language pairs and metadata variants share recordings and are correlated, not independent examples.
Run after reserving the CPU slot
python tools/media-eval/prepare_music_validation.py fixtures/music-validation-v1/fixture.json .searchi/qa/music-validation-v1 .searchi/media-runtime/bin/ffmpeg.exe .searchi/media-runtime/bin/ffprobe.exe
cargo test --manifest-path tools/media-eval/Cargo.toml --bin metadata_validation --locked
$env:SEARCHI_FORCE_CPU = '1'
cargo run --manifest-path tools/media-eval/Cargo.toml --bin metadata_validation --locked -- fixtures/music-validation-v1/fixture.json .searchi/qa/music-validation-v1/prepared.json .searchi/models/embeddinggemma2 .searchi/runtime/auto/Release/searchi-gemma-worker.exe .searchi/media-runtime/bin/ffmpeg.exe .searchi/media-runtime/bin/ffprobe.exe .searchi/qa/music-validation-v1/results.json
Preparation already completed without inference. Its frozen plan SHA-256 is
8598312836f3c027b578f9f93ca1cd2d38d708bc0fd713fe174f10776934cbb2;
fixture SHA-256 is
f9d7785ff937020b802e6c56252e64bbc3cd22e9a09709ddce9b82f9adc1b577.
Use a fresh output directory for another preparation; existing plans and result
files are preserved. Decoder hashes and actual container tags are checked.
The diagnostic uses actual Google EmbeddingGemma 2 on CPU, one thread. Each of
the eight distinct audio sources is encoded once and both photos at 280 tokens.
Metadata fields use neutral File metadata, the unchanged 2048-byte whitelist
policy and maximum over each file's real evidence. All field, source and variant
citations remain explicit; a metadata hit has no fabricated timestamp. Four
lanes compare media-only, original tags, missing tags and misleading tags without
tuned weights or cutoff. Report positive recall and no-answer failure separately.
This direct cosine diagnostic is not production ANN/hybrid validation.