101 lines
6.3 KiB
Markdown
101 lines
6.3 KiB
Markdown
# Independent English/German speech preparation v1
|
|
|
|
No ASR, VAD or Google inference has run on this fixture. The ten topic/absence
|
|
queries and two source recordings were frozen before preparation. The sources
|
|
are human LibriVox readings of Article 1 of the Universal Declaration of Human
|
|
Rights, unrelated to the earlier NASA topics. They share translated content and
|
|
are correlated; this is not a representative multilingual speech benchmark.
|
|
|
|
| Source | Human reader | Recording permission | Original bytes |
|
|
| --- | --- | --- | ---: |
|
|
| [English Article 1](https://commons.wikimedia.org/wiki/File:Universal_Declaration_of_Human_Rights_-_eng_-_drb_-_Art1.ogg) | David Barnes | LibriVox public-domain dedication / copyrighted free use | 168186 |
|
|
| [German Article 1](https://commons.wikimedia.org/wiki/File:Universal_Declaration_of_Human_Rights_-_ger_-_robu_-_Art1.ogg) | Roman Buettner | LibriVox public-domain dedication / copyrighted free use | 151136 |
|
|
|
|
The publisher pages explicitly identify the human readers, document and language.
|
|
The primary reference text is [UN Article 1 in English](https://www.un.org/about-us/universal-declaration-of-human-rights)
|
|
and [UNRIC Article 1 in German](https://unric.org/de/allgemeine-erklaerung-menschenrechte/).
|
|
The manifest records reference keywords and topic paraphrases, never model
|
|
captions or recognition-derived ground truth. Full primary text is externally
|
|
linked; it must not be supplied as an ASR prompt or indexed as if recognized.
|
|
|
|
Only **319322 new bytes** were downloaded. Sources are checked against official
|
|
Commons API size/SHA-1 and acquired SHA-256. The existing independently licensed
|
|
US Navy drum cadence supplies the instrumental control, with its previously
|
|
locked source bytes. Including that reused source, original inputs total
|
|
1893869 bytes. Generated silence is exact zero PCM, released CC0.
|
|
|
|
## Four-clip pool and honest timestamp scope
|
|
|
|
Prepared audio is mono 16-kHz s16 WAV. Each human recording remains complete,
|
|
with five seconds of exact leading silence; no reference words are cropped.
|
|
The resulting English/German durations are 22.512 and 16.219 seconds. Music and
|
|
silence controls each cover exactly thirty seconds. Core WAVs total 3159568 bytes.
|
|
The music waveform is the first thirty seconds of the already locked drum source;
|
|
no descriptive tags are used to gate recognition.
|
|
|
|
Actual source audio stream starts are verified as zero, and the five-second
|
|
padding/PCM source mapping is exact. **The publishers do not provide phrase-level
|
|
timed captions.** Neither the padding boundary nor the whole article bounds prove
|
|
the first phoneme or each sentence's start. Strict utterance-point scoring is
|
|
explicitly unavailable; do not count a start anywhere in the article as precise
|
|
localization success. Independent listening/timed annotations remain required.
|
|
|
|
Six positive EN/DE queries describe equal dignity/rights, reason/conscience and
|
|
brotherly treatment. Either language's recording is relevant: translated topic
|
|
equivalence is intentional. Four absent queries ask about Mars funding and salary/
|
|
paid leave. Silence and instrumental music remain in every candidate pool, and
|
|
any recognized hallucination remains an explicit measured failure/competitor.
|
|
Per-language recognized concept coverage is separate from semantic file recall.
|
|
|
|
## Separate seam primitives
|
|
|
|
`seam-plan.json` freezes two artificial repeat controls, outside the four-clip
|
|
retrieval pool. English source samples repeat twice and German samples three
|
|
times after the same known padding, producing 40.024 and 38.657 seconds. Their
|
|
2517880 additional WAV bytes and exact piecewise source maps stress a 30-second
|
|
window with two-second overlap. These joins are artificial repetitions, not
|
|
natural uninterrupted speech. Do not extrapolate their quality to long talks.
|
|
|
|
Independent byte checks verified each derived piece equals the original decoded
|
|
source PCM and the prefix is all zeros. Compare full-source VAD on these short
|
|
controls with bounded independent windows, recording interval disagreement,
|
|
source coverage, seam omissions/duplicates and processing cost. Restarted VAD
|
|
need not reproduce a continuously initialized VAD's probabilities. Preserve
|
|
differences rather than changing thresholds or treating an unverified reference
|
|
as infallible. The same labels must not choose gate intervals or discard a seam.
|
|
|
|
## Preparation and acceptance
|
|
|
|
```powershell
|
|
python -m unittest discover -s tools/speech-eval -p test_prepare_validation.py
|
|
python tools/speech-eval/prepare_validation.py fixtures/speech-validation-v1/fixture.json .searchi/speech-validation-v1-seams .searchi/media-runtime/bin/ffmpeg.exe .searchi/media-runtime/bin/ffprobe.exe
|
|
```
|
|
|
|
Preparation already completed; existing files are preserved, so use a fresh
|
|
owned directory for another preparation. Five structural tests passed. Exact
|
|
padding and every seam's source pieces were checked independently without a
|
|
model. No recognition/quality score follows. Hashes are frozen in `inputs.lock.json`:
|
|
|
|
- Fixture: `e3a850d7a2385e250408f57fe51f4620ce80a20f8625ced42205f02cc46d517d`.
|
|
- Seam plan: `5da4400814c4965fc12a87bb00c36665300730a702a69acd65a72b9f5201ac9e`.
|
|
- Prepared `.searchi/speech-validation-v1-seams/prepared.json`:
|
|
`085f34f0af863656a28e7a905746973a54d1123bb07053541e09c8af0a58bb3f`.
|
|
|
|
The earlier four-clip-only plan
|
|
`4635e772834b0eed847459f5f4d47d7c99a591f0adba2c60ce75712557092279`
|
|
is preserved under its original ignored directory and was never inferred.
|
|
|
|
Before accepted scoring, require every source/window and all ten queries, strict
|
|
UTF-8 and bounded times, zero excluded failed competitors, unchanged worker/model/
|
|
decoder hashes and exact source-byte provenance. Report genuine German recognition
|
|
and all preregistered reference concepts, source-level recall/MRR, absent-return
|
|
rate and gate false positives separately. Both negative controls must retain
|
|
zero accepted recognized speech segments; raw Google audio search stays available.
|
|
Measure decode/VAD/ASR/alignment/Google/storage, cold model load, foreground impact
|
|
and warm reuse separately. ASR remains bounded to thirty seconds even for seam
|
|
controls. Pause between work units for foreground priority; do not claim native
|
|
preemption during one inference call.
|
|
|
|
This plan does not adopt a provider, change production search or close fine
|
|
timestamp, long-source, accent, representative-noise or human-listening gates.
|