Freeze independent English German speech and boundary validation inputs
This commit is contained in:
1 parent
62dd3cc5bb
commit
bc9459785d
9 files changed
+590
No files matched your search
@@ -0,0 +1,100 @@
|
||||
# Independent English/German speech preparation v1
|
||||
|
||||
No ASR, VAD or Google inference has run on this fixture. The ten topic/absence
|
||||
queries and two source recordings were frozen before preparation. The sources
|
||||
are human LibriVox readings of Article 1 of the Universal Declaration of Human
|
||||
Rights, unrelated to the earlier NASA topics. They share translated content and
|
||||
are correlated; this is not a representative multilingual speech benchmark.
|
||||
|
||||
| Source | Human reader | Recording permission | Original bytes |
|
||||
| --- | --- | --- | ---: |
|
||||
| [English Article 1](https://commons.wikimedia.org/wiki/File:Universal_Declaration_of_Human_Rights_-_eng_-_drb_-_Art1.ogg) | David Barnes | LibriVox public-domain dedication / copyrighted free use | 168186 |
|
||||
| [German Article 1](https://commons.wikimedia.org/wiki/File:Universal_Declaration_of_Human_Rights_-_ger_-_robu_-_Art1.ogg) | Roman Buettner | LibriVox public-domain dedication / copyrighted free use | 151136 |
|
||||
|
||||
The publisher pages explicitly identify the human readers, document and language.
|
||||
The primary reference text is [UN Article 1 in English](https://www.un.org/about-us/universal-declaration-of-human-rights)
|
||||
and [UNRIC Article 1 in German](https://unric.org/de/allgemeine-erklaerung-menschenrechte/).
|
||||
The manifest records reference keywords and topic paraphrases, never model
|
||||
captions or recognition-derived ground truth. Full primary text is externally
|
||||
linked; it must not be supplied as an ASR prompt or indexed as if recognized.
|
||||
|
||||
Only **319322 new bytes** were downloaded. Sources are checked against official
|
||||
Commons API size/SHA-1 and acquired SHA-256. The existing independently licensed
|
||||
US Navy drum cadence supplies the instrumental control, with its previously
|
||||
locked source bytes. Including that reused source, original inputs total
|
||||
1893869 bytes. Generated silence is exact zero PCM, released CC0.
|
||||
|
||||
## Four-clip pool and honest timestamp scope
|
||||
|
||||
Prepared audio is mono 16-kHz s16 WAV. Each human recording remains complete,
|
||||
with five seconds of exact leading silence; no reference words are cropped.
|
||||
The resulting English/German durations are 22.512 and 16.219 seconds. Music and
|
||||
silence controls each cover exactly thirty seconds. Core WAVs total 3159568 bytes.
|
||||
The music waveform is the first thirty seconds of the already locked drum source;
|
||||
no descriptive tags are used to gate recognition.
|
||||
|
||||
Actual source audio stream starts are verified as zero, and the five-second
|
||||
padding/PCM source mapping is exact. **The publishers do not provide phrase-level
|
||||
timed captions.** Neither the padding boundary nor the whole article bounds prove
|
||||
the first phoneme or each sentence's start. Strict utterance-point scoring is
|
||||
explicitly unavailable; do not count a start anywhere in the article as precise
|
||||
localization success. Independent listening/timed annotations remain required.
|
||||
|
||||
Six positive EN/DE queries describe equal dignity/rights, reason/conscience and
|
||||
brotherly treatment. Either language's recording is relevant: translated topic
|
||||
equivalence is intentional. Four absent queries ask about Mars funding and salary/
|
||||
paid leave. Silence and instrumental music remain in every candidate pool, and
|
||||
any recognized hallucination remains an explicit measured failure/competitor.
|
||||
Per-language recognized concept coverage is separate from semantic file recall.
|
||||
|
||||
## Separate seam primitives
|
||||
|
||||
`seam-plan.json` freezes two artificial repeat controls, outside the four-clip
|
||||
retrieval pool. English source samples repeat twice and German samples three
|
||||
times after the same known padding, producing 40.024 and 38.657 seconds. Their
|
||||
2517880 additional WAV bytes and exact piecewise source maps stress a 30-second
|
||||
window with two-second overlap. These joins are artificial repetitions, not
|
||||
natural uninterrupted speech. Do not extrapolate their quality to long talks.
|
||||
|
||||
Independent byte checks verified each derived piece equals the original decoded
|
||||
source PCM and the prefix is all zeros. Compare full-source VAD on these short
|
||||
controls with bounded independent windows, recording interval disagreement,
|
||||
source coverage, seam omissions/duplicates and processing cost. Restarted VAD
|
||||
need not reproduce a continuously initialized VAD's probabilities. Preserve
|
||||
differences rather than changing thresholds or treating an unverified reference
|
||||
as infallible. The same labels must not choose gate intervals or discard a seam.
|
||||
|
||||
## Preparation and acceptance
|
||||
|
||||
```powershell
|
||||
python -m unittest discover -s tools/speech-eval -p test_prepare_validation.py
|
||||
python tools/speech-eval/prepare_validation.py fixtures/speech-validation-v1/fixture.json .searchi/speech-validation-v1-seams .searchi/media-runtime/bin/ffmpeg.exe .searchi/media-runtime/bin/ffprobe.exe
|
||||
```
|
||||
|
||||
Preparation already completed; existing files are preserved, so use a fresh
|
||||
owned directory for another preparation. Five structural tests passed. Exact
|
||||
padding and every seam's source pieces were checked independently without a
|
||||
model. No recognition/quality score follows. Hashes are frozen in `inputs.lock.json`:
|
||||
|
||||
- Fixture: `e3a850d7a2385e250408f57fe51f4620ce80a20f8625ced42205f02cc46d517d`.
|
||||
- Seam plan: `5da4400814c4965fc12a87bb00c36665300730a702a69acd65a72b9f5201ac9e`.
|
||||
- Prepared `.searchi/speech-validation-v1-seams/prepared.json`:
|
||||
`085f34f0af863656a28e7a905746973a54d1123bb07053541e09c8af0a58bb3f`.
|
||||
|
||||
The earlier four-clip-only plan
|
||||
`4635e772834b0eed847459f5f4d47d7c99a591f0adba2c60ce75712557092279`
|
||||
is preserved under its original ignored directory and was never inferred.
|
||||
|
||||
Before accepted scoring, require every source/window and all ten queries, strict
|
||||
UTF-8 and bounded times, zero excluded failed competitors, unchanged worker/model/
|
||||
decoder hashes and exact source-byte provenance. Report genuine German recognition
|
||||
and all preregistered reference concepts, source-level recall/MRR, absent-return
|
||||
rate and gate false positives separately. Both negative controls must retain
|
||||
zero accepted recognized speech segments; raw Google audio search stays available.
|
||||
Measure decode/VAD/ASR/alignment/Google/storage, cold model load, foreground impact
|
||||
and warm reuse separately. ASR remains bounded to thirty seconds even for seam
|
||||
controls. Pause between work units for foreground priority; do not claim native
|
||||
preemption during one inference call.
|
||||
|
||||
This plan does not adopt a provider, change production search or close fine
|
||||
timestamp, long-source, accent, representative-noise or human-listening gates.
|
||||
Reference in new issue
Block a user