Validate gated Google retrieval and record unchanged localization quality

This commit is contained in:
karl committed 2026-10-13 13:49:23 +02:00
1 parent af04cec27c
commit 59908adfa6
5 files changed
+267 -5

No files matched your search

+35 -2
View File
@@ -243,11 +243,44 @@ segments versus nineteen; segment count does not prove transcript accuracy.
Music/sound ASR text vanished through explicit no-speech decisions, not manual
phrase filtering. Original Google audio retrieval remains independent.
The next gate is genuine Google retrieval on all eight unchanged queries and
independent speech recall/alignment assessment. Do not select a production
Genuine Google retrieval on all eight unchanged queries is now measured below;
independent speech recall/alignment assessment remains open. Do not select a production
policy from timing or this tiny corpus alone. A future persistent ASR session
could reduce model-start overhead but must be measured independently.
## Genuine Google comparison result
The updated Rust diagnostic verifies the exact embedded frozen VAD bytes,
reconstructs every source interval window and requires explicit no-speech status
for the four empty competitors. The original ungated coverage contract remains
strict. Six Rust tests passed, including omission/tampering rejection; the
agent also ran strict Clippy and built the release executable. Its recorded
SHA-256 matches the current executable:
`5a8ecb9e2ff571610cfe9f0dcd4876953d8975ff4a84972d321cd91636ceaf84`.
Actual Google EmbeddingGemma 2 CPU document/query inference completed using the
normal auto-capable worker with CPU forced. All twenty recognized segments
entered retrieval without relevance-label filtering; all five original source
identities remained verified. The complete, eligible ranking report is ignored
`.searchi/speech/gated-base-google-ranks-v2.json`, SHA-256
`1647cf5826dbeaf43f8c47209ca40af6bc2b68cd56f0c5f44eda36ed2eab8a30`.
The scorer report SHA-256 is
`3d15e72764cf2329b31caf9020af9182a91acb7229b13d7dec1b24464351c0dc`.
An initial run used a nonexistent worker path and retained its incomplete
verification checkpoint; it performed no successful inference and was not scored.
| Fixed eight-query metric | Ungated base | VAD-gated base |
| --- | ---: | ---: |
| Correct top file | 8/8 | 8/8 |
| Top start point within labelled interval | 4/8 | 4/8 |
| Top interval overlaps labelled interval | 6/8 | 6/8 |
The speech gate removed incidental music/sound transcripts and reduced this
pool's processing cost, but did **not** improve these localization metrics.
It is not selected as a production default. More independent speech coverage,
recognition and alignment work is needed before adding transcription enrichment.
The original Google media embeddings remain the product's media search path.
Open gates: actual one-thread audit, detection/recognition
comparison, ordinary and DTW timestamp regression, independent held-out quality,
background starvation/cancellation/crash recovery, packaging/license review,