Genuine Google transcript ranking diagnostic
This internal experiment ranks actual recognized ASR text with the shared Google EmbeddingGemma 2 adapter. It is separate from production ANN/hybrid search and does not add speech recognition to the application.
All valid segments from the NASA video and four music/sound competitors enter the same candidate pool. Original source filenames are the only document titles; recognized text is unchanged. Frozen relevance intervals never select, rewrite, filter or rank candidates. General Google search queries use the normal 512-token query role; document encoding uses 2048 tokens. The document process is dropped before opening the query process. Both use forced CPU and one thread.
Long segments use the actual Google tokenizer's source-preserving windows. Every vector's original segment and UTF-8 byte range are recorded. Exact cosine scores are computed from genuine Google vectors and aggregated by maximum over token windows belonging to the same original ASR segment. All vector and segment ranks are retained; at most the first 100 segment hits per query are exported for the existing scorer. Ties use stable source order. No confidence threshold or oracle matching is applied.
The additional frozen gated-vad-v1 diagnostic policy accepts only the exact
VAD report SHA-256 093be2005b57fe16a8f85161cc9da8966e70e2050cf19fa4122531e304dd70df.
The ASR report embeds its original UTF-8 bytes and parsed document; both must
agree. Source probes, decoder hashes and the pinned base Whisper worker must
match. The ranker independently rebuilds every contiguous window from the frozen
bounded source-sample intersections, retaining interval identity and explicit
sub-millisecond tail remainder. Omitted intervals, changed windows or claimed
no-speech status without frozen empty intervals are rejected before inference.
All five sources remain explicit competitors; no relevance label selects which
source receives ASR. Existing ungated reports still require full audio coverage.
cargo test --manifest-path tools/speech-rank/Cargo.toml --locked
cargo build --release --manifest-path tools/speech-rank/Cargo.toml --locked
$env:SEARCHI_FORCE_CPU = '1'
tools/speech-rank/target/release/searchi-speech-rank.exe .searchi/speech/base-results.json fixtures/media/fixture.json C:/path/to/verified-models C:/path/to/searchi-gemma-worker.exe .searchi/speech/base-google-ranks.json
python tools/speech-eval/score.py --rankings .searchi/speech/base-google-ranks.json --output .searchi/speech/base-google-score.json
Run only after the shared inference CPU slot is released. The output directory
must exist, and the output filename must be new. create_new reserves evidence
before costly work; atomic JSON checkpoints retain incomplete phases. Inputs
are bounded, source/model hashes verified, and recognized segment timestamps
reconstructed from actual Whisper JSON to reject stale normalized text. Reports
from truncated decoding or an older policy are rejected. The artifact records
the actual ASR bytes, worker/runner/model hashes and process/timing information.
Invalid ASR windows are explicitly marked ineligible with reasons, never quietly
discarded. A report with any excluded window has quality_score_eligible: false:
do not publish a quality score from a reduced competitor pool. Valid hallucinated
music or sound text remains eligible and must compete. complete reports prove
the diagnostic finished, not speech accuracy.
Exported start_ms/end_ms refer to the full original media timeline. They are
ASR segment approximations, potentially affected by 30-second recognition
boundaries; they are not word alignment. A token window retains its containing
segment's same interval. The existing eight frozen NASA queries constitute a tiny
diagnostic, not held-out calibration, general media quality or production proof.
The optional frozen ordinary-token-sentences-v1 input retains those original
segments and supplies full Whisper JSON. The ranker independently reconstructs
exact recognized sentences from ordinary token times, excludes structural tokens
from passage text, and verifies the declared policy and Python contract hashes.
Changed text, missing/out-of-range/unordered timestamps, forged normalized
passages, omitted windows or partial provenance are rejected. All five sources
remain competitors. Real alignment failures are retained as excluded windows;
such a report cannot receive an accepted quality score. The old ungated/gated
policies are unchanged. Token times remain approximate recognized-speech
coordinates; this experiment does not make them verified quotations or add
production transcription.
After the complete base token-sentence ASR run and coordinator CPU-slot grant,
use the existing executable command with input
.searchi/speech/base-token-sentences-v1.json and a new output filename, then the
same frozen eight-query scorer. Do not overwrite old results or select passages
with relevance labels.
The separate dtw-token-sentences-v1 experiment independently validates native
DTW centisecond points and uses the first/last token points as approximate
sentence spans. These are not token durations. Ordinary interval failures remain
in the embedded ASR evidence and are independently checked; they do not authorize
fabricating or clamping DTW points. Missing/invalid DTW points exclude a window
and prevent accepted scoring. Use a separate DTW report/output and retain the
failed ordinary-v1 run. The frozen five sources/eight queries remain unchanged.