Files

Genuine Google transcript ranking diagnostic

This internal experiment ranks actual recognized ASR text with the shared Google EmbeddingGemma 2 adapter. It is separate from production ANN/hybrid search and does not add speech recognition to the application.

All valid segments from the NASA video and four music/sound competitors enter the same candidate pool. Original source filenames are the only document titles; recognized text is unchanged. Frozen relevance intervals never select, rewrite, filter or rank candidates. General Google search queries use the normal 512-token query role; document encoding uses 2048 tokens. The document process is dropped before opening the query process. Both use forced CPU and one thread.

Long segments use the actual Google tokenizer's source-preserving windows. Every vector's original segment and UTF-8 byte range are recorded. Exact cosine scores are computed from genuine Google vectors and aggregated by maximum over token windows belonging to the same original ASR segment. All vector and segment ranks are retained; at most the first 100 segment hits per query are exported for the existing scorer. Ties use stable source order. No confidence threshold or oracle matching is applied.

The additional frozen gated-vad-v1 diagnostic policy accepts only the exact VAD report SHA-256 093be2005b57fe16a8f85161cc9da8966e70e2050cf19fa4122531e304dd70df. The ASR report embeds its original UTF-8 bytes and parsed document; both must agree. Source probes, decoder hashes and the pinned base Whisper worker must match. The ranker independently rebuilds every contiguous window from the frozen bounded source-sample intersections, retaining interval identity and explicit sub-millisecond tail remainder. Omitted intervals, changed windows or claimed no-speech status without frozen empty intervals are rejected before inference. All five sources remain explicit competitors; no relevance label selects which source receives ASR. Existing ungated reports still require full audio coverage.

cargo test --manifest-path tools/speech-rank/Cargo.toml --locked
cargo build --release --manifest-path tools/speech-rank/Cargo.toml --locked
$env:SEARCHI_FORCE_CPU = '1'
tools/speech-rank/target/release/searchi-speech-rank.exe .searchi/speech/base-results.json fixtures/media/fixture.json C:/path/to/verified-models C:/path/to/searchi-gemma-worker.exe .searchi/speech/base-google-ranks.json
python tools/speech-eval/score.py --rankings .searchi/speech/base-google-ranks.json --output .searchi/speech/base-google-score.json

Run only after the shared inference CPU slot is released. The output directory must exist, and the output filename must be new. create_new reserves evidence before costly work; atomic JSON checkpoints retain incomplete phases. Inputs are bounded, source/model hashes verified, and recognized segment timestamps reconstructed from actual Whisper JSON to reject stale normalized text. Reports from truncated decoding or an older policy are rejected. The artifact records the actual ASR bytes, worker/runner/model hashes and process/timing information.

Invalid ASR windows are explicitly marked ineligible with reasons, never quietly discarded. A report with any excluded window has quality_score_eligible: false: do not publish a quality score from a reduced competitor pool. Valid hallucinated music or sound text remains eligible and must compete. complete reports prove the diagnostic finished, not speech accuracy.

Exported start_ms/end_ms refer to the full original media timeline. They are ASR segment approximations, potentially affected by 30-second recognition boundaries; they are not word alignment. A token window retains its containing segment's same interval. The existing eight frozen NASA queries constitute a tiny diagnostic, not held-out calibration, general media quality or production proof.

The optional frozen ordinary-token-sentences-v1 input retains those original segments and supplies full Whisper JSON. The ranker independently reconstructs exact recognized sentences from ordinary token times, excludes structural tokens from passage text, and verifies the declared policy and Python contract hashes. Changed text, missing/out-of-range/unordered timestamps, forged normalized passages, omitted windows or partial provenance are rejected. All five sources remain competitors. Real alignment failures are retained as excluded windows; such a report cannot receive an accepted quality score. The old ungated/gated policies are unchanged. Token times remain approximate recognized-speech coordinates; this experiment does not make them verified quotations or add production transcription.

After the complete base token-sentence ASR run and coordinator CPU-slot grant, use the existing executable command with input .searchi/speech/base-token-sentences-v1.json and a new output filename, then the same frozen eight-query scorer. Do not overwrite old results or select passages with relevance labels.

The separate dtw-token-sentences-v1 experiment independently validates native DTW centisecond points and uses the first/last token points as approximate sentence spans. These are not token durations. Ordinary interval failures remain in the embedded ASR evidence and are independently checked; they do not authorize fabricating or clamping DTW points. Missing/invalid DTW points exclude a window and prevent accepted scoring. Use a separate DTW report/output and retain the failed ordinary-v1 run. The frozen five sources/eight queries remain unchanged.

The separately frozen gated-dtw-v1 combination embeds exact manifest bytes with SHA d3bf453f21dac03fd029344af2fe700480cf686d2ba5b5d7fcc79134dda1c563. The ranker verifies component hashes and the original frozen VAD bytes, rebuilds all twelve speech windows from source intersections, requires explicit no-speech status for empty competitors, then independently reconstructs DTW points. All five sources remain mandatory; dropped windows, fabricated no-speech claims, tampered policy bytes and invalid points are rejected. This combination leaves the old gated base and ungated DTW policies unchanged. Structural token tests provide no new quality result; actual model inference remains a separate gate.