83 lines
4.1 KiB
Markdown
83 lines
4.1 KiB
Markdown
|
# Bounded speech and timeline verification
|
||
|
|
|
||
|
|
2026-10-14, Windows development VM, CPU only (`SEARCHI_FORCE_CPU=1`).
|
||
|
|
This closes a focused reassessment of the existing speech implementation;
|
||
|
|
it does not introduce another ASR model or claim perfect speech recognition.
|
||
|
|
Google EmbeddingGemma 2 remains the retrieval model. The pinned local
|
||
|
|
whisper.cpp base model, Silero VAD and DTW supply recognized passages and
|
||
|
|
estimated source positions.
|
||
|
|
|
||
|
|
## Practical improvement already retained
|
||
|
|
|
||
|
|
The previous frozen publisher-timed diagnostic improved strict timestamp-start
|
||
|
|
hits from **4/8 to 6/8** with VAD plus DTW. Both remaining orange-soil queries
|
||
|
|
still missed their passage. See [the unchanged comparison](SPEECH-DTW.md).
|
||
|
|
That diagnostic is one English recording with correlated queries, not a current
|
||
|
|
end-to-end hybrid score. It cannot establish word-level precision.
|
||
|
|
|
||
|
|
The shipped implementation uses bounded 30-second windows with two-second
|
||
|
|
overlap, source-preserving decoding, speech gating, and sentence locations from
|
||
|
|
first/last token points. It retains errors rather than inventing transcripts.
|
||
|
|
No new timing offsets, label-specific heuristics or larger model were added:
|
||
|
|
the remaining measured misses concern recognized content, and the available
|
||
|
|
evidence does not justify increasing indexing cost merely to move timestamps.
|
||
|
|
|
||
|
|
## New genuine runtime check
|
||
|
|
|
||
|
|
The exact 30-second PCM from the earlier failed long-source service run was
|
||
|
|
retested through the current normal distribution worker, forced onto CPU.
|
||
|
|
The existing Rust integration test copies the runtime and models into genuine
|
||
|
|
Unicode paths, verifies their inventory, transcribes twice, and requires stable,
|
||
|
|
nonempty, bounded passages and zero warm model-loading time.
|
||
|
|
|
||
|
|
| Measurement | First call | Warm call |
|
||
|
|
| --- | ---: | ---: |
|
||
|
|
| Native VAD | 97.8375 ms | 113.021 ms |
|
||
|
|
| Native ASR | 47,312.2 ms | 48,474.65 ms |
|
||
|
|
| Native model loading | 235.8515 ms | 0 ms |
|
||
|
|
| Transport | 44.8708 ms | 48.6905 ms |
|
||
|
|
| CPU fallback | 0 ms | 0 ms |
|
||
|
|
|
||
|
|
The test **passed** in 129.23 seconds including inventory verification,
|
||
|
|
relocation and cleanup. These are indexing timings, not search latency.
|
||
|
|
Successful isolated repetition does not explain or erase the earlier service
|
||
|
|
transport failure, which remains recorded in [speech integration](SPEECH-INTEGRATION.md).
|
||
|
|
The existing next-source recovery remains in place.
|
||
|
|
|
||
|
|
| Verified input/artifact | SHA-256 |
|
||
|
|
| --- | --- |
|
||
|
|
| Exact previously failed PCM | `03ef72d2a4cb926b7d31fbaef757a6cb954a55d6d6961085ea6992728c8d9e37` |
|
||
|
|
| Normal speech worker | `9c34aa2b12a4c2ab876a1573045ef9a31e19e7dfff76a2b34efdf45555ad97aa` |
|
||
|
|
| Runtime inventory | `98ca365d4a298b23ca240967e005750db83613b09eaa89ed714afb711b42c87d` |
|
||
|
|
|
||
|
|
Reproduction uses `crates/searchi-speech/tests/native_cpu.rs`, with
|
||
|
|
`SEARCHI_TEST_SPEECH_WORKER=.searchi/s-a3/runtime/searchi-speech-worker.exe`,
|
||
|
|
the pinned base/Silero files in `.searchi/speech/models`, and
|
||
|
|
`SEARCHI_TEST_SPEECH_PCM=.searchi/speech-runtime/nasa-exact-crash/input.f32`:
|
||
|
|
|
||
|
|
```powershell
|
||
|
|
$env:SEARCHI_FORCE_CPU = '1'
|
||
|
|
cargo test -p searchi-speech --test native_cpu --locked -- --ignored --nocapture
|
||
|
|
```
|
||
|
|
|
||
|
|
## Actual media source-clock checks
|
||
|
|
|
||
|
|
Both real FFmpeg integration tests passed in 3.53 seconds:
|
||
|
|
|
||
|
|
- Audio beginning two seconds into a video decodes at its actual source origin;
|
||
|
|
a whole-file decode preserves the preceding silence.
|
||
|
|
- An internal timestamp gap, bounded seek and final tail preserve exact source
|
||
|
|
sample positions without shifting later speech towards the beginning.
|
||
|
|
|
||
|
|
```powershell
|
||
|
|
$env:SEARCHI_FORCE_CPU = '1'
|
||
|
|
$env:SEARCHI_TEST_FFMPEG = 'C:/Users/karl/searchi/.searchi/media-runtime/bin/ffmpeg.exe'
|
||
|
|
$env:SEARCHI_TEST_FFPROBE = 'C:/Users/karl/searchi/.searchi/media-runtime/bin/ffprobe.exe'
|
||
|
|
cargo test -p searchi-service --test speech_stream_offset_cpu --locked -- --ignored --nocapture
|
||
|
|
```
|
||
|
|
|
||
|
|
These checks establish source-clock handling, not recognition accuracy. Speech
|
||
|
|
positions remain estimates; overlapping windows and imperfect transcripts can
|
||
|
|
still produce repeated or incomplete passages. The previously recorded tiny
|
||
|
|
English/German positive retrieval check and music/silence negative recognition
|
||
|
|
controls remain useful evidence, without a new broad media-quality rating.
|