Ship floating launcher, automatic discovery and bounded media search improvements

This commit is contained in:
karl committed 2026-10-14 11:44:15 +02:00
1 parent 01279a6b25
commit 521966a2ac
43 files changed
+3165 -267

No files matched your search

+83
View File
@@ -0,0 +1,83 @@
# Bounded speech and timeline verification
2026-10-14, Windows development VM, CPU only (`SEARCHI_FORCE_CPU=1`).
This closes a focused reassessment of the existing speech implementation;
it does not introduce another ASR model or claim perfect speech recognition.
Google EmbeddingGemma 2 remains the retrieval model. The pinned local
whisper.cpp base model, Silero VAD and DTW supply recognized passages and
estimated source positions.
## Practical improvement already retained
The previous frozen publisher-timed diagnostic improved strict timestamp-start
hits from **4/8 to 6/8** with VAD plus DTW. Both remaining orange-soil queries
still missed their passage. See [the unchanged comparison](SPEECH-DTW.md).
That diagnostic is one English recording with correlated queries, not a current
end-to-end hybrid score. It cannot establish word-level precision.
The shipped implementation uses bounded 30-second windows with two-second
overlap, source-preserving decoding, speech gating, and sentence locations from
first/last token points. It retains errors rather than inventing transcripts.
No new timing offsets, label-specific heuristics or larger model were added:
the remaining measured misses concern recognized content, and the available
evidence does not justify increasing indexing cost merely to move timestamps.
## New genuine runtime check
The exact 30-second PCM from the earlier failed long-source service run was
retested through the current normal distribution worker, forced onto CPU.
The existing Rust integration test copies the runtime and models into genuine
Unicode paths, verifies their inventory, transcribes twice, and requires stable,
nonempty, bounded passages and zero warm model-loading time.
| Measurement | First call | Warm call |
| --- | ---: | ---: |
| Native VAD | 97.8375 ms | 113.021 ms |
| Native ASR | 47,312.2 ms | 48,474.65 ms |
| Native model loading | 235.8515 ms | 0 ms |
| Transport | 44.8708 ms | 48.6905 ms |
| CPU fallback | 0 ms | 0 ms |
The test **passed** in 129.23 seconds including inventory verification,
relocation and cleanup. These are indexing timings, not search latency.
Successful isolated repetition does not explain or erase the earlier service
transport failure, which remains recorded in [speech integration](SPEECH-INTEGRATION.md).
The existing next-source recovery remains in place.
| Verified input/artifact | SHA-256 |
| --- | --- |
| Exact previously failed PCM | `03ef72d2a4cb926b7d31fbaef757a6cb954a55d6d6961085ea6992728c8d9e37` |
| Normal speech worker | `9c34aa2b12a4c2ab876a1573045ef9a31e19e7dfff76a2b34efdf45555ad97aa` |
| Runtime inventory | `98ca365d4a298b23ca240967e005750db83613b09eaa89ed714afb711b42c87d` |
Reproduction uses `crates/searchi-speech/tests/native_cpu.rs`, with
`SEARCHI_TEST_SPEECH_WORKER=.searchi/s-a3/runtime/searchi-speech-worker.exe`,
the pinned base/Silero files in `.searchi/speech/models`, and
`SEARCHI_TEST_SPEECH_PCM=.searchi/speech-runtime/nasa-exact-crash/input.f32`:
```powershell
$env:SEARCHI_FORCE_CPU = '1'
cargo test -p searchi-speech --test native_cpu --locked -- --ignored --nocapture
```
## Actual media source-clock checks
Both real FFmpeg integration tests passed in 3.53 seconds:
- Audio beginning two seconds into a video decodes at its actual source origin;
a whole-file decode preserves the preceding silence.
- An internal timestamp gap, bounded seek and final tail preserve exact source
sample positions without shifting later speech towards the beginning.
```powershell
$env:SEARCHI_FORCE_CPU = '1'
$env:SEARCHI_TEST_FFMPEG = 'C:/Users/karl/searchi/.searchi/media-runtime/bin/ffmpeg.exe'
$env:SEARCHI_TEST_FFPROBE = 'C:/Users/karl/searchi/.searchi/media-runtime/bin/ffprobe.exe'
cargo test -p searchi-service --test speech_stream_offset_cpu --locked -- --ignored --nocapture
```
These checks establish source-clock handling, not recognition accuracy. Speech
positions remain estimates; overlapping windows and imperfect transcripts can
still produce repeated or incomplete passages. The previously recorded tiny
English/German positive retrieval check and music/silence negative recognition
controls remain useful evidence, without a new broad media-quality rating.