Add local spoken passages with Google retrieval and isolated runtime setup

This commit is contained in:
karl committed 2026-10-13 20:18:44 +02:00
1 parent bc9459785d
commit dab703413e
55 files changed
+3633 -70

No files matched your search

+79
View File
@@ -0,0 +1,79 @@
# Local speech runtime
This worker uses pinned whisper.cpp `d1be6fde11ac6e0407606b4e42fe72d34add8037`,
multilingual base and Silero v6.2.0. Google EmbeddingGemma 2 continues to embed
recognized passages; speech recognition never replaces Google semantic search.
Model download/verification belongs to searchi's shared asset setup.
Normal builds use one auto runtime with dynamic CPU and Vulkan (Windows/Linux)
or Metal (macOS) backends. Missing acceleration SDKs fail configuration. Native
speech GGML libraries belong to this isolated runtime directory, never Google's
directory. The explicit `--device cpu` build is an internal VM testing tool.
```sh
python tools/speech-runtime/build.py --source PATH_TO_CLEAN_PINNED_WHISPER_CPP --output NEW_OWNED_BUILD_DIRECTORY --device auto
```
Output `runtime/runtime-manifest.json` records every runtime file's exact size/SHA,
the pinned upstream commit and build-input hashes. The Rust client checks all
inventory files before launch and incorporates the inventory hash in its source
completion fingerprint. A changed backend DLL therefore invalidates reuse.
The builder compiles only; native Linux/macOS loading, full system-dependency
closure, actual GPU speed/fallback and release packaging still require evidence.
Windows auto builds reuse the verified Vulkan SDK/loader provisioning and selected
Visual Studio CRT bundling helpers. The isolated runtime inventories its own
`vulkan-1.dll`, full release CRT set and their redistribution notices. Shared helper
inputs (including imported provenance/macOS inspection modules) are hashed along
with the five speech build inputs; app-root DLLs are not assumed to satisfy them.
Actual internal v3 CPU evidence: English and German fixture tests passed with
`SEARCHI_FORCE_CPU=1`, the Auto client API, supplementary Unicode model/runtime
paths, repeated stable bounded passages and persistent model reuse. English ASR
was 26.31/27.80 seconds, German 28.13/22.08 seconds (first/warm) on this shared VM;
warm model loading was zero and fallback cost zero. Worker SHA-256 was
`299c1b784eac46b31d228453a399e1378f327fc9ed5e34ab950c804da33e433a`;
inventory SHA-256 `4904995685a5e55e0a790d9cc409dc8458e39c30cda4097c5d70bf75cb5c976c`.
This static CPU artifact does not prove normal dynamic-loader isolation, GPU
execution, speech retrieval quality or precise independently annotated timing.
The persistent worker receives binary requests: operation byte (1 live VAD,
2 contiguous recognition), little-endian u32 sample count and mono little-endian
f32 PCM, at most 480000 finite samples. Responses have a four-byte little-endian
JSON byte length, maximum 4 MiB, strict UTF-8. No arbitrary source paths or stored
VAD reports enter requests. Model paths are startup arguments; Windows uses
`wmain` and explicit UTF-16 → UTF-8 conversion. Native logs are consumed without
forwarding recognized bytes. Error text exposed by the parent is a fixed code.
VAD resets state for every ≤30-second source window; this is a **new live-window
policy**, not identical to the historical whole-source frozen diagnostic. It
retains original source intersections and sends actual contiguous speech PCM to
base ASR. Base loads lazily and remains warm. DTW uses the base preset, disabled
flash attention and one CPU thread. Adjacent native token byte pieces may be
joined to recover an actual UTF-8 codepoint; incomplete recognized bytes fail
closed. Sentence spans use first/last genuine DTW points and can have zero span.
No text replacement, clamped DTW time or label-specific passage exists.
The service admits a complete ≤30-second transcribe call to its shared background
budget. Metrics report native load, VAD, ASR and parent transport separately.
This does not preempt an executing native call or bound decoder/ANN CPU. A worker
operation has a 600-second IO/inference deadline; malformed framing, oversized
output, process exit and timeout stop/reap the owned worker. Native worker creates
no subprocesses. Cancellation between source windows belongs to the service.
Structural checks:
```sh
cargo test -p searchi-speech --locked
cargo clippy -p searchi-speech --all-targets --locked -- -D warnings
python -m unittest discover -s tools/speech-runtime
```
Owned subprocess transport fixtures require Python and can be run explicitly
with `cargo test -p searchi-speech actual_subprocess -- --ignored`; no model loads.
The native `searchi-speech-worker --self-test` checks strict UTF-8/JSON encoding
without loading models. Actual speech proof is a separate ignored CPU test using
`SEARCHI_TEST_SPEECH_WORKER`, `SEARCHI_TEST_SPEECH_BASE`,
`SEARCHI_TEST_SPEECH_VAD`, and verified `SEARCHI_TEST_SPEECH_PCM` (mono16k f32LE).
It relocates the runtime and exact models to owned Unicode paths, checks genuine
passages and persistent-context reuse, and cleans only that fixture.
Run actual inference only after the coordinator grants its CPU slot.