Add local spoken passages with Google retrieval and isolated runtime setup
This commit is contained in:
1 parent
bc9459785d
commit
dab703413e
55 files changed
+3633
-70
No files matched your search
@@ -0,0 +1,79 @@
|
||||
# Local speech runtime
|
||||
|
||||
This worker uses pinned whisper.cpp `d1be6fde11ac6e0407606b4e42fe72d34add8037`,
|
||||
multilingual base and Silero v6.2.0. Google EmbeddingGemma 2 continues to embed
|
||||
recognized passages; speech recognition never replaces Google semantic search.
|
||||
Model download/verification belongs to searchi's shared asset setup.
|
||||
|
||||
Normal builds use one auto runtime with dynamic CPU and Vulkan (Windows/Linux)
|
||||
or Metal (macOS) backends. Missing acceleration SDKs fail configuration. Native
|
||||
speech GGML libraries belong to this isolated runtime directory, never Google's
|
||||
directory. The explicit `--device cpu` build is an internal VM testing tool.
|
||||
|
||||
```sh
|
||||
python tools/speech-runtime/build.py --source PATH_TO_CLEAN_PINNED_WHISPER_CPP --output NEW_OWNED_BUILD_DIRECTORY --device auto
|
||||
```
|
||||
|
||||
Output `runtime/runtime-manifest.json` records every runtime file's exact size/SHA,
|
||||
the pinned upstream commit and build-input hashes. The Rust client checks all
|
||||
inventory files before launch and incorporates the inventory hash in its source
|
||||
completion fingerprint. A changed backend DLL therefore invalidates reuse.
|
||||
The builder compiles only; native Linux/macOS loading, full system-dependency
|
||||
closure, actual GPU speed/fallback and release packaging still require evidence.
|
||||
Windows auto builds reuse the verified Vulkan SDK/loader provisioning and selected
|
||||
Visual Studio CRT bundling helpers. The isolated runtime inventories its own
|
||||
`vulkan-1.dll`, full release CRT set and their redistribution notices. Shared helper
|
||||
inputs (including imported provenance/macOS inspection modules) are hashed along
|
||||
with the five speech build inputs; app-root DLLs are not assumed to satisfy them.
|
||||
|
||||
Actual internal v3 CPU evidence: English and German fixture tests passed with
|
||||
`SEARCHI_FORCE_CPU=1`, the Auto client API, supplementary Unicode model/runtime
|
||||
paths, repeated stable bounded passages and persistent model reuse. English ASR
|
||||
was 26.31/27.80 seconds, German 28.13/22.08 seconds (first/warm) on this shared VM;
|
||||
warm model loading was zero and fallback cost zero. Worker SHA-256 was
|
||||
`299c1b784eac46b31d228453a399e1378f327fc9ed5e34ab950c804da33e433a`;
|
||||
inventory SHA-256 `4904995685a5e55e0a790d9cc409dc8458e39c30cda4097c5d70bf75cb5c976c`.
|
||||
This static CPU artifact does not prove normal dynamic-loader isolation, GPU
|
||||
execution, speech retrieval quality or precise independently annotated timing.
|
||||
|
||||
The persistent worker receives binary requests: operation byte (1 live VAD,
|
||||
2 contiguous recognition), little-endian u32 sample count and mono little-endian
|
||||
f32 PCM, at most 480000 finite samples. Responses have a four-byte little-endian
|
||||
JSON byte length, maximum 4 MiB, strict UTF-8. No arbitrary source paths or stored
|
||||
VAD reports enter requests. Model paths are startup arguments; Windows uses
|
||||
`wmain` and explicit UTF-16 → UTF-8 conversion. Native logs are consumed without
|
||||
forwarding recognized bytes. Error text exposed by the parent is a fixed code.
|
||||
|
||||
VAD resets state for every ≤30-second source window; this is a **new live-window
|
||||
policy**, not identical to the historical whole-source frozen diagnostic. It
|
||||
retains original source intersections and sends actual contiguous speech PCM to
|
||||
base ASR. Base loads lazily and remains warm. DTW uses the base preset, disabled
|
||||
flash attention and one CPU thread. Adjacent native token byte pieces may be
|
||||
joined to recover an actual UTF-8 codepoint; incomplete recognized bytes fail
|
||||
closed. Sentence spans use first/last genuine DTW points and can have zero span.
|
||||
No text replacement, clamped DTW time or label-specific passage exists.
|
||||
|
||||
The service admits a complete ≤30-second transcribe call to its shared background
|
||||
budget. Metrics report native load, VAD, ASR and parent transport separately.
|
||||
This does not preempt an executing native call or bound decoder/ANN CPU. A worker
|
||||
operation has a 600-second IO/inference deadline; malformed framing, oversized
|
||||
output, process exit and timeout stop/reap the owned worker. Native worker creates
|
||||
no subprocesses. Cancellation between source windows belongs to the service.
|
||||
|
||||
Structural checks:
|
||||
|
||||
```sh
|
||||
cargo test -p searchi-speech --locked
|
||||
cargo clippy -p searchi-speech --all-targets --locked -- -D warnings
|
||||
python -m unittest discover -s tools/speech-runtime
|
||||
```
|
||||
|
||||
Owned subprocess transport fixtures require Python and can be run explicitly
|
||||
with `cargo test -p searchi-speech actual_subprocess -- --ignored`; no model loads.
|
||||
The native `searchi-speech-worker --self-test` checks strict UTF-8/JSON encoding
|
||||
without loading models. Actual speech proof is a separate ignored CPU test using
|
||||
`SEARCHI_TEST_SPEECH_WORKER`, `SEARCHI_TEST_SPEECH_BASE`,
|
||||
`SEARCHI_TEST_SPEECH_VAD`, and verified `SEARCHI_TEST_SPEECH_PCM` (mono16k f32LE).
|
||||
It relocates the runtime and exact models to owned Unicode paths, checks genuine
|
||||
passages and persistent-context reuse, and cleans only that fixture.
|
||||
Run actual inference only after the coordinator grants its CPU slot.
|
||||
Reference in new issue
Block a user