searchi/tools/speech-runtime/README.md

# Local speech runtime

This worker uses pinned whisper.cpp `d1be6fde11ac6e0407606b4e42fe72d34add8037`,
multilingual base and Silero v6.2.0. Google EmbeddingGemma 2 continues to embed
recognized passages; speech recognition never replaces Google semantic search.
Model download/verification belongs to searchi's shared asset setup.

Normal builds use one auto runtime with dynamic CPU and Vulkan (Windows/Linux)
or Metal (macOS) backends. Missing acceleration SDKs fail configuration. Native
speech GGML libraries belong to this isolated runtime directory, never Google's
directory. The explicit `--device cpu` build is an internal VM testing tool.

```sh
python tools/speech-runtime/build.py --source PATH_TO_CLEAN_PINNED_WHISPER_CPP --output NEW_OWNED_BUILD_DIRECTORY --device auto
```

Output `runtime/runtime-manifest.json` records every runtime file's exact size/SHA,
the pinned upstream commit and build-input hashes. The Rust client checks all
inventory files before launch and incorporates the inventory hash in its source
completion fingerprint. A changed backend DLL therefore invalidates reuse.
The builder compiles only; native Linux/macOS loading, full system-dependency
closure, actual GPU speed/fallback and release packaging still require evidence.
Normal Linux builds reject staged ELF files with build-tree RUNPATH or undeclared
dependencies. Each shared target uses `$ORIGIN` for relocation; a system Vulkan
loader is permitted only for the dynamically loaded Vulkan backend. The declared
system C/C++ libraries remain a distribution compatibility requirement. A full
relocated recognition test is still required after compilation.
Windows auto builds reuse the verified Vulkan SDK/loader provisioning and selected
Visual Studio CRT bundling helpers. The isolated runtime inventories its own
`vulkan-1.dll`, full release CRT set and their redistribution notices. Shared helper
inputs (including imported provenance/macOS inspection modules) are hashed along
with the five speech build inputs; app-root DLLs are not assumed to satisfy them.

Actual internal v3 CPU evidence: English and German fixture tests passed with
`SEARCHI_FORCE_CPU=1`, the Auto client API, supplementary Unicode model/runtime
paths, repeated stable bounded passages and persistent model reuse. English ASR
was 26.31/27.80 seconds, German 28.13/22.08 seconds (first/warm) on this shared VM;
warm model loading was zero and fallback cost zero. Worker SHA-256 was
`299c1b784eac46b31d228453a399e1378f327fc9ed5e34ab950c804da33e433a`;
inventory SHA-256 `4904995685a5e55e0a790d9cc409dc8458e39c30cda4097c5d70bf75cb5c976c`.
This static CPU artifact does not prove normal dynamic-loader isolation, GPU
execution, speech retrieval quality or precise independently annotated timing.

The subsequent **normal Windows auto** build completed with all 27 inventoried
files (58,913,466 bytes). Its worker SHA-256 is
`0b716fa17e8216ed46c7755d15acab20f658254b99780c7a68bab03d765c55bc`;
inventory SHA-256 `0e116c604141af147bd91736ec0722a0660dcda8c9da008e494c464753f4e5a1`.
The same English Unicode/context test passed with forced CPU: first/warm ASR
41.31/38.46 seconds, warm load zero, fallback zero. Packaging compilation overlapped
this shared VM run, so these are functional timings, not a clean benchmark.
Fifty snapshots of the owned worker observed `ggml-cpu-cascadelake.dll`,
`ggml.dll`, `ggml-base.dll` and `whisper.dll`; neither `ggml-vulkan.dll` nor
`vulkan-1.dll` was observed. The pinned CPU-only loader branch loads only the CPU
module. Sampling alone does not prove that a module was never loaded.
Ignored observation artifact SHA-256:
`5a090b34e6a0d1ad88893295bc8db54e01f818db2ec20aca3c097d277720e158`.
Actual GPU execution/fallback and other operating systems remain unexecuted.

Windows MSBuild's nested Vulkan shader compiler failed with a long build output
path; the failed log SHA-256 is
`59f1820d6ac6adac6f525d8084c442c1ed261753eb34fcce916dc157041f777c`.
An unchanged-source build using a shorter owned output path completed. Prefer a
short build output directory on Windows; no CPU-only distribution was substituted.

If a worker transport dies, the failing source retains its error. `begin_source`
permits one verified same-fingerprint restart at the next source boundary, keeping
the current CPU/device choice. Malformed protocol, UTF-8 or alignment outputs do
not authorize recovery or repeated inference of the failing source.

The persistent worker receives binary requests: operation byte (1 live VAD,
2 contiguous recognition), little-endian u32 sample count and mono little-endian
f32 PCM, at most 480000 finite samples. Responses have a four-byte little-endian
JSON byte length, maximum 4 MiB, strict UTF-8. No arbitrary source paths or stored
VAD reports enter requests. Model paths are startup arguments; Windows uses
`wmain` and explicit UTF-16 → UTF-8 conversion. Native logs are consumed without
forwarding recognized bytes. Error text exposed by the parent is a fixed code.

VAD resets state for every ≤30-second source window; this is a **new live-window
policy**, not identical to the historical whole-source frozen diagnostic. It
retains original source intersections and sends actual contiguous speech PCM to
base ASR. Base loads lazily and remains warm. DTW uses the base preset, disabled
flash attention and one CPU thread. Adjacent native token byte pieces may be
joined to recover an actual UTF-8 codepoint; incomplete recognized bytes fail
closed. Sentence spans use first/last genuine DTW points and can have zero span.
No text replacement, clamped DTW time or label-specific passage exists.

The service admits a complete ≤30-second transcribe call to its shared background
budget. Metrics report native load, VAD, ASR and parent transport separately.
This does not preempt an executing native call or bound decoder/ANN CPU. A worker
operation has a 600-second IO/inference deadline; malformed framing, oversized
output, process exit and timeout stop/reap the owned worker. Native worker creates
no subprocesses. Cancellation between source windows belongs to the service.

Structural checks:

```sh
cargo test -p searchi-speech --locked
cargo clippy -p searchi-speech --all-targets --locked -- -D warnings
python -m unittest discover -s tools/speech-runtime
```

Owned subprocess transport fixtures require Python and can be run explicitly
with `cargo test -p searchi-speech actual_subprocess -- --ignored`; no model loads.
The native `searchi-speech-worker --self-test` checks strict UTF-8/JSON encoding
without loading models. Actual speech proof is a separate ignored CPU test using
`SEARCHI_TEST_SPEECH_WORKER`, `SEARCHI_TEST_SPEECH_BASE`,
`SEARCHI_TEST_SPEECH_VAD`, and verified `SEARCHI_TEST_SPEECH_PCM` (mono16k f32LE).
It relocates the runtime and exact models to owned Unicode paths, checks genuine
passages and persistent-context reuse, and cleans only that fixture.
Run actual inference only after the coordinator grants its CPU slot.