6.8 KiB
Local speech runtime
This worker uses pinned whisper.cpp d1be6fde11ac6e0407606b4e42fe72d34add8037,
multilingual base and Silero v6.2.0. Google EmbeddingGemma 2 continues to embed
recognized passages; speech recognition never replaces Google semantic search.
Model download/verification belongs to searchi's shared asset setup.
Normal builds use one auto runtime with dynamic CPU and Vulkan (Windows/Linux)
or Metal (macOS) backends. Missing acceleration SDKs fail configuration. Native
speech GGML libraries belong to this isolated runtime directory, never Google's
directory. The explicit --device cpu build is an internal VM testing tool.
python tools/speech-runtime/build.py --source PATH_TO_CLEAN_PINNED_WHISPER_CPP --output NEW_OWNED_BUILD_DIRECTORY --device auto
Output runtime/runtime-manifest.json records every runtime file's exact size/SHA,
the pinned upstream commit and build-input hashes. The Rust client checks all
inventory files before launch and incorporates the inventory hash in its source
completion fingerprint. A changed backend DLL therefore invalidates reuse.
The builder compiles only; native Linux/macOS loading, full system-dependency
closure, actual GPU speed/fallback and release packaging still require evidence.
Normal Linux builds reject staged ELF files with build-tree RUNPATH or undeclared
dependencies. Each shared target uses $ORIGIN for relocation; a system Vulkan
loader is permitted only for the dynamically loaded Vulkan backend. The declared
system C/C++ libraries remain a distribution compatibility requirement. A full
relocated recognition test is still required after compilation.
Windows auto builds reuse the verified Vulkan SDK/loader provisioning and selected
Visual Studio CRT bundling helpers. The isolated runtime inventories its own
vulkan-1.dll, full release CRT set and their redistribution notices. Shared helper
inputs (including imported provenance/macOS inspection modules) are hashed along
with the five speech build inputs; app-root DLLs are not assumed to satisfy them.
Actual internal v3 CPU evidence: English and German fixture tests passed with
SEARCHI_FORCE_CPU=1, the Auto client API, supplementary Unicode model/runtime
paths, repeated stable bounded passages and persistent model reuse. English ASR
was 26.31/27.80 seconds, German 28.13/22.08 seconds (first/warm) on this shared VM;
warm model loading was zero and fallback cost zero. Worker SHA-256 was
299c1b784eac46b31d228453a399e1378f327fc9ed5e34ab950c804da33e433a;
inventory SHA-256 4904995685a5e55e0a790d9cc409dc8458e39c30cda4097c5d70bf75cb5c976c.
This static CPU artifact does not prove normal dynamic-loader isolation, GPU
execution, speech retrieval quality or precise independently annotated timing.
The subsequent normal Windows auto build completed with all 27 inventoried
files (58,913,466 bytes). Its worker SHA-256 is
0b716fa17e8216ed46c7755d15acab20f658254b99780c7a68bab03d765c55bc;
inventory SHA-256 0e116c604141af147bd91736ec0722a0660dcda8c9da008e494c464753f4e5a1.
The same English Unicode/context test passed with forced CPU: first/warm ASR
41.31/38.46 seconds, warm load zero, fallback zero. Packaging compilation overlapped
this shared VM run, so these are functional timings, not a clean benchmark.
Fifty snapshots of the owned worker observed ggml-cpu-cascadelake.dll,
ggml.dll, ggml-base.dll and whisper.dll; neither ggml-vulkan.dll nor
vulkan-1.dll was observed. The pinned CPU-only loader branch loads only the CPU
module. Sampling alone does not prove that a module was never loaded.
Ignored observation artifact SHA-256:
5a090b34e6a0d1ad88893295bc8db54e01f818db2ec20aca3c097d277720e158.
Actual GPU execution/fallback and other operating systems remain unexecuted.
Windows MSBuild's nested Vulkan shader compiler failed with a long build output
path; the failed log SHA-256 is
59f1820d6ac6adac6f525d8084c442c1ed261753eb34fcce916dc157041f777c.
An unchanged-source build using a shorter owned output path completed. Prefer a
short build output directory on Windows; no CPU-only distribution was substituted.
If a worker transport dies, the failing source retains its error. begin_source
permits one verified same-fingerprint restart at the next source boundary, keeping
the current CPU/device choice. Malformed protocol, UTF-8 or alignment outputs do
not authorize recovery or repeated inference of the failing source.
The persistent worker receives binary requests: operation byte (1 live VAD,
2 contiguous recognition), little-endian u32 sample count and mono little-endian
f32 PCM, at most 480000 finite samples. Responses have a four-byte little-endian
JSON byte length, maximum 4 MiB, strict UTF-8. No arbitrary source paths or stored
VAD reports enter requests. Model paths are startup arguments; Windows uses
wmain and explicit UTF-16 → UTF-8 conversion. Native logs are consumed without
forwarding recognized bytes. Error text exposed by the parent is a fixed code.
VAD resets state for every ≤30-second source window; this is a new live-window policy, not identical to the historical whole-source frozen diagnostic. It retains original source intersections and sends actual contiguous speech PCM to base ASR. Base loads lazily and remains warm. DTW uses the base preset, disabled flash attention and one CPU thread. Adjacent native token byte pieces may be joined to recover an actual UTF-8 codepoint; incomplete recognized bytes fail closed. Sentence spans use first/last genuine DTW points and can have zero span. No text replacement, clamped DTW time or label-specific passage exists.
The service admits a complete ≤30-second transcribe call to its shared background budget. Metrics report native load, VAD, ASR and parent transport separately. This does not preempt an executing native call or bound decoder/ANN CPU. A worker operation has a 600-second IO/inference deadline; malformed framing, oversized output, process exit and timeout stop/reap the owned worker. Native worker creates no subprocesses. Cancellation between source windows belongs to the service.
Structural checks:
cargo test -p searchi-speech --locked
cargo clippy -p searchi-speech --all-targets --locked -- -D warnings
python -m unittest discover -s tools/speech-runtime
Owned subprocess transport fixtures require Python and can be run explicitly
with cargo test -p searchi-speech actual_subprocess -- --ignored; no model loads.
The native searchi-speech-worker --self-test checks strict UTF-8/JSON encoding
without loading models. Actual speech proof is a separate ignored CPU test using
SEARCHI_TEST_SPEECH_WORKER, SEARCHI_TEST_SPEECH_BASE,
SEARCHI_TEST_SPEECH_VAD, and verified SEARCHI_TEST_SPEECH_PCM (mono16k f32LE).
It relocates the runtime and exact models to owned Unicode paths, checks genuine
passages and persistent-context reuse, and cleans only that fixture.
Run actual inference only after the coordinator grants its CPU slot.