93 lines
5.1 KiB
Markdown
93 lines
5.1 KiB
Markdown
# Local spoken passages
|
|
|
|
Google EmbeddingGemma 2 remains the semantic model for text, images, audio and
|
|
video. An isolated local whisper.cpp worker adds recognized text to the same
|
|
Google index. It does not replace raw audio evidence or introduce a hosted
|
|
embedding backend.
|
|
|
|
The service covers an audio track with contiguous windows of at most thirty
|
|
seconds, overlapping by two seconds. Live Silero detection resets per window;
|
|
multilingual base recognition uses DTW token points. This differs from the
|
|
historical whole-source diagnostic in [SPEECH-DTW.md](SPEECH-DTW.md). Its 6/8
|
|
timing score cannot be carried over to this implementation.
|
|
|
|
Recognized passages retain source start/end points and exact recognized UTF-8.
|
|
A point span can have zero duration. The UI starts playback at the estimated
|
|
start and explains recognition uncertainty in the small match detail. A source
|
|
point is not a verified complete utterance boundary.
|
|
|
|
Raw Google media and recognized speech have independent completion policies.
|
|
Adding or retrying recognition reuses compatible raw vectors. Optional speech
|
|
failures preserve raw search evidence and report partial coverage. Excerpts
|
|
require current source bytes/revisions and the issued recognized passage;
|
|
same-stat overwrites invalidate them. CLI, UI and MCP use this shared behavior.
|
|
|
|
Desktop setup downloads and verifies approximately 149 MB of base/VAD model
|
|
data in addition to Google assets. Google query readiness stays independent.
|
|
The native worker is shipped in an isolated runtime directory. Normal packages
|
|
require both acceleration and CPU support; Windows VM execution is CPU only.
|
|
Automatic retries use source revision and speech policy, preventing generation
|
|
publication from creating an endless retry loop.
|
|
|
|
Each native call uses one admitted background CPU thread. Detailed indexing
|
|
metrics separate admission, decoding, VAD, model loading, ASR, transport and
|
|
fallback cost. A running native call cannot be preempted by a foreground query
|
|
on a one-thread budget. Raw media decoding and ANN work are outside this
|
|
inference budget.
|
|
|
|
## Evidence in this source change
|
|
|
|
- Shared lifecycle, passage bounds, partial failures, provenance and policy
|
|
tests passed. The complete workspace test command passed with genuine-model
|
|
tests explicitly ignored pending their dedicated run.
|
|
- Two genuine CPU FFmpeg tests passed: a two-second audio origin, an internal
|
|
timestamp gap, seeks and tails preserve the source timeline.
|
|
- Desktop Rust: 48 tests and strict Clippy passed. Frontend: 62 tests and
|
|
formatting passed, including a stable partial-speech retry key.
|
|
- The actual CLI speech setup verified/reused the pinned base and Silero cache.
|
|
- Existing atomic staging tests passed; Windows package tests reject missing,
|
|
changed, stale and internal CPU-only speech runtimes before replacing an
|
|
existing archive.
|
|
|
|
## Checks still required before delivery
|
|
|
|
- [x] Current CPU native worker English/German context reuse and Unicode paths.
|
|
- [x] Actual shared-service ASR plus Google lifecycle integration.
|
|
- [ ] Frozen independent four-file/ten-query and separate seam evaluation.
|
|
- [ ] Normal hardware-detecting runtime staging and Windows package startup.
|
|
- [ ] Publish/install the verified Windows update.
|
|
- [ ] Native playback and graphical interaction, actual GPU execution, and
|
|
other OS runtime checks.
|
|
|
|
The independent fixture has no publisher phrase timestamps. It can measure
|
|
retrieval and negative controls, but cannot establish precise word-level timing
|
|
accuracy. Music relevance and general no-answer rejection remain separate open
|
|
quality work; diagnostic integration alone does not raise the media rating.
|
|
|
|
## Genuine CPU functional proofs
|
|
|
|
The internal v3 worker SHA is
|
|
`299c1b784eac46b31d228453a399e1378f327fc9ed5e34ab950c804da33e433a`;
|
|
its inventory SHA is
|
|
`4904995685a5e55e0a790d9cc409dc8458e39c30cda4097c5d70bf75cb5c976c`.
|
|
English/German tests relocated the worker and real models to paths containing
|
|
`München`, `測試` and `🚀`. Each repeated recognition returned stable bounded
|
|
passages, and the second call reported zero additional model-load time. These
|
|
checks selected `Auto` in the Rust client with `SEARCHI_FORCE_CPU=1`, so native
|
|
execution was explicitly CPU. They do not validate dynamic acceleration loading.
|
|
|
|
The actual service lifecycle test passed in 113.50 seconds. Adding recognition,
|
|
an injected optional failure on policy upgrade, and a genuine retry all retained
|
|
byte-identical raw Google vectors with zero additional raw-audio encodes.
|
|
Recognized results, exact verified excerpts, stable policy/source retry identity
|
|
and the retained Google query reader were checked. This uses a query derived
|
|
from recognition to test the contract; it is not an independent relevance score.
|
|
Raw log `.searchi/qa/service-speech-google-cpu-v3.log`: 1292 bytes, SHA
|
|
`170700adf9315ae15ae3ab5a7df6a77ab01dc5b776be179c861d2461cfc48c03`.
|
|
|
|
The first normal auto compilation reached Vulkan setup but failed on Windows
|
|
nested MSBuild path length. Its log is retained, SHA
|
|
`59f1820d6ac6adac6f525d8084c442c1ed261753eb34fcce916dc157041f777c`.
|
|
An unchanged build in a shorter owned output directory is underway; no CPU-only
|
|
distribution was substituted.
|