66 lines
3.6 KiB
Markdown
66 lines
3.6 KiB
Markdown
# Local spoken passages
|
|
|
|
Google EmbeddingGemma 2 remains the semantic model for text, images, audio and
|
|
video. An isolated local whisper.cpp worker adds recognized text to the same
|
|
Google index. It does not replace raw audio evidence or introduce a hosted
|
|
embedding backend.
|
|
|
|
The service covers an audio track with contiguous windows of at most thirty
|
|
seconds, overlapping by two seconds. Live Silero detection resets per window;
|
|
multilingual base recognition uses DTW token points. This differs from the
|
|
historical whole-source diagnostic in [SPEECH-DTW.md](SPEECH-DTW.md). Its 6/8
|
|
timing score cannot be carried over to this implementation.
|
|
|
|
Recognized passages retain source start/end points and exact recognized UTF-8.
|
|
A point span can have zero duration. The UI starts playback at the estimated
|
|
start and explains recognition uncertainty in the small match detail. A source
|
|
point is not a verified complete utterance boundary.
|
|
|
|
Raw Google media and recognized speech have independent completion policies.
|
|
Adding or retrying recognition reuses compatible raw vectors. Optional speech
|
|
failures preserve raw search evidence and report partial coverage. Excerpts
|
|
require current source bytes/revisions and the issued recognized passage;
|
|
same-stat overwrites invalidate them. CLI, UI and MCP use this shared behavior.
|
|
|
|
Desktop setup downloads and verifies approximately 149 MB of base/VAD model
|
|
data in addition to Google assets. Google query readiness stays independent.
|
|
The native worker is shipped in an isolated runtime directory. Normal packages
|
|
require both acceleration and CPU support; Windows VM execution is CPU only.
|
|
Automatic retries use source revision and speech policy, preventing generation
|
|
publication from creating an endless retry loop.
|
|
|
|
Each native call uses one admitted background CPU thread. Detailed indexing
|
|
metrics separate admission, decoding, VAD, model loading, ASR, transport and
|
|
fallback cost. A running native call cannot be preempted by a foreground query
|
|
on a one-thread budget. Raw media decoding and ANN work are outside this
|
|
inference budget.
|
|
|
|
## Evidence in this source change
|
|
|
|
- Shared lifecycle, passage bounds, partial failures, provenance and policy
|
|
tests passed. The complete workspace test command passed with genuine-model
|
|
tests explicitly ignored pending their dedicated run.
|
|
- Two genuine CPU FFmpeg tests passed: a two-second audio origin, an internal
|
|
timestamp gap, seeks and tails preserve the source timeline.
|
|
- Desktop Rust: 48 tests and strict Clippy passed. Frontend: 62 tests and
|
|
formatting passed, including a stable partial-speech retry key.
|
|
- The actual CLI speech setup verified/reused the pinned base and Silero cache.
|
|
- Existing atomic staging tests passed; Windows package tests reject missing,
|
|
changed, stale and internal CPU-only speech runtimes before replacing an
|
|
existing archive.
|
|
|
|
## Checks still required before delivery
|
|
|
|
- [ ] Current native worker English/German context reuse and Unicode paths.
|
|
- [ ] Actual shared-service ASR plus Google lifecycle integration.
|
|
- [ ] Frozen independent four-file/ten-query and separate seam evaluation.
|
|
- [ ] Normal hardware-detecting runtime staging and Windows package startup.
|
|
- [ ] Publish/install the verified Windows update.
|
|
- [ ] Native playback and graphical interaction, actual GPU execution, and
|
|
other OS runtime checks.
|
|
|
|
The independent fixture has no publisher phrase timestamps. It can measure
|
|
retrieval and negative controls, but cannot establish precise word-level timing
|
|
accuracy. Music relevance and general no-answer rejection remain separate open
|
|
quality work; diagnostic integration alone does not raise the media rating.
|