Add local spoken passages with Google retrieval and isolated runtime setup
This commit is contained in:
1 parent
bc9459785d
commit
dab703413e
55 files changed
+3633
-70
No files matched your search
@@ -0,0 +1,65 @@
|
||||
# Local spoken passages
|
||||
|
||||
Google EmbeddingGemma 2 remains the semantic model for text, images, audio and
|
||||
video. An isolated local whisper.cpp worker adds recognized text to the same
|
||||
Google index. It does not replace raw audio evidence or introduce a hosted
|
||||
embedding backend.
|
||||
|
||||
The service covers an audio track with contiguous windows of at most thirty
|
||||
seconds, overlapping by two seconds. Live Silero detection resets per window;
|
||||
multilingual base recognition uses DTW token points. This differs from the
|
||||
historical whole-source diagnostic in [SPEECH-DTW.md](SPEECH-DTW.md). Its 6/8
|
||||
timing score cannot be carried over to this implementation.
|
||||
|
||||
Recognized passages retain source start/end points and exact recognized UTF-8.
|
||||
A point span can have zero duration. The UI starts playback at the estimated
|
||||
start and explains recognition uncertainty in the small match detail. A source
|
||||
point is not a verified complete utterance boundary.
|
||||
|
||||
Raw Google media and recognized speech have independent completion policies.
|
||||
Adding or retrying recognition reuses compatible raw vectors. Optional speech
|
||||
failures preserve raw search evidence and report partial coverage. Excerpts
|
||||
require current source bytes/revisions and the issued recognized passage;
|
||||
same-stat overwrites invalidate them. CLI, UI and MCP use this shared behavior.
|
||||
|
||||
Desktop setup downloads and verifies approximately 149 MB of base/VAD model
|
||||
data in addition to Google assets. Google query readiness stays independent.
|
||||
The native worker is shipped in an isolated runtime directory. Normal packages
|
||||
require both acceleration and CPU support; Windows VM execution is CPU only.
|
||||
Automatic retries use source revision and speech policy, preventing generation
|
||||
publication from creating an endless retry loop.
|
||||
|
||||
Each native call uses one admitted background CPU thread. Detailed indexing
|
||||
metrics separate admission, decoding, VAD, model loading, ASR, transport and
|
||||
fallback cost. A running native call cannot be preempted by a foreground query
|
||||
on a one-thread budget. Raw media decoding and ANN work are outside this
|
||||
inference budget.
|
||||
|
||||
## Evidence in this source change
|
||||
|
||||
- Shared lifecycle, passage bounds, partial failures, provenance and policy
|
||||
tests passed. The complete workspace test command passed with genuine-model
|
||||
tests explicitly ignored pending their dedicated run.
|
||||
- Two genuine CPU FFmpeg tests passed: a two-second audio origin, an internal
|
||||
timestamp gap, seeks and tails preserve the source timeline.
|
||||
- Desktop Rust: 48 tests and strict Clippy passed. Frontend: 62 tests and
|
||||
formatting passed, including a stable partial-speech retry key.
|
||||
- The actual CLI speech setup verified/reused the pinned base and Silero cache.
|
||||
- Existing atomic staging tests passed; Windows package tests reject missing,
|
||||
changed, stale and internal CPU-only speech runtimes before replacing an
|
||||
existing archive.
|
||||
|
||||
## Checks still required before delivery
|
||||
|
||||
- [ ] Current native worker English/German context reuse and Unicode paths.
|
||||
- [ ] Actual shared-service ASR plus Google lifecycle integration.
|
||||
- [ ] Frozen independent four-file/ten-query and separate seam evaluation.
|
||||
- [ ] Normal hardware-detecting runtime staging and Windows package startup.
|
||||
- [ ] Publish/install the verified Windows update.
|
||||
- [ ] Native playback and graphical interaction, actual GPU execution, and
|
||||
other OS runtime checks.
|
||||
|
||||
The independent fixture has no publisher phrase timestamps. It can measure
|
||||
retrieval and negative controls, but cannot establish precise word-level timing
|
||||
accuracy. Music relevance and general no-answer rejection remain separate open
|
||||
quality work; diagnostic integration alone does not raise the media rating.
|
||||
Reference in new issue
Block a user