Publish first usable Windows release instructions and validation

This commit is contained in:
karl committed 2026-10-13 11:20:23 +02:00
1 parent 5eeba29c0d
commit eabb1fbb54
9 files changed
+490 -19

No files matched your search

+151 -7
View File
@@ -174,9 +174,10 @@ transcripts, worker errors and inferred content never become executable instruct
- [ ] Validate normal package dependency closure and missing-driver CPU fallback
on each native OS; execute physical GPU tests outside this Windows VM.
Next implementation slice: isolated worker + manifest verification + genuine NASA
proof. Keep the initial runtime experiment separate from schema/ranking changes
until it establishes that recognized segments actually fix the measured problem.
The runtime and genuine retrieval diagnostics below inform future work; they do
not add a production speech feature. Further ASR/VAD research is deferred while
the usable Windows desktop takes priority. Revisit gating/alignment before schema
or ranking changes that would expose unreliable recognized speech to users.
## Actual internal CPU diagnostic, 2026-10-13
@@ -221,7 +222,150 @@ timestamp output. They do not establish speech recognition quality.
Historical raw/checkpoint hashes and invalid classification remain under ignored
`.searchi/speech/invalid-truncated-audio.json`; model bytes and actual raw outputs
are excluded from Git. The owned sequential session 20331 is terminal with exit 1
following intentional invalid-experiment termination. Corrected base/small
inference is paused pending the coordinator's CPU slot, followed by independent
caption review and genuine Google transcript ranking. Neither candidate is an
accepted product default; the media quality rating remains 4/10.
following intentional invalid-experiment termination. A subsequent corrected run
also detected a source-timeline issue: NASA's container ends at 85.333333 seconds,
but its primary audio ends at 85.262 seconds. Session 6691 failed closed on the
72 ms tail discrepancy, preserving its completed checkpoint. The harness now
uses the actual primary audio stream's duration/start/index, absolute seeking
with `-seek_timestamp 1`, and explicit stream mapping. Missing stream duration
is an actionable error; container duration is never silently substituted.
Twelve tests passed in 1.937 seconds after that correction, including actual
FFmpeg mux/decode with a longer video, audio offset 0.125 seconds and container
offset five seconds. The audio timeline was 5.125–36.125 seconds; its one-second
tail decoded 15997 samples, within the unchanged one-millisecond rounding bound.
The zero-start 44.1/48 kHz tests still decoded exactly 480000/16000 samples.
## Corrected base candidate evidence
The corrected base evaluation completed all five assets with zero timestamp or
sample-coverage validation failures. Report SHA:
`9e33984afac9480452f506070dee0156fcc8d7f0d6882c8f5feeeaf5cae57de2`.
Executed frozen harness SHA:
`16e29bb70d3e1eeaad1f5b8ec223bff1a6f359215bcf8dabe52512a451855346`.
| Source | Windows | Recognized segments | ASR process seconds | Decode seconds |
| --- | ---: | ---: | ---: | ---: |
| NASA | 3 | 19 | 23.322 | 0.542 |
| Piano | 9 | 9 | 210.720 | 2.167 |
| Violin | 1 | 1 | 4.471 | 0.212 |
| Bark | 1 | 0 | 4.373 | 0.113 |
| Roaring effect | 1 | 1 | 4.732 | 0.111 |
These are observed serial CPU process times including model startup, not warmed
throughput or a controlled hardware comparison. NASA's decoded windows contained
480000, 480000 and 404192 samples, covering the actual primary audio range.
Manual comparison with publisher captions finds the Artemis funding and Mars
statements recognized, but the initial Moon-choice statement is merged into a
segment starting at zero and the orange-soil exchange is omitted. Recognition
does not prove Google ranking or a precise timestamp success rate.
The instrumental piano emitted an unrelated water phrase and repetitive numbers
or punctuation in several windows. Music descriptions also appear. These must
remain competing retrieved candidates in the next diagnostic; filtering them
by knowledge of labels would hide failure. The source's published instrumental
description supports a hallucination concern, but independent audio listening
remains open. The current report field `quality_failed=false` means only that
structural timestamp/sample validation passed; **it does not certify speech or
search quality**. No model default is accepted.
## Corrected small candidate evidence
Owned sequential session 89200 finished with exit zero after evaluating all five
assets, base then small. Both used the same frozen corrected harness and verified
model bytes. The complete small report SHA is
`1660c95c71c9b8b1cb3b77f8f1ec422fb0954f2aacd2c16ea773901c0cfbe099`.
| Source | Windows | Valid recognized segments | ASR process seconds | Decode seconds |
| --- | ---: | ---: | ---: | ---: |
| NASA | 3 | 19 | 86.250 | 0.601 |
| Piano | 9 | 8 | 1176.426 | 2.399 |
| Violin | 1 | 1 | 16.045 | 0.162 |
| Bark | 1 | 0 | 16.701 | 0.112 |
| Roaring effect | 1 | 1 | 25.082 | 0.113 |
Small emitted repetitive inferred words in non-English languages on the
instrumental piano, with substantial CPU cost. Two actual timestamp outputs
failed validation: the 14.454-second piano tail returned a 0–30-second segment,
and the 2.039-second bark window returned a 0–2.5-second segment. Decoded sample
counts were correct (231254 and 32618 respectively); these are ASR timestamp
failures, not the historical resampling bug. Raw JSON is retained; no timestamp
was clamped. The report's `complete=true` establishes complete evaluation,
while `quality_failed=true` records failed structural validation.
NASA's small recognition retained funding and Mars passages but misrecognized
the orange-soil words and still produced coarse segment boundaries. No numeric
timestamp retrieval improvement follows from caption inspection. The small
ranker may retain diagnostic results, but its reduced valid-window pool must be
`quality_score_eligible=false`; the scorer refuses an accepted numeric quality
score for incomplete or reduced competitor pools. Thirteen harness regressions,
including real FFmpeg coverage/timeline checks and this scoring gate, passed in
the coordinator's verified-FFmpeg run (2.100 seconds).
Final process inspection found no owned ASR/runner processes; the CPU slot was
released for genuine Google transcript ranking. The current run removed its
temporary decoded WAV/JSON. An older scratch directory from intentionally stopped
invalid session 20331 remains ignored: automatic approval review rejected native
PowerShell recursive cleanup with only “blocked by policy” as its reason. No
alternate deletion method was attempted.
Neither ASR model is an accepted product default. Production Google transcript integration,
speech activity gating, tighter alignment, hallucination control, German speech,
resource scheduling and native distribution still require evidence. Publish raw
Google media search before completing derived transcription so slow ASR cannot
block initial semantic readiness. Transcript publication needs its own bounded
completed-source work and atomic eligibility update. The production media rating
remains 4/10.
## Actual Google transcript retrieval diagnostic
The coordinator's owned session 2984 completed base and small ranking with exit
zero. The shared real Google adapter ran on CPU, one thread: document encoding
used the normal indexing role (2048-token context), then the document process was
dropped before opening the query role (512 tokens). Both reported
`google/embeddinggemma-2`, 768 dimensions and `b11461-no-logits-v1`. The diagnostic
runner was a Debug build with SHA
`be096c6dfe158b991520c060ca83478bdd0cd5c34d64fab4a2413d8b01de8f15`;
the actual native Google worker SHA was
`c01cb3ec13cfca722ee768e8350e1399a8b9a4d14a24a35adc3473a5cfc82a73`.
The base experiment ranked all **30 valid recognized segments from all five
sources**, preserving music/sound descriptions and hallucinated text as
competitors. Actual Google tokenizer windows preserved original UTF-8 source
ranges. Segment ranking used exact cosine and maximum over its token windows;
the frozen labels did not select or rewrite candidates. The eight unchanged
English/German NASA queries produced:
| Metric | Result |
| --- | ---: |
| Correct file at rank one | 8/8 |
| Strict timestamp start hit at rank one | 4/8 |
| Segment overlap at rank one | 6/8 |
Both Moon-choice queries selected the merged segment starting at zero, before
the labelled 1.551–3.603-second utterance. Both orange-soil queries selected an
unrelated passage starting at 24.320 seconds because the recognized exchange
was missing. Funding queries selected 43.920 seconds and Mars queries selected
60 seconds, within the independently frozen intervals. Correct-file success
does not establish precise spoken-passage localization.
The complete base Google report SHA is
`2fbc860f992c3ec561f54137e30744b00265455792e360010ab1fc78ab25173a`;
the accepted diagnostic scoring artifact SHA is
`8f92c8eeed41db7e78be0e188500e441d53b4fa49fa8c0353a77df74643ce895`.
Raw artifacts remain ignored under `.searchi/speech`. This is **direct exact-cosine
retrieval of recognized text**, not production ANN/hybrid retrieval of raw audio.
The historical production 0/8 audio-window result is useful motivation but is
not an equivalent head-to-head experiment; no production improvement or score
change is claimed. Eight correlated queries about four topics in one English
video do not validate German speech, held-out relevance or abstention.
The small diagnostic also completed, with report SHA
`aca99d36b3bdf013b1a1074c33f7232de1c1b79af62d00236d53267ce510c9c1`.
Its two invalid ASR windows (piano tail and bark) were explicitly excluded:
`complete=true`, `quality_score_eligible=false`, `excluded_window_count=2`.
Diagnostic ranks are retained, but **no accepted numeric quality score** is
published for that reduced competitor pool. Neither model is adopted. Next steps
are speech gating/alignment proof, representative negative controls, actual
production integration and an equivalent shared-pipeline rerun; the media rating
remains 4/10.