Document installed Windows 0.1.5 and complete Google speech timing improvement
This commit is contained in:
karl committed
1 parent
512ec4ea9f
commit
5f9df976eb
9 files changed
+176
-22
No files matched your search
@@ -39,12 +39,11 @@ sound.
|
||||
Use **All files** to narrow results by folder, file extension, modification
|
||||
date, file size or media duration. Choose a result to inspect it before opening.
|
||||
|
||||
The current source also offers **Artist (exact)** and **Track title (exact)**
|
||||
Windows 0.1.5 offers **Artist (exact)** and **Track title (exact)**
|
||||
filters for music. Enter the complete tag, with or without a text query. These
|
||||
filters ignore letter case and surrounding spaces; files without the matching
|
||||
music tag stay out of the results. Existing indexes need a media indexing pass
|
||||
before these new filters can use their tags. This improvement is not yet in the
|
||||
v0.1.4 downloads.
|
||||
before these new filters can use their tags. Windows 0.1.5 includes these filters; older downloads need an update.
|
||||
|
||||
### Find the implementation
|
||||
|
||||
@@ -86,13 +85,13 @@ autoplays when you select a result.
|
||||
|
||||
## Get searchi
|
||||
|
||||
**Windows x64 version 0.1.4 is available.**
|
||||
**Windows x64 version 0.1.5 is available.**
|
||||
|
||||
- [Download the Windows installer](https://tea.karlcom.de/karl/searchi/releases/download/v0.1.4/searchi_0.1.4_x64-setup.exe) — about 100 MB.
|
||||
- [Checksums](https://tea.karlcom.de/karl/searchi/releases/download/v0.1.4/SHA256SUMS-0.1.4.txt) · [Release notes](https://tea.karlcom.de/karl/searchi/releases/tag/v0.1.4).
|
||||
- [Download the Windows installer](https://tea.karlcom.de/karl/searchi/releases/download/v0.1.5/searchi_0.1.5_x64-setup.exe) — about 100 MB.
|
||||
- [Checksums](https://tea.karlcom.de/karl/searchi/releases/download/v0.1.5/SHA256SUMS-0.1.5.txt) · [Release notes](https://tea.karlcom.de/karl/searchi/releases/tag/v0.1.5).
|
||||
|
||||
Already installed? Close searchi before installing the update. Version 0.1.4
|
||||
also fixes indexing and previews for archived files dated before 1970.
|
||||
Already installed? Close searchi before installing the update. Version 0.1.5 adds exact music artist/title filters and retains the fixes for
|
||||
archived files dated before 1970.
|
||||
It includes isolated PDF/DOCX processing and bounded media previews.
|
||||
|
||||
The installer creates a launcher and sets up WebView2 if needed. This initial
|
||||
|
||||
@@ -27,7 +27,7 @@ searchi search --db /path/to/index.db "live recording" --media-artist "Nina Simo
|
||||
```
|
||||
|
||||
Replace `/path/to/index.db` with your existing media-indexed database. The desktop controls are **Artist (exact)** and
|
||||
**Track title (exact)**. v0.1.4 downloads predate these additions.
|
||||
**Track title (exact)**. Windows 0.1.5 includes these controls; v0.1.4 and older downloads predate them.
|
||||
|
||||
Windows CPU checks: the full ordinary workspace suite passed (asset-dependent
|
||||
checks remain explicitly ignored). Shared migration/lifecycle/eligibility tests, five CLI unit
|
||||
|
||||
+19
-9
@@ -1,6 +1,6 @@
|
||||
# Production readiness roadmap
|
||||
|
||||
Status: Windows and Linux v0.1.4 previews published; initial Windows v0.1.0 installed. Broader production gates remain open. Windows VM tests use CPU only.
|
||||
Status: Windows v0.1.5 published and installed; Linux v0.1.4 previews published. Broader production gates remain open. Windows VM tests use CPU only.
|
||||
|
||||
Karl's release priority, 2026-10-13: publish a usable first Windows version and
|
||||
install it on DEV-WIN11 now. Extensive platform/GPU, clean-machine, signing and
|
||||
@@ -16,20 +16,30 @@ Google EmbeddingGemma 2 remains the main text/image/audio/video search model.
|
||||
known two-file music memories from 2/4 to 4/4; no-answer remained 5/5 failing.
|
||||
This is hypothesis evidence, not independent production validation.
|
||||
See [music evidence](MUSIC-METADATA.md).
|
||||
- [ ] Validate and integrate metadata embeddings on independent unseen music and
|
||||
the full mixed pool with real field citations and stale-source guards.
|
||||
- [x] Validate the tag hypothesis on frozen unseen licensed music plus fixed mixed
|
||||
distractors: 8/14 → 11/14 with original tags, one genuine regression; misleading
|
||||
tags fell to 5/14. Missing tags exactly preserved media-only results. No-answer
|
||||
remained 4/4 failing. See fixtures/music-validation-v1/README.md.
|
||||
- [ ] Improve and integrate metadata evidence with source citations/guards and full
|
||||
mixed regression; unconditional field-max policy is not accepted as universal.
|
||||
- [x] Evaluate pinned compact multilingual local reranker: quantized serial/batch
|
||||
parity failed; F32 parity passed but six-pair warm cost missed the 200 ms target.
|
||||
No suppression/default integration. See LOCAL-RERANKING.md.
|
||||
- [x] Implement exact artist/title constraints in shared storage, lexical/ANN
|
||||
eligibility, CLI/MCP and desktop controls; metadata-only search skips inference.
|
||||
Unit/lifecycle evidence is recorded separately from actual model validation.
|
||||
- [x] Actual Google CPU constraint integration passed with genuine tagged WAVs,
|
||||
semantic/hybrid limit-one retrieval, empty absent identities, no-inference
|
||||
metadata lookup and warmed lifecycle refresh; see [evidence](MEDIA-CONSTRAINTS.md).
|
||||
- [ ] Package and install an updated desktop release with these exact filters.
|
||||
- [ ] Evaluate finer local ASR token/word timing and source-preserving phrase windows
|
||||
with Google text retrieval. Ordinary-token run completed but failed interval
|
||||
validation; DTW run stopped on invalid native UTF-8 and remains incomplete.
|
||||
Neither receives an accepted score. Future diagnostics retain bounded failing
|
||||
raw bytes and atomic checkpoints; see tools/speech-eval/README.md.
|
||||
- [x] Publish and install Windows 0.1.5 with exact music filters, verified public
|
||||
bytes, source/runtime provenance and actual CPU startup/upgrade; see
|
||||
[release evidence](WINDOWS-0.1.5.md). Native interaction and other OS gates stay open.
|
||||
- [x] Evaluate finer source-preserving local ASR timing with actual Google retrieval:
|
||||
the frozen VAD + DTW combination completed with all five sources retained and
|
||||
improved strict start points from 4/8 to 6/8. Rejected ordinary-token and ungated DTW runs remain
|
||||
preserved; no partial-pool scores were accepted. See SPEECH-DTW.md.
|
||||
- [ ] Validate independent speech/languages and integrate accepted alignment with
|
||||
model setup, packaging, cancellation, source guards and shared engine clients.
|
||||
- [ ] Validate richer no-answer evidence on new independent positive/negative data; do not retune the failed universal cosine cutoff on old held-out results.
|
||||
- [ ] Integrate accepted improvements into shared engine/UI/headless clients with verified setup, lifecycle and CPU performance evidence. Diagnostics alone do not raise the media quality rating.
|
||||
- [x] Publish v0.1.0 Windows installer/checksum, verify public download bytes,
|
||||
|
||||
+61
-2
@@ -1,7 +1,8 @@
|
||||
# DTW diagnostic: strict UTF-8 failure
|
||||
|
||||
Status: internal CPU diagnosis, 2026-10-13. No accepted speech/retrieval score,
|
||||
production ASR integration or new media rating follows from this evidence.
|
||||
Status: internal CPU diagnosis and completed tiny VAD/DTW comparison, 2026-10-13.
|
||||
Production ASR integration and broader independent acceptance remain open;
|
||||
no new media rating follows from this evidence.
|
||||
Google EmbeddingGemma 2 remains the production semantic model.
|
||||
|
||||
The original fifteen-window DTW experiment stopped on strict UTF-8 parsing.
|
||||
@@ -86,3 +87,61 @@ It lacks remaining piano windows and three other competitors. Neither that
|
||||
partial report nor this one-window diagnosis can receive an accepted quality
|
||||
score. Packaging, independent bilingual speech, silence/music hallucination,
|
||||
resource peaks and actual production hybrid integration remain open.
|
||||
|
||||
## Complete frozen VAD + DTW follow-up
|
||||
|
||||
A subsequent separately frozen combination used the same original Silero VAD
|
||||
source intersections and base DTW point policy. All five sources remained
|
||||
explicit: twelve speech windows in NASA, and four no-speech competitors that
|
||||
launched no ASR because of the unchanged actual VAD decisions. No label selected
|
||||
the gate or removed the malformed piano window. Raw Google audio retrieval stays
|
||||
unchanged. This combination finished with zero parsing/alignment failures and
|
||||
21 sentence passages. Source/tool hashes were revalidated; the combination
|
||||
manifest SHA is `d3bf453f21dac03fd029344af2fe700480cf686d2ba5b5d7fcc79134dda1c563`.
|
||||
|
||||
Independent Rust verification reconstructed every VAD window and DTW point from
|
||||
raw evidence before actual Google document/query CPU inference. Both models used
|
||||
one thread; indexing role retained 2048 tokens and query role 512. The normal
|
||||
auto-capable Google worker reported `google/embeddinggemma-2`, 768 dimensions,
|
||||
CPU and `b11461-no-logits-v1`. The scorer accepted the complete report with zero
|
||||
excluded windows against the unchanged eight queries.
|
||||
|
||||
| Fixed metric | Ungated base | Frozen VAD base | Frozen VAD + DTW |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| Correct top file | 8/8 | 8/8 | 8/8 |
|
||||
| Strict timestamp-start hits | 4/8 | 4/8 | **6/8** |
|
||||
| Labelled interval overlap | 6/8 | 6/8 | 6/8 |
|
||||
|
||||
The two gains against **gated base** are the Mars queries: start changed from
|
||||
54.440 seconds, before the labelled 56.539–63.496 interval, to 58.820 seconds.
|
||||
Moon starts changed from ungated 0.000 seconds to gated-base 1.700 seconds and
|
||||
gated-DTW 1.820 seconds; both gated starts already pass the frozen 1.551–3.603
|
||||
interval. Funding queries still pass. Both orange-soil queries still fail;
|
||||
their top DTW point spans are 26.690–27.030 seconds, outside the topic. Correct
|
||||
file identity alone does not solve this missing recognized passage.
|
||||
|
||||
These are four topics in **one English NASA recording**, with eight correlated
|
||||
English/German search queries. They do not validate German speech or independent
|
||||
accents. VAD produced no recognized passages for the four competitors, so 8/8
|
||||
file identity is especially weak evidence; all five source identities remain
|
||||
verified, but this is not broad music/no-answer quality. The direct recognized-
|
||||
text diagnostic is not production hybrid search. No production default,
|
||||
transcription integration or improved media self-rating follows.
|
||||
|
||||
ASR process time summed to 88090.51 ms; window decoding summed to 1784.10 ms.
|
||||
This excludes probe/hash/checkpoint and the earlier VAD cost. Google indexing
|
||||
session initialization plus document encoding took 3086.03 ms, query-session
|
||||
startup 919.69 ms. Eight query/ranking measurements had median 96.83 ms and
|
||||
nearest-rank p95 128.75 ms; these tiny shared-VM measurements are not desktop
|
||||
first-input, clean-room or large-library latency. The already running user app
|
||||
retained its warmed CPU model; it received no further coordinated search inputs.
|
||||
|
||||
| Retained ignored artifact | SHA-256 |
|
||||
| --- | --- |
|
||||
| `gated-base-dtw-results-v1.json` | `3b6a791df6503a7a4e91ed708be7b722d1c124a2cf6c5a290cd94296bebc3c41` |
|
||||
| `gated-base-dtw-google-ranks-v1.json` | `f743265a53ade09b36fef25fe4e90b70a46cd2610a12aaa976f179ab9835fe91` |
|
||||
| `gated-base-dtw-google-score-v1.json` | `4cfbc0f540ed8edcbed6e45f55fac80ee11867597152928d3407dabd7db34502` |
|
||||
| Actual Google worker | `c01cb3ec13cfca722ee768e8350e1399a8b9a4d14a24a35adc3473a5cfc82a73` |
|
||||
| Actual ranking executable | `58f4b1cf90cbf403220b60579d72ced9dd7bc795bd05a7386820e9ab84b5f519` |
|
||||
|
||||
The original failed ungated DTW and ordinary-token reports remain unchanged.
|
||||
@@ -0,0 +1,48 @@
|
||||
# Windows 0.1.5
|
||||
|
||||
Published unsigned preview from source `5ab24518b4276cc69dcd7a867743e5fedf88104b`.
|
||||
Exact artist/title controls constrain genuine music tags before every search lane.
|
||||
Empty metadata-only requests skip query inference. Existing music needs a new
|
||||
media indexing pass; legacy flattened tags are not guessed into artist identities.
|
||||
Google EmbeddingGemma 2 remains the semantic backend. Experimental tag-vector
|
||||
ranking and speech alignment are not included in this release.
|
||||
|
||||
Embedded frontend and NSIS release build passed. The underlying change passed
|
||||
the ordinary workspace suite, 48 native desktop tests and 61 frontend tests;
|
||||
strict Clippy, formatting and frontend build passed. The actual Google CPU
|
||||
constraint/lifecycle integration passed, including missing identities, strict
|
||||
limit-one retrieval, same-stat invalidation, media refresh and deletion. The
|
||||
current extracted PDF/DOCX worker additionally passed its genuine parser test.
|
||||
|
||||
Installer: 100473168 bytes, SHA-256
|
||||
`e2e8da5097b8989ba3ecab95781c730a5c2dc5adb1a76b66bb5ee21c39006b4e`.
|
||||
Public full installer/checksum downloads matched local bytes. The extracted
|
||||
parser helper verified current source provenance; normal CPU/Vulkan runtime
|
||||
passed 26 dependency records. Models remain managed first-use downloads/cache.
|
||||
|
||||
The extracted app started with isolated data, remained responsive, and launched
|
||||
its bundled genuine Google query worker with `--device cpu --role query
|
||||
--threads 1`. Only its owned process tree was closed. Native UI interactions
|
||||
were not executed.
|
||||
|
||||
The actual silent installer then upgraded the existing Windows installation
|
||||
from 0.1.0 to 0.1.5, exit zero. Settings and a SQLite-consistent index backup
|
||||
were created beforehand. Installed app SHA matched the extracted executable:
|
||||
`dc05f18952c02da0478b39621266d51953a55154894c86962d0bece78bde7ac2`.
|
||||
The first post-install launch used the OS default profile; inspection confirmed
|
||||
it and the prior explicit profile both had zero indexed files and selected roots.
|
||||
The final launch restored the previous `C:/Users/karl/.searchi/desktop` profile,
|
||||
whose schema migrated from 13 to 14 with its zero-file count retained. NSIS had
|
||||
replaced the custom VM launcher, so the desktop shortcut was restored to a
|
||||
process-local CPU/software-WebView launcher using that same prior profile.
|
||||
The distributed installer remains the normal hardware-detecting build.
|
||||
|
||||
The installed app was restarted, responsive as final PID 1192, with its bundled Google
|
||||
CPU query worker observed. It was left running for Karl. This is this-machine
|
||||
upgrade/startup evidence, not clean-machine or visual/keyboard/playback proof.
|
||||
|
||||
Raw evidence: ignored `.searchi/qa/windows-015-build.json`,
|
||||
`windows-015-public.json`, `windows-desktop-015-smoke.json`,
|
||||
`windows-015-upgrade.json`, `windows-015-installed-launch.json`.
|
||||
Backup: `.searchi/qa/upgrade-015-backup`. Physical GPU, signatures, native macOS
|
||||
and broader platform gates remain open; Linux downloads remain 0.1.4.
|
||||
@@ -138,3 +138,9 @@ frozen sample exposes a real regression and sensitivity to misleading metadata.
|
||||
No policy was retuned after seeing these scores. Production ANN/hybrid evidence,
|
||||
more representative music, human listening, missing-content calibration and
|
||||
source-specific confidence remain open. The experimental media rating is unchanged.
|
||||
|
||||
Recorded manifests use their exact observed bytes. The initial Git commits
|
||||
normalized CRLF JSON while the local run retained CRLF; commit `512ec4e` preserves
|
||||
the unchanged observed JSON bytes with `-text` attributes. Git archive validation
|
||||
now matches all eight music/reranker frozen JSON inputs byte-for-byte. No source,
|
||||
label, policy or recorded output was altered to make a result pass.
|
||||
@@ -840,5 +840,7 @@
|
||||
"existing_app_query_worker": "Warmed CPU session remained idle; no root model requests during reserved slot"
|
||||
},
|
||||
"production_policy_changed": false,
|
||||
"deployment_conclusion": "Original metadata improves aggregate retrieval but introduces a previously-correct German guitar failure; misleading metadata reduces top1 and no-answer remains unsolved. Unconditional max policy not validated for deployment. No post-score policy tuning."
|
||||
"deployment_conclusion": "Original metadata improves aggregate retrieval but introduces a previously-correct German guitar failure; misleading metadata reduces top1 and no-answer remains unsolved. Unconditional max policy not validated for deployment. No post-score policy tuning.",
|
||||
"fixture_git_byte_preservation_commit": "512ec4e",
|
||||
"provenance_note": "Initial Git metadata line-ending normalization differed from recorded local JSON bytes; commit 512ec4e retains those original observed bytes unchanged. Source/labels/policy/results were not retuned."
|
||||
}
|
||||
@@ -42,3 +42,8 @@ F32 parity passed; quantized score differences persisted. Mixed-length padded
|
||||
batching was slower. Neither result establishes general absence calibration or
|
||||
sufficient default-search latency. Detailed costs and possible VM contention are
|
||||
recorded in the linked report documentation.
|
||||
|
||||
Frozen JSON plans/manifests have `-text` Git attributes. Commit `512ec4e` restores
|
||||
the original observed CRLF bytes that early commits normalized, without changing
|
||||
policies, inputs or failed reports. Fresh Git archives now retain the exact
|
||||
recorded manifest/plan SHA-256 values across operating systems.
|
||||
@@ -65,7 +65,32 @@ base processing previously took about 65 seconds including VAD cost; DTW cost
|
||||
and memory have not been measured. Native DTW workspace defaults to an additional
|
||||
128 MiB, which is not a total memory ceiling. Strict parse failures retain bounded
|
||||
raw bytes and atomically persist incomplete status, then stop instead of skipping
|
||||
a competitor. No full rerun has established this combined policy.
|
||||
a competitor. Plan preparation alone establishes no quality outcome.
|
||||
DTW native logs are retained byte-exact as bounded base64 plus SHA so malformed
|
||||
recognized bytes in diagnostic stderr cannot cause lossy decoding. Recognized
|
||||
JSON remains strict UTF-8 and cannot be repaired by this log representation.
|
||||
|
||||
The frozen combination has now completed actual one-thread CPU ASR and Google
|
||||
ranking: all five source identities, twelve original speech windows, 21 passages,
|
||||
zero excluded windows; **6/8 strict start hits**, versus gated base 4/8, and
|
||||
unchanged 6/8 overlaps. The gain against gated base is the Mars topic; Moon
|
||||
already passed that baseline. Orange-soil recognition remains missing. This is
|
||||
one English recording/four topics/eight correlated EN/DE queries, not German
|
||||
speech validation or production hybrid evidence. See [actual evidence and hashes](../../docs/SPEECH-DTW.md).
|
||||
|
||||
The retained ASR/ranking/scorer artifact SHA values are respectively
|
||||
`3b6a791df6503a7a4e91ed708be7b722d1c124a2cf6c5a290cd94296bebc3c41`,
|
||||
`f743265a53ade09b36fef25fe4e90b70a46cd2610a12aaa976f179ab9835fe91`,
|
||||
and `4cfbc0f540ed8edcbed6e45f55fac80ee11867597152928d3407dabd7db34502`.
|
||||
Original failed diagnostics and source labels were not overwritten.
|
||||
|
||||
```powershell
|
||||
$env:SEARCHI_FORCE_CPU = '1'
|
||||
python tools/speech-vad/gated_asr.py --whisper .searchi/speech/build-cpu/bin/Release/whisper-cli.exe --model .searchi/speech/models/ggml-base.bin --ffmpeg .searchi/media-runtime/bin/ffmpeg.exe --ffprobe .searchi/media-runtime/bin/ffprobe.exe --vad-report .searchi/speech/vad-v6-results.json --scratch-parent .searchi/speech/gated-dtw-scratch --output .searchi/speech/gated-base-dtw-results-v1.json --dtw-sentences
|
||||
tools/speech-rank/target/release/searchi-speech-rank.exe .searchi/speech/gated-base-dtw-results-v1.json fixtures/media/fixture.json C:/Users/karl/.searchi/models/embeddinggemma2 .searchi/runtime/auto/Release/searchi-gemma-worker.exe .searchi/speech/gated-base-dtw-google-ranks-v1.json
|
||||
python tools/speech-eval/score.py --rankings .searchi/speech/gated-base-dtw-google-ranks-v1.json --output .searchi/speech/gated-base-dtw-google-score-v1.json
|
||||
```
|
||||
|
||||
These are exact internal commands from the recorded run. Existing outputs are
|
||||
evidence; use new filenames for another comparison and obtain the coordinator's
|
||||
CPU slot. They are not user installation instructions.
|
||||
Reference in new issue
Block a user