diff --git a/README.md b/README.md index 4cff208..155c0d6 100644 --- a/README.md +++ b/README.md @@ -79,12 +79,15 @@ autoplays when you select a result. ## Get searchi -**searchi is in active development. Ready-to-install downloads have not been -published yet.** Windows, macOS and Linux are the intended desktop platforms; -packaged releases and platform testing are still being completed. +**The first Windows x64 version is available.** -Check the [releases page](https://tea.karlcom.de/karl/searchi/releases) for -published downloads, or let your local AI agent handle setup for you. +- [Download the Windows installer](https://tea.karlcom.de/karl/searchi/releases/download/v0.1.0/searchi_0.1.0_x64-setup.exe) — about 99 MB. +- [Download the portable ZIP](https://tea.karlcom.de/karl/searchi/releases/download/v0.1.0/searchi-0.1.0-windows-x64.zip) — about 145 MB. Extract the entire folder and open `searchi-desktop.exe`. +- [Checksums](https://tea.karlcom.de/karl/searchi/releases/download/v0.1.0/SHA256SUMS.txt) · [Release notes](https://tea.karlcom.de/karl/searchi/releases/tag/v0.1.0). + +The installer creates a launcher and sets up WebView2 if needed. This initial +release is unsigned. macOS and Linux packages follow; source build instructions +are available now. You can also let your local AI agent handle setup. ### Install with your agent — copy and paste @@ -95,7 +98,8 @@ Install searchi on this computer and get it ready for my first search. Official repository: https://tea.karlcom.de/karl/searchi Detect my operating system and processor architecture. Use a suitable official -release if one is available, verifying any published checksums. Otherwise, +release from the README or releases page, including the initial Windows release, +verifying its published checksums. Otherwise, set up the current source version using the repository's installation/build instructions. Handle the required tools and dependencies for me; don't just give me a list of commands to run. Keep the installation in a dedicated folder @@ -119,10 +123,10 @@ search followed by a search in my own words. ``` Your agent needs internet access and permission to install software locally. -Since installers aren't published yet, setup currently involves building the -app and may take a while. The local search model downloads approximately -**865 MB**. Windows, macOS and Linux release validation is still in progress; -your agent should report any platform limitation it encounters. +Windows x64 setup can use the published installer; other platforms currently +build from source. The local search model downloads approximately **865 MB** +once, and cached assets are reused. Your agent should report any platform +limitation it encounters. Prefer to set it up yourself? See the [build and launch instructions](docs/BUILDING.md). diff --git a/docs/BUILDING.md b/docs/BUILDING.md index b9c8f2c..1aeade2 100644 --- a/docs/BUILDING.md +++ b/docs/BUILDING.md @@ -87,7 +87,8 @@ cargo build --manifest-path apps/desktop/src-tauri/Cargo.toml --release --featur ``` The crate rejects Release compilation without this feature; the Tauri build -CLI enables it automatically. A development-server URL is not a release gate. +CLI also needs `--features custom-protocol` explicitly for this crate. A +development-server URL is not a release gate. Record native window interaction separately from process startup, browser screenshots and worker inference; those are different release gates. diff --git a/docs/DEPENDENCY-REVIEW.md b/docs/DEPENDENCY-REVIEW.md index 51667a8..f59bccf 100644 --- a/docs/DEPENDENCY-REVIEW.md +++ b/docs/DEPENDENCY-REVIEW.md @@ -9,7 +9,8 @@ and native desktop validation remain pending; the broad security gate stays open Recorded 2026-10-13 against commit `4e60038c2f3b0a7b1d2dd0f4361309a735ee140f`. This is an advisory inventory and dependency-path review, not a complete security -assessment or a cleared production release gate. No dependencies were changed. +assessment or a cleared production release gate. The initial inventory changed +no dependencies; the subsequent compatible backport is documented below. ## Method and reproducible inputs diff --git a/docs/MEDIA-QUALITY.md b/docs/MEDIA-QUALITY.md index 51470e1..8901ce0 100644 --- a/docs/MEDIA-QUALITY.md +++ b/docs/MEDIA-QUALITY.md @@ -92,6 +92,41 @@ Measured hybrid p95 was **146.83 ms** on this run. [Recorded output](../fixtures ## Scoring and release gates +### Local recognized-speech supplement: actual Google diagnostic + +A subsequent internal experiment transcribed the unchanged NASA, piano, violin, +bark and roaring-effect sources locally, then ranked actual recognized passages +with Google EmbeddingGemma 2 on CPU, one thread. The base candidate supplied all +30 valid segments from all five sources; music descriptions and hallucinated +words remained competitors. Exact cosine ranking over source-preserving Google +tokenizer windows, with a maximum per original segment, found the right file for +**8/8** frozen NASA queries, but achieved **4/8 strict timestamp start hits** and +**6/8 segment-overlap hits**. The Moon statement was merged into a segment starting +at zero; the orange-soil exchange was missing. Funding and Mars passage starts +passed their independent intervals. + +This is a direct recognized-text retrieval diagnostic, **not the production +ANN/hybrid raw-audio pipeline**. The older production 0/8 point-hit result and +these 4/8 hits are not equivalent head-to-head measurements and cannot establish +a deployed improvement. The tiny paired English/German query set addresses four +topics in one English recording; German speech and general ASR quality remain +unverified. No model default, calibrated confidence or improved rating follows. + +The small candidate's Google diagnostic completed but excluded two invalid ASR +timestamp windows (piano tail and bark), marking +`quality_score_eligible=false`. Its reduced competitor pool receives **no +accepted numeric quality score**. The base score report SHA is +`8f92c8eeed41db7e78be0e188500e441d53b4fa49fa8c0353a77df74643ce895`; +actual Google ranking reports are SHA +`2fbc860f992c3ec561f54137e30744b00265455792e360010ab1fc78ab25173a` +(base) and +`aca99d36b3bdf013b1a1074c33f7232de1c1b79af62d00236d53267ce510c9c1` +(small). Raw recognized text remains ignored. See +[speech alignment evidence](SPEECH-ALIGNMENT.md) for worker/model provenance, +corrected decoder validation, measured transcription cost and explicit failures. +Production integration, speech gating/alignment and an equivalent pipeline rerun +remain open; the experimental media self-rating stays **4/10**. + Report recall@1/@5, MRR, language/type breakdowns, segment-overlap hits, precise timestamp point hits, and failure categories. No-answer return rate is distinct from positive recall. Do not invent an arbitrary cosine rejection threshold from these results; calibrate on a separate development set, then validate on held-out negatives. Self-rating rubric is declared before scores: **1–2/10** broken core retrieval; **3–4/10** experimental retrieval with missing essential capabilities or unvalidated abstention/metadata; **5–6/10** useful limited beta after independent basic retrieval/filter/metadata checks; **7–8/10** strong pilot with high recall and timestamp quality on representative corpora; **9–10/10** requires production release evidence across all operating systems, large representative libraries, calibrated no-answer behavior, and complete lifecycle/security/resource verification. A small diagnostic alone cannot justify a production score. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 74e0249..b929118 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -2,6 +2,11 @@ Status: implementation started; no production release yet. Windows VM tests use CPU only. +Karl's release priority, 2026-10-13: publish a usable first Windows version and +install it on DEV-WIN11 now. Extensive platform/GPU, clean-machine, signing and +large-library checks are deferred follow-up, not blockers for this initial +release. Keep their evidence and unchecked production gates intact. + ## Required product scope (user clarification) - Desktop UI is primary; semantic setup/indexing/search must require no terminal. diff --git a/docs/SPEECH-ALIGNMENT.md b/docs/SPEECH-ALIGNMENT.md index fffd097..577ac3b 100644 --- a/docs/SPEECH-ALIGNMENT.md +++ b/docs/SPEECH-ALIGNMENT.md @@ -174,9 +174,10 @@ transcripts, worker errors and inferred content never become executable instruct - [ ] Validate normal package dependency closure and missing-driver CPU fallback on each native OS; execute physical GPU tests outside this Windows VM. -Next implementation slice: isolated worker + manifest verification + genuine NASA -proof. Keep the initial runtime experiment separate from schema/ranking changes -until it establishes that recognized segments actually fix the measured problem. +The runtime and genuine retrieval diagnostics below inform future work; they do +not add a production speech feature. Further ASR/VAD research is deferred while +the usable Windows desktop takes priority. Revisit gating/alignment before schema +or ranking changes that would expose unreliable recognized speech to users. ## Actual internal CPU diagnostic, 2026-10-13 @@ -221,7 +222,150 @@ timestamp output. They do not establish speech recognition quality. Historical raw/checkpoint hashes and invalid classification remain under ignored `.searchi/speech/invalid-truncated-audio.json`; model bytes and actual raw outputs are excluded from Git. The owned sequential session 20331 is terminal with exit 1 -following intentional invalid-experiment termination. Corrected base/small -inference is paused pending the coordinator's CPU slot, followed by independent -caption review and genuine Google transcript ranking. Neither candidate is an -accepted product default; the media quality rating remains 4/10. +following intentional invalid-experiment termination. A subsequent corrected run +also detected a source-timeline issue: NASA's container ends at 85.333333 seconds, +but its primary audio ends at 85.262 seconds. Session 6691 failed closed on the +72 ms tail discrepancy, preserving its completed checkpoint. The harness now +uses the actual primary audio stream's duration/start/index, absolute seeking +with `-seek_timestamp 1`, and explicit stream mapping. Missing stream duration +is an actionable error; container duration is never silently substituted. + +Twelve tests passed in 1.937 seconds after that correction, including actual +FFmpeg mux/decode with a longer video, audio offset 0.125 seconds and container +offset five seconds. The audio timeline was 5.125–36.125 seconds; its one-second +tail decoded 15997 samples, within the unchanged one-millisecond rounding bound. +The zero-start 44.1/48 kHz tests still decoded exactly 480000/16000 samples. + +## Corrected base candidate evidence + +The corrected base evaluation completed all five assets with zero timestamp or +sample-coverage validation failures. Report SHA: +`9e33984afac9480452f506070dee0156fcc8d7f0d6882c8f5feeeaf5cae57de2`. +Executed frozen harness SHA: +`16e29bb70d3e1eeaad1f5b8ec223bff1a6f359215bcf8dabe52512a451855346`. + +| Source | Windows | Recognized segments | ASR process seconds | Decode seconds | +| --- | ---: | ---: | ---: | ---: | +| NASA | 3 | 19 | 23.322 | 0.542 | +| Piano | 9 | 9 | 210.720 | 2.167 | +| Violin | 1 | 1 | 4.471 | 0.212 | +| Bark | 1 | 0 | 4.373 | 0.113 | +| Roaring effect | 1 | 1 | 4.732 | 0.111 | + +These are observed serial CPU process times including model startup, not warmed +throughput or a controlled hardware comparison. NASA's decoded windows contained +480000, 480000 and 404192 samples, covering the actual primary audio range. +Manual comparison with publisher captions finds the Artemis funding and Mars +statements recognized, but the initial Moon-choice statement is merged into a +segment starting at zero and the orange-soil exchange is omitted. Recognition +does not prove Google ranking or a precise timestamp success rate. + +The instrumental piano emitted an unrelated water phrase and repetitive numbers +or punctuation in several windows. Music descriptions also appear. These must +remain competing retrieved candidates in the next diagnostic; filtering them +by knowledge of labels would hide failure. The source's published instrumental +description supports a hallucination concern, but independent audio listening +remains open. The current report field `quality_failed=false` means only that +structural timestamp/sample validation passed; **it does not certify speech or +search quality**. No model default is accepted. + +## Corrected small candidate evidence + +Owned sequential session 89200 finished with exit zero after evaluating all five +assets, base then small. Both used the same frozen corrected harness and verified +model bytes. The complete small report SHA is +`1660c95c71c9b8b1cb3b77f8f1ec422fb0954f2aacd2c16ea773901c0cfbe099`. + +| Source | Windows | Valid recognized segments | ASR process seconds | Decode seconds | +| --- | ---: | ---: | ---: | ---: | +| NASA | 3 | 19 | 86.250 | 0.601 | +| Piano | 9 | 8 | 1176.426 | 2.399 | +| Violin | 1 | 1 | 16.045 | 0.162 | +| Bark | 1 | 0 | 16.701 | 0.112 | +| Roaring effect | 1 | 1 | 25.082 | 0.113 | + +Small emitted repetitive inferred words in non-English languages on the +instrumental piano, with substantial CPU cost. Two actual timestamp outputs +failed validation: the 14.454-second piano tail returned a 0–30-second segment, +and the 2.039-second bark window returned a 0–2.5-second segment. Decoded sample +counts were correct (231254 and 32618 respectively); these are ASR timestamp +failures, not the historical resampling bug. Raw JSON is retained; no timestamp +was clamped. The report's `complete=true` establishes complete evaluation, +while `quality_failed=true` records failed structural validation. + +NASA's small recognition retained funding and Mars passages but misrecognized +the orange-soil words and still produced coarse segment boundaries. No numeric +timestamp retrieval improvement follows from caption inspection. The small +ranker may retain diagnostic results, but its reduced valid-window pool must be +`quality_score_eligible=false`; the scorer refuses an accepted numeric quality +score for incomplete or reduced competitor pools. Thirteen harness regressions, +including real FFmpeg coverage/timeline checks and this scoring gate, passed in +the coordinator's verified-FFmpeg run (2.100 seconds). + +Final process inspection found no owned ASR/runner processes; the CPU slot was +released for genuine Google transcript ranking. The current run removed its +temporary decoded WAV/JSON. An older scratch directory from intentionally stopped +invalid session 20331 remains ignored: automatic approval review rejected native +PowerShell recursive cleanup with only “blocked by policy” as its reason. No +alternate deletion method was attempted. + +Neither ASR model is an accepted product default. Production Google transcript integration, +speech activity gating, tighter alignment, hallucination control, German speech, +resource scheduling and native distribution still require evidence. Publish raw +Google media search before completing derived transcription so slow ASR cannot +block initial semantic readiness. Transcript publication needs its own bounded +completed-source work and atomic eligibility update. The production media rating +remains 4/10. + +## Actual Google transcript retrieval diagnostic + +The coordinator's owned session 2984 completed base and small ranking with exit +zero. The shared real Google adapter ran on CPU, one thread: document encoding +used the normal indexing role (2048-token context), then the document process was +dropped before opening the query role (512 tokens). Both reported +`google/embeddinggemma-2`, 768 dimensions and `b11461-no-logits-v1`. The diagnostic +runner was a Debug build with SHA +`be096c6dfe158b991520c060ca83478bdd0cd5c34d64fab4a2413d8b01de8f15`; +the actual native Google worker SHA was +`c01cb3ec13cfca722ee768e8350e1399a8b9a4d14a24a35adc3473a5cfc82a73`. + +The base experiment ranked all **30 valid recognized segments from all five +sources**, preserving music/sound descriptions and hallucinated text as +competitors. Actual Google tokenizer windows preserved original UTF-8 source +ranges. Segment ranking used exact cosine and maximum over its token windows; +the frozen labels did not select or rewrite candidates. The eight unchanged +English/German NASA queries produced: + +| Metric | Result | +| --- | ---: | +| Correct file at rank one | 8/8 | +| Strict timestamp start hit at rank one | 4/8 | +| Segment overlap at rank one | 6/8 | + +Both Moon-choice queries selected the merged segment starting at zero, before +the labelled 1.551–3.603-second utterance. Both orange-soil queries selected an +unrelated passage starting at 24.320 seconds because the recognized exchange +was missing. Funding queries selected 43.920 seconds and Mars queries selected +60 seconds, within the independently frozen intervals. Correct-file success +does not establish precise spoken-passage localization. + +The complete base Google report SHA is +`2fbc860f992c3ec561f54137e30744b00265455792e360010ab1fc78ab25173a`; +the accepted diagnostic scoring artifact SHA is +`8f92c8eeed41db7e78be0e188500e441d53b4fa49fa8c0353a77df74643ce895`. +Raw artifacts remain ignored under `.searchi/speech`. This is **direct exact-cosine +retrieval of recognized text**, not production ANN/hybrid retrieval of raw audio. +The historical production 0/8 audio-window result is useful motivation but is +not an equivalent head-to-head experiment; no production improvement or score +change is claimed. Eight correlated queries about four topics in one English +video do not validate German speech, held-out relevance or abstention. + +The small diagnostic also completed, with report SHA +`aca99d36b3bdf013b1a1074c33f7232de1c1b79af62d00236d53267ce510c9c1`. +Its two invalid ASR windows (piano tail and bark) were explicitly excluded: +`complete=true`, `quality_score_eligible=false`, `excluded_window_count=2`. +Diagnostic ranks are retained, but **no accepted numeric quality score** is +published for that reduced competitor pool. Neither model is adopted. Next steps +are speech gating/alignment proof, representative negative controls, actual +production integration and an equivalent shared-pipeline rerun; the media rating +remains 4/10. diff --git a/docs/SPEECH-VAD.md b/docs/SPEECH-VAD.md new file mode 100644 index 0000000..272c52d --- /dev/null +++ b/docs/SPEECH-VAD.md @@ -0,0 +1,147 @@ +# Speech gating and timestamp research + +Research recorded 2026-10-13 against whisper.cpp **v1.9.5**, exact commit +`d1be6fde11ac6e0407606b4e42fe72d34add8037`. No VAD inference, alignment +experiment, product integration or default change has been performed. The +corrected ungated ASR observations remain in [speech alignment](SPEECH-ALIGNMENT.md). +The instrumental piano's large CPU cost and unrelated text motivate testing; +they do not establish that any particular VAD model fixes the problem. + +## Frozen baseline parameters + +Use the pinned implementation's defaults first, before looking at new results. +Record these values explicitly in an experiment manifest rather than trusting +future upstream defaults: + +| VAD option | Pinned default | +| --- | ---: | +| Enabled | false | +| Threshold | 0.5 | +| Minimum speech | 250 ms | +| Minimum silence for splitting | 100 ms | +| Maximum speech | `FLT_MAX` (no practical limit) | +| Speech padding | 30 ms | +| Samples overlap | 0.1 seconds | + +Pinned CLI options include `--vad`, `--vad-model`, `--vad-threshold`, +`--vad-min-speech-duration-ms`, `--vad-min-silence-duration-ms`, +`--vad-max-speech-duration-s`, `--vad-speech-pad-ms`, and +`--vad-samples-overlap`. Detection is a speech probability decision, not a +transcription confidence threshold. [Pinned CLI source](https://github.com/ggml-org/whisper.cpp/blob/d1be6fde11ac6e0407606b4e42fe72d34add8037/examples/cli/cli.cpp#L107) + +Relevant decoding defaults are temperature 0, temperature increment 0.2, +entropy threshold 2.4, log-probability threshold -1, no-speech threshold 0.6, +word timestamp threshold 0.01, unlimited segment character length, and flash +attention enabled. The greedy/beam default uses five candidates. Preserve the +ungated experiment's explicit language `auto`, one processor and CPU policy. +Changing decoding parameters together with VAD would confound the comparison. +These are inspected source defaults, not a recommended tuned policy. + +## Two implementation traps + +The integrated VAD path creates `whisper_vad_context_params` using the library +default: **four threads, GPU disabled**. It does not copy the transcription +`--threads` value into `n_threads`. Therefore `whisper-cli --threads 1 --vad` +does **not** prove a one-thread VAD budget. Source evidence is +`src/whisper.cpp:4588` and `:6826`. The standalone +`examples/vad-speech-segments/speech.cpp:110` explicitly copies its own CLI +thread count. Prefer that VAD-only path or a small, separately hashed diagnostic +wrapper that sets `n_threads=1` and `use_gpu=false` before loading VAD. +[Pinned library implementation](https://github.com/ggml-org/whisper.cpp/blob/d1be6fde11ac6e0407606b4e42fe72d34add8037/src/whisper.cpp#L4588) + +The pinned full-JSON serializer reads raw token-data `t0/t1`; the dedicated +`whisper_full_get_token_t0/t1` accessors apply the VAD-to-original timeline map. +Their implementations differ. Segment timestamps use mapped accessors, so valid +segment offsets do not validate token offsets. Inspect CLI lines 765–789 and +library lines 8249–8318. Treat integrated-VAD token JSON as unsafe for source +seeking until a regression proves correct mapping or the diagnostic serializer +uses mapped accessors while preserving merged UTF-8 tokens. + +## Candidate assets, verified without inference + +The upstream downloader lists both Silero v5.1.2 and v6.2.0. The pinned README +uses v6.2.0; choose it as the **first preregistered candidate**, not because of +NASA results. Both exports were downloaded into ignored +`.searchi/speech/vad-research/` and locally verified against Hugging Face LFS +metadata. Repository revision: `9ffd54a1e1ee413ddf265af9913beaf518d1639b`. +[Pinned export files](https://huggingface.co/ggml-org/whisper-vad/tree/9ffd54a1e1ee413ddf265af9913beaf518d1639b) + +| Export | Bytes | SHA-256 | +| --- | ---: | --- | +| `ggml-silero-v5.1.2.bin` | 885098 | `29940d98d42b91fbd05ce489f3ecf7c72f0a42f027e4875919a28fb4c04ea2cf` | +| `ggml-silero-v6.2.0.bin` | 885098 | `2aa269b785eeb53a82983a20501ddf7c1d9c48e33ab63a41391ac6c9f7fb6987` | + +Download via `https://huggingface.co/ggml-org/whisper-vad/resolve/` +followed by the exact revision and filename, never floating `main`. +The export model card declares MIT. Silero's upstream MIT license was preserved +from commit `1e261b036686cd0017d500ee96acd1c4ba572a9d`: 1075 bytes, +SHA-256 `2e63e9a38b6e8fc0c7bc37ce174caca1862870856c6daf5697cfb785e925520b`. +The card does not provide a reproducible original-weight-to-export commit chain; +do not claim one. Distribution needs the actual license notice and a reviewed +manifest, beyond this research cache. +[Export model card](https://huggingface.co/ggml-org/whisper-vad), +[upstream license](https://github.com/snakers4/silero-vad/blob/1e261b036686cd0017d500ee96acd1c4ba572a9d/LICENSE) + +## Word and token timestamps + +`--output-json-full` enables token timestamps and emits token text, probability, +offsets and `t_dtw`. Tokens are subword units, potentially merged for valid UTF-8; +they are not guaranteed to be words. `--max-len` and `--split-on-word` change +segment boundaries but do not prove alignment accuracy. The upstream describes +word timestamps as experimental. +[Pinned timestamp documentation](https://github.com/ggml-org/whisper.cpp/blob/d1be6fde11ac6e0407606b4e42fe72d34add8037/README.md#word-level-timestamp-experimental) + +`--dtw base` and `--dtw small` select model-specific alignment presets. They +require `--no-flash-attn`: otherwise this pinned library warns and disables DTW. +The default DTW memory budget is 128 MiB. Neither token time nor DTW repairs an +omitted or hallucinated word. Measure ordinary full JSON first, then DTW as a +separate preregistered experiment; retain process flags and actual enabled-state +logs. Do not call either forced alignment or a precise word-seeking guarantee. + +## Proposed experiment and integration sequence + +1. Build a CPU-only **diagnostic** VAD-only worker from the exact pinned source, + with one explicit thread. Test timestamp-unit conversion: VAD accessor/example + output is centiseconds, convert to milliseconds exactly once. Verify source + offsets, padding, overlapping intervals and 30-second boundary behavior. +2. Freeze the v6.2.0 asset and the default parameter table above before inference. + Run detection over every existing speech/music/sound competitor, preserving + all decisions and probabilities. Record no-speech completion separately from + error; zero detected speech means no ASR process, not a successful transcript. +3. Feed detected **original contiguous intervals** to unchanged ASR, retaining + their absolute source offset. Avoid concatenating disjoint speech intervals + for the first experiment; that needs a piecewise timestamp map. Deduplicate + padded overlap by source identity/times, never relevance labels. No manually + filtered piano phrases, repetitive numbers or other convenient exclusions. +4. Compare complete ungated and gated pools using the same eight unchanged NASA + queries and genuine Google text retrieval. Report VAD/decode/model-load/ASR/ + tokenization/Google/ANN/publication times separately. Any processing failure + invalidates the comparison rather than shrinking the competitor pool. +5. Freeze an independent media corpus before selecting any production policy: + quiet/distant speech, accents, languages, very short utterances, speech over + instrumental music, singing, silence, synthetic effects and longer recordings. + Use independent listening/transcripts and publisher captions where possible. + Measure missed speech, false speech, hallucinated text, recognition accuracy, + timestamp error, retrieval, CPU cost and memory. Tune only on calibration; + preserve unseen held-out sources and queries. + +VAD may classify instruments or singing as speech and may reject quiet words or +short phrases under the 250-ms default. That can reduce relevant retrieval recall +even while removing bad candidates. A music speech gate must never disable +Google's original audio embeddings: music/sound/image/video retrieval stays +available independently of transcription. Expected efficiency is a hypothesis; +no local speedup or false-positive rate has been measured. + +If later evidence warrants production ASR, make it a resumable, bounded, +low-priority enrichment stage. Publish verified filename/text/Google media search +before transcription; retain prepared readers/query caches during enrichment. +Reuse the shared inference admission budget and yield between bounded source +intervals. Track parser/source SHA and revision through decoding, recognition +and publication; reject stale output. Persist provenance and explicit completion +or no-speech status. Foreground search must remain usable while ASR is pending; +an already executing encode cannot be claimed preemptible on a one-thread budget. + +Open gates: native VAD build, actual one-thread audit, detection/recognition +comparison, ordinary and DTW timestamp regression, independent held-out quality, +background starvation/cancellation/crash recovery, packaging/license review, +native OS and physical GPU fallback. None is closed by this research document. diff --git a/docs/WINDOWS-FIRST-RELEASE.md b/docs/WINDOWS-FIRST-RELEASE.md new file mode 100644 index 0000000..f0d2636 --- /dev/null +++ b/docs/WINDOWS-FIRST-RELEASE.md @@ -0,0 +1,125 @@ +# First usable Windows build + +Status: first Windows installer and portable package built and locally installed; +unsigned initial release, not production certification. +Other-platform and million-record benchmarks are deferred at Karl's request. + +The normal Windows x64 distribution includes the same hardware-detecting native +Google worker, optimized/baseline CPU modules, Vulkan provider and loader, and +app-local Microsoft C++ runtime. It is not a special CPU distribution. VM +inference and WebView rendering checks force CPU/software operation. Real GPU +execution and clean-machine/older-CPU compatibility remain unexecuted. + +## Build procedure + +The checked-in Tauri bundle setting remains unchanged. An ignored temporary +override enables NSIS and the silent WebView2 download bootstrapper: + +```json +{"bundle":{"active":true,"windows":{"webviewInstallMode":{"type":"downloadBootstrapper","silent":true}}}} +``` + +From `apps/desktop`, with one Cargo job: + +```powershell +$env:CARGO_BUILD_JOBS = '1' +$env:SEARCHI_FORCE_CPU = '1' +$env:WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS = '--disable-gpu --disable-accelerated-video-decode' +npm run tauri -- build --features custom-protocol --bundles nsis --config ../../.searchi/qa/windows-first-release-config.json +``` + +The explicit `custom-protocol` feature is essential: this CLI invocation did +not add it automatically, and the first attempt correctly failed the release +guard before packaging. The successful artifact must contain the embedded +frontend rather than depend on a development server. + +The portable ZIP is prepared from that same executable using +`python tools/package_windows.py`, which verifies normal CPU/GPU dependency +inventory, architecture, pinned decoder provenance and file hashes. Models, +personal settings and indexes are excluded. Managed Google assets download on +first use; cached, hash-verified assets are reused. + +## Evidence to record before publication + +- [x] Completed embedded-frontend Release build and source/frontend/executable hashes. +- [x] Portable ZIP member integrity and pinned decoder startup. +- [x] Extracted normal worker auto-to-CPU query and baseline-only fallback. +- [x] Extracted app starts outside the source checkout with isolated data and + model cache, and genuine local Google readiness/index generation. +- [x] Installer bytes/hash and generated resource mapping inspected. +- [ ] Execute installation/upgrade/uninstallation on a clean Windows machine. + +## Executed Windows VM evidence, 2026-10-13 + +Source commit: `5eeba29`. The embedded frontend bundle is +`index-CtE-o2qW.js`, SHA-256 +`deb3cdbeb4c4d34f17c0249674b37f7fd5cbdf7a01fef50c76d2eb50870817c5`. +Release compilation completed in 1m58s after dependencies were built. + +| Artifact | Bytes | SHA-256 | +| --- | ---: | --- | +| Portable ZIP | 145,419,558 | `dbab40f77902a5183d986222fadd8417aa46f053ee6066d73a06e7f893692060` | +| Embedded-frontend executable | 23,950,336 | `9f9243e15dc7fc907c44af37e45226a24eed642a32e5dc82959fab92516ff564` | +| NSIS x64 installer | 98,894,969 | `1494372cfd9b87dd4de38fd347fe0d18f886a51542e2f49218699aa84979bb3d` | + +The ZIP contains 39 individually hash-verified application/resource files. No +models, user settings or indexes are included. The normal worker SHA remains +`c01cb3ec13cfca722ee768e8350e1399a8b9a4d14a24a35adc3473a5cfc82a73`. + +The real extracted worker ran its query role with 512 tokens, `device=auto` +and a child-only disabled Vulkan provider. It selected CPU, loaded no Vulkan +module and returned a finite normalized 768-dimensional Google code-query +vector. Optimized `ggml-cpu-cascadelake.dll` and baseline-only `ggml-cpu-x64.dll` +both passed, including a Unicode relocation directory. Startup/query/exit took +1.722s and 1.844s respectively; these are not warm-search or first-UI-input +benchmarks. The packaged pinned FFprobe helper also started successfully. + +The extracted desktop launched outside the source tree with isolated settings, +data and hardlinked hash-verified Google assets. Two owned text/code fixtures +were indexed through the shared engine. The app automatically published actual +Google generation 1 with two chunks and a 6,544-byte ANN blob; publication was +observed 10.106s after launch. It remained alive for another five seconds. The +owned application process tree was then deliberately terminated. This proves +native process/setup/indexing operation, not visual interaction or paint timing. + +Ignored raw evidence: `.searchi/qa/windows-first-preview-worker.json`, +`.searchi/qa/windows-first-preview-baseline.json`, and +`.searchi/packages/first-preview-qa-Gemäß_日本/result.json`. + +The initial NSIS attempt failed after its official ZIP was downloaded and hash +verified, during tool extraction (`access denied`). A bundle-only retry uses +the cached compiler and preserves the verified executable bytes with +`--no-binary-patching`; it does not disable OS security or clear caches. +That retry completed successfully with official NSIS 3.11. Its generated +script installs app-local worker/CPU/GPU/decoder resources and downloads the +Microsoft WebView2 bootstrapper when needed. The produced installer is +`apps/desktop/src-tauri/target/release/bundle/nsis/searchi_0.1.0_x64-setup.exe`. +No installer was executed during this evidence run: even a custom destination +registers uninstall information and creates user shortcuts. User installation +is handled separately. There is no signing, clean-machine WebView2 download, +upgrade or uninstall claim. + +Native visual interaction, fresh-machine WebView2 acquisition, installer +upgrade/uninstall, SmartScreen/signing and actual GPU performance remain +separate checks. A working preview does not close those production release gates. + +## Local DEV-WIN11 installation + +The installer was executed on the existing Windows 11 development machine. +Installation with an explicit `C:/Users/karl/Applications/searchi` destination +returned exit code zero; uninstall registration reports version 0.1.0 at that +location. The installed executable SHA matches the packaged executable above. + +An explicit destination avoids this Codex MSIX host's AppData filesystem +virtualization. Existing local searchi data was preserved in an independent +`C:/Users/karl/.searchi/desktop` profile; both genuine Google assets were +hash-verified and hardlinked into the independent managed cache. No user folder +was selected automatically. The desktop launcher sets only process-local CPU +and software-WebView flags for this VM; the distributed package remains the +normal hardware-detecting build. + +The installed app was started with a native `searchi` window (PID 7716, window +handle 8324222) and its packaged query worker (PID 19864), using one CPU thread +and the independent cache. A desktop shortcut was created. This demonstrates +local installation/startup and package selection, not a clean-machine test or +native visual interaction audit. The app is left running for Karl to use. diff --git a/tools/speech-eval/README.md b/tools/speech-eval/README.md index 5f8c9ca..1e5cc1e 100644 --- a/tools/speech-eval/README.md +++ b/tools/speech-eval/README.md @@ -25,6 +25,13 @@ model load, not retained-session throughput. This deliberately simple experiment does not implement overlap/dedup or production budgets. Segment JSON offsets are already milliseconds and get the window offset added exactly once. +Windows follow the probed primary audio stream's start/duration/index, with +explicit absolute `-seek_timestamp 1` seeking and stream mapping. Missing audio +stream duration is rejected with an actionable error. Container duration cannot +silently extend a shorter audio stream. Resampling occurs before sample trimming; +decoded frames must match the planned duration within at most one millisecond. +Every actual frame count and full probe JSON is retained in the report. + Each completed window writes an incomplete checkpoint; the final report marks complete evaluation, with `quality_failed` and explicit `validation_failures` for invalid ASR timestamps. Bounded raw ASR JSON remains in each checkpoint before @@ -42,6 +49,8 @@ the existing eight frozen NASA intervals unchanged. Input has `fixture_sha256`, `asr_report_sha256`, `google_worker_sha256`, and `rankings` mapping query ids to ordered `{asset,start_ms,end_ms}` records. It reports top1 overlap and strict start point hits; it never chooses a segment by looking at the answer intervals. +The ranking report must have `complete=true`, `quality_score_eligible=true` and +no excluded windows; a reduced competitor pool cannot receive an accepted score. Hash fields must be checked against retained artifacts by the evaluator; they are provenance declarations, not a replacement for inspecting those artifacts.