searchi/docs/VALIDATION.md

# Validation evidence

Development host: Windows VM, x86_64 MSVC, stable Rust 1.99.0. All local
inference, FFmpeg decoding and WebView tests use CPU only. UI launches set
`WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS='--disable-gpu --disable-accelerated-video-decode'`
and `SEARCHI_FORCE_CPU=1`.

## Current integration checkpoint (2026-10-12)

The subsequent rebuild-policy fix passes the Windows workspace suite (93
passed, 12 explicitly ignored) and strict service Clippy. A separately executed
genuine Google CPU regression verifies that the same reader retains its
supplier-contract result after lock, begin, activation and cleanup failures,
in both text-only and media-policy directions, and that retry succeeds.
SQLite triggers inject storage failures; actual Google document vectors supply
the searchable generation. No synthetic relevance vectors or media decoding
are used in that test. The additional ignored test accounts for the changed
default-suite ignore count.

The reliability checkpoint passes 93 workspace tests on Windows and 93 on
native Linux (Ubuntu 24.04 under WSL); each explicitly ignores 11 optional
integration tests. Linux uses its native Rust toolchain and an isolated ext4
build cache. This verifies headless core portability, not a Linux desktop,
Linux native Google worker, GPU or installer. Windows desktop passes 17 tests;
strict workspace and desktop Clippy checks pass. Five genuine Google model
integration tests pass in forced-CPU mode.

Worker IPC now applies one deadline across sending and receiving, with bounded
queues and owned writer/reader threads. Transport failures terminate and reap
the worker; eligible GPU transport failures retry on CPU. Invalid protocol
responses remain errors. Fifteen semantic unit tests pass, including a blocked
pipe and disconnected writer; one optional integration test is ignored.

Resource staging passes 24 tests on Windows (one symlink-permission skip) and
23 on Linux (two Windows-only skips). Locking, recovery, rollback preservation,
link rejection and concurrent edits before the final swap are covered. Actual
automatic-backend staging also succeeds. These are process-failure checks;
power-loss durability and concurrent arbitrary writes after the final check
remain release gates.

The latest unsigned Windows archive launches outside the checkout with an
embedded frontend at `http://tauri.localhost/`, isolated data, one real indexed
document and a ready Google CPU semantic generation. The archive is
145,217,175 bytes, SHA-256
`fb454a9dbf258f51044796ab24a5942a8f90804d0824ca0e2dc5f59b7847ec3b`.
Native accessibility confirms startup and status only. Input and screenshot
capture remain denied by the environment; native interaction, appearance,
clean-machine installation and GPU behavior are unverified. Release builds
without the embedded-frontend feature now fail explicitly.

The following package-dispatch evidence predates this reliability checkpoint:

The normal Windows x64 package now includes baseline and optimized CPU DLLs
with CPU/OS feature checks, plus the Vulkan backend. Shared DLLs use a common
MSVC CRT: a first `/MT` prototype crashed because model file handles crossed
separate CRT instances; the corrected `/MD` package passes real-model startup.
All 25 runtime DLLs and the worker have their non-Windows PE imports satisfied
by packaged files. Windows system UCRT/WinRT and WebView2 remain platform
requirements. This is dependency inspection, not clean-machine certification.

Both extracted-archive tests passed: automatic CPU dispatch selected
`ggml-cpu-cascadelake.dll`, and removing optimized modules from the temporary
extraction selected `ggml-cpu-x64.dll`. Both produced genuine normalized 768d
Google code vectors, started FFprobe, verified all 39 listed files, and confirmed
the Vulkan module was not loaded. The unsigned archive is 145,196,878 bytes;
its digest is recorded in [dispatch evidence](evidence/windows-package-cpu-dispatch.json)
and [baseline evidence](evidence/windows-package-cpu-baseline.json).
Recorded startup/query/exit times are diagnostic smoke samples, not comparable
benchmarks: cache state and concurrent checks differ. Baseline inference on
this modern VM does not certify older physical CPUs or operating systems.
Three builder, five staging and eleven archive regression tests passed.
Two real Google code tests and two text-reference/tail/debug parity tests also
passed using the new shared worker in forced-CPU mode. Text reference cosine
similarities range from 0.99983106 to 0.99993036; debug/normal query cosine is
1.00000000. These verify runtime equivalence, not broad retrieval quality.
The genuine Google media parity test also passed on this shared CPU runtime:
image budgets 280/140/70 and the speech fixture all meet the 0.999 reference
cosine threshold. Image/audio quality limitations and GPU gates remain open.

The following evidence describes the earlier static worker checkpoint:

The Windows-x64 universal worker built with CPU and Vulkan enabled, using the
pinned official LunarG 1.4.363.0 SDK (download hash and publisher signature
verified). Copy-only installation and later provenance/swap validation ran in
the workspace; no driver/registry installation or GPU inference occurred.
The desktop embedded-frontend Release build completed and was rebuilt with
startup fallback. The generated unsigned Windows archive includes its worker,
app-local Vulkan loader, FFmpeg/FFprobe and named notices. Six packaging
regressions verify exact archived-byte hashes, changes during copying, notices,
architecture mismatch, failed-write cleanup and preservation of previous output.

The actual extracted package's worker was requested in `auto` mode, with
`SEARCHI_FORCE_CPU` absent and Vulkan disabled only in that child. It detected
a CPU-only backend registry and produced a real normalized 768d code vector;
its decoder started successfully. All 13 recorded package files passed hash
checks. [Raw package evidence](evidence/windows-package-cpu-fallback.json).
The final local archive is 140,971,051 bytes; its digest is included in that
evidence. It has not been published as a production release.
This is controlled no-provider fallback, not a physical GPU/driver test or
native GUI/clean-machine certification. Actual GPU precision/parity, native
application interaction, signed installers and other OS/architecture gates stay open.
Two real code tests and existing Google text-reference/tail parity passed on
the GPU-capable worker's forced-CPU path. Four startup regressions check one
CPU retry after automatic startup transport failure while preserving invalid
protocol/model rejection and explicit-CPU behavior. Eight Vulkan-provisioning
tests verify cache provenance, bounds, atomic loader reuse and SDK swap rollback.
The integrated workspace and native desktop tests passed, along with strict
all-target Rust Clippy, frontend Release build and formatting checks. Gitea
Actions remain disabled at Karl's request.

Offline staging now verifies the native license and every bundled Google asset
before modifying resources. Two Python regressions verify missing-model rejection
and same-size corruption in the second model leaving the existing staged worker
untouched. Actual CPU worker staging passed. This preflight does not prove atomic
recovery from interruptions during the subsequent copies; that gate remains open.

Gitea Actions were disabled for **karl/searchi** at Karl's explicit request;
the API read-back confirmed `has_actions:false`. Fifteen old stuck runs and
the subsequent repair run were cancelled. Diagnosis found unavailable native
Windows/macOS runner labels and a document-pagination test assuming filesystem
traversal order. The test was corrected and its four local document tests plus
strict core Clippy passed. The Linux-only workflow repair was pushed, but its
verification run was cancelled when Karl requested disabling CI. It must not
be reported as a passing remote run. Local checks and remaining release gates
continue to apply; enabling Actions again requires Karl's explicit request.

- Native desktop: **15/15 tests passed**, including structured trace/basic responses
  and four optional-answer source/consent/revision tests.
  Post-generation cached query warmup is implemented but its interactive first-query gate remains open.
  Frontend: **39/39 tests passed**, TypeScript/Vite production build and
  formatting passed. Preview policy tests cover fractional timestamp seeking,
  image labels, one active/latest pending request, stale replies and Escape.
- Semantic adapter: **8 default tests passed**; the explicit 48 MP resource test
  remains opt-in. Strict targeted all-target Clippy and formatting passed.
  Protocol bounds, invalid vectors, artifact size/checksum mismatch, old-worker
  capability defaults and oversized JPEG rejection are covered.
- The CPU native helper rebuilt and staged successfully with genuine Google
  MTMD image/audio support. Its handshake reports `media_supported:true`;
  older text workers default false rather than claiming media readiness.
- Actual Google CPU media parity passed separately in **52.85 seconds**:
  screenshot 280 cosine 0.99994194, photo 140: 0.99974583, frame 70: 0.99921928 and
  speech: 0.99983318 against pinned MIT DigUp vectors. All exceed the 0.999 gate.
  This verifies runtime parity, not broad retrieval quality.
- The release resource test encodes/decodes an actual 8000x6000 JPEG with EXIF
  rotation and verifies region colors after scaled decoding/resizing. Decode
  plus resize took 176–220 ms. A sampled whole-test peak was 146.86 MiB, including
  source fixture generation; it is not decoder-only RSS or neural quality.
- The independent production media pipeline has run on 20 real assets/65 frozen
  queries using actual Google inference, FFmpeg, metadata, persistence and HNSW.
  It encoded 20/20 files with no failures in the recorded 30-second-window run.
  Broad file recall is promising; precise speech timestamps, music confusion
  and no-answer behavior remain failed quality gates. Preserve the earlier
  direct harness failure separately. [Raw results and limitations](MEDIA-QUALITY.md).
- Actual document seed regression passed 24/24 semantic/hybrid queries. This is
  a small synthetic regression, not a production relevance corpus.
- The explicit 100,000-vector release ANN diagnostic passed 98.5% recall@10,
  HNSW p95 1.37 ms and warm service p95 6.40 ms with zero warm passage hashes.
  Vectors are synthetic index benchmark data, never production embeddings.
- The fixed one-million-file first-input report records 40 inputs below 50 ms in
  that run: [recorded benchmark](../crates/searchi-core/benchmarks/results/windows-cpu-full-candidate-first-final.json).
  These measurements do not establish every query or concurrent-indexing goal.

The integrated CPU workspace checkpoint passed **78 default tests** (7 optional AI,
8 assets, 8 CLI/MCP, 24 core, 4 media, 8 semantic, 19 service), with **10 explicit
opt-in tests ignored**. Workspace and native all-target Clippy passed with
warnings denied. The later semantic CPU-restart counter change passed workspace
Clippy and the actual CPU query diagnostics test. Earlier aggregate counts are historical.

The updated pinned C++ helper compiled in an independent CPU build directory and
was staged for desktop resources without interrupting media evaluation. Actual
Google query diagnostics passed separately in 18.09 seconds including model
initialization and two queries: 13 tokens, tokenization 149 us, batch preparation
22 us, inference 112914 us, pooling/normalization 17 us. Rust worker wait/response
decode measured 113960 us. Debug/normal embedding cosine was 0.99999999. These
are one diagnostic sample under VM contention, not production latency evidence.
See [performance boundaries and raw overhead samples](PERFORMANCE.md).

## Native UI and platform limits

The branded launcher launched natively. Accessibility inspection showed real
controls and a warmed Google CPU worker. Native screenshots, input automation
and complete image/audio/video playback interactions remain blocked by Windows
access/capture errors recorded in [design QA](../design-qa.md). Passing pure
preview policy tests does not establish real WebView codec playback or visual
fidelity. Default-application opening is not exact timestamp navigation.

Windows x86-64 verified decoder provisioning/staging has executed. Manifest
entries for Linux and ARM64 are configuration, not evidence of native execution.
A vetted macOS decoder package remains missing. Linux/macOS worker/desktop
execution, production GPU parity/fallback, installer signing, upgrade/rollback,
licenses/source distribution, large libraries and calibrated absence handling
remain unexecuted release gates. FFmpeg subprocess memory is not sandbox-capped;
bounded outputs/timeouts are not an absolute RSS guarantee.

Gitea needs matching registered runners. GitHub workflows are supplied for
mirroring; they do not run merely because code is pushed to Gitea. The current
workflow checks do not automatically certify downloaded Google assets or the
separately built worker. CI configuration is not remote job evidence.

## README capture evidence (2026-10-12)

The README now includes four visually inspected browser captures of the
production frontend over a real, retained Google CPU engine: light/dark PDF
search, image search and granular performance details. The independent capture
backend passed three tests and strict Clippy. Live capture assertions verified
the first PDF result and page 2, its notice clause, loaded photo pixels, and
actual worker inference counters. All captures use an owned headless browser
with CPU flags and licensed/generated fixture content; no recorded responses
are replayed. These checks supplement native desktop QA, which remains open.
Literal highlights are checked in result snippets and the actual preview text.
The small match explanation is closed by default and can be opened to inspect
the real page location. Media placeholder text receives no invented word marks.
[Reproduction](../tools/desktop-capture/README.md),
[runtime/query/image evidence](screenshots/capture-manifest.json).

## Agent and optional-answer evidence (2026-10-12)

The official Rust MCP SDK serves three read-only stdio tools: search, stats and
opaque-handle source retrieval. Two actual executable tests verify initialization,
tool discovery, real SQLite results, structured output, invalid inputs, UNC/device
filter rejection before filesystem resolution, source modification/deletion and
reindex invalidation. One-operation admission, 64 KiB input frames and a 256-handle
cache bound the adapter. Its agent skill passed the Skill Creator validator.

A separate actual Google CPU MCP smoke check used the existing seven-file README
fixture index. Hybrid search selected Supplier Agreement.pdf, page 2, returned its
sixty-day notice passage through read_source, and selected Mountain Lake.jpg for
the image-description query. The observed warm document engine time was 69.80 ms
in one run; this small debug fixture is not a production benchmark or p95 claim.

Seven actual loopback HTTP tests verify the optional OpenAI-compatible adapter's
request path/messages, header-only authentication, pure offline validation,
input/response bounds, real timeout behavior, citation consistency and sanitized
errors. Replies are controlled HTTP fixtures; no hosted provider or live summary
model was invoked. Native tests verify that the shown excerpt equals the outgoing
text, filename/path/credential exclusion, disabled/unknown/stale-source rejection
and preparation revision handling. Frontend tests cover disabled defaults,
configuration validation, byte budgets and stale-answer state.

The desktop's optional answers are separate from search, disabled by default and
explicitly requested. Config and keys are session-only; model output is plain text
and labeled as a draft. Answers currently cover one reviewed document excerpt.
Native window interaction, live-provider compatibility/quality/latency, Linux/macOS
agent execution and release packaging remain open gates.

## Historical evidence

The source-code checkpoint uses the actual rebuilt/staged CPU Google worker:
9/9 frozen code queries rank their expected source file first (1/9 with lexical
retrieval alone), with exact MCP source excerpts and exclusion of a matching
ordinary document. This is a tiny synthetic regression, not a general code
benchmark. [Protocol and raw results](CODE-QUALITY.md).
Workspace checks passed 84 default tests; model/decoder/resource checks remain
opt-in. Separate actual CPU code tests verify task/debug vector behavior and
normalized-tokenizer source windows, including long Unicode code. The capture
backend's three tests, frontend build/format and strict Rust Clippy also pass.
Existing real Google text-reference and long-tail parity passed after the
tokenizer fix; regular document search retains its pinned reference behavior.
Native/GPU/platform and broad independent code-quality gates remain open.

The [EmbeddingGemma 2 audit](EMBEDDINGGEMMA2-AUDIT.md) records new CPU diagnostics:
24 uncached native queries per 1/2/4-thread setting, paired real MCP repeat/filter
measurements, 20 unchanged result-set comparisons, actual Release startup/first
query, and document single-versus-serial-batch parity. Six cache-control tests
verify eviction, fingerprint/trim behavior, failures, unchanged filtering/source
lifecycle and accurate hit traces. The frontend readiness regression retains an
old committed generation during rebuilds and blocks initial/activation states.
The four real-engine browser captures were regenerated and visually inspected;
the performance capture deliberately uses an unseen query and proves actual
worker inference rather than treating a cached vector as a new neural pass.
Native visual/paint and production media relevance gates remain open.

Initial engine/CLI checks covered retrieval, persistence, filesystem changes,
exclusions/filters, migration, concurrent readers, PDF pages, DOCX paragraphs,
corrupt replacement, encrypted PDF errors and bounded compressed input.
The first actual Google text adapter matched five pinned reference vectors
above 0.999, including multilingual queries and long-document window coverage.

The rejected MiniLM/ONNX experiment achieved 5/6 top-1 on its six-query fixture;
it is retained under the opt-in historical proof only and is not a production
fallback. [Historical experiment](../tools/semantic-proof/HISTORICAL.md).
Repository metadata/avatar updates were read back successfully. Historical
foundation/default test totals changed as implementation expanded and must not
be presented as the current aggregate.

Reproduction commands and exact interfaces are in [BUILDING.md](BUILDING.md),
[SEMANTIC.md](SEMANTIC.md), and [MEDIA-QUALITY.md](MEDIA-QUALITY.md).