searchi/docs/INCREMENTAL-SEMANTIC-VALIDATION.md

Completed-source reuse validation

2026-10-13. Windows VM, CPU-only inference (SEARCHI_FORCE_CPU=1), normal hardware-detecting worker, pinned Google EmbeddingGemma 2 backbone/projector, one inference thread. FFmpeg/ffprobe are the verified app-local helpers.

Genuine Google integration

cargo test -p searchi-service --test incremental_google_cpu --test google_cpu --locked -- --ignored --nocapture --test-threads=1 executes actual models. A counting adapter forwards every operation to GemmaEmbedder; no synthetic vectors satisfy the integration assertions.

• Document and code initial encoding: two calls. Unchanged rebuild and folder move: zero extra calls, bit-identical vectors, remapped current paths/locations. A basename rename adds one call because the document title is in Google's prompt. A single content change adds one call; unrelated code stays identical.
• Mixed document/code/image/audio/video fixture: five source groups. Unchanged rebuild makes zero additional document/image/audio encoder calls and retains exact vector bytes and citations. Replacing the BMP while preserving its size and exact mtime adds one image call; the other four groups stay identical.
• Rejected activation retains the prior active generation. Reopening resumes completed changed-document vectors with zero document calls; updated text citations are returned after publication. Existing policy-change regression covers lock admission, generation creation, repeated activation failures, same-instance reader policy and resumed retry in both directions.

Synthetic colors/sine waves in media fixtures prove reuse, not media relevance. Tests are correctness checks, not latency or throughput benchmarks.

Storage and failure boundaries

Four unit regressions cover schema-6 legacy search without unsafe reuse, completed versus partial restart state, revision-checked publication, transactional completion failure and malformed stored text windows. Another fault-only pipeline test covers primary inference error preservation when partial-source cleanup also fails. Its provider returns no simulated vectors.

Executed checks

• Windows workspace: 98 passed, 15 explicitly ignored; no failures.
• Native Ubuntu 24.04 headless workspace: 98 passed, 15 explicitly ignored.
• Windows native desktop unit suite: 17 passed.
• Strict Windows/Linux workspace and Windows native-desktop Clippy passed; Windows workspace/desktop formatting passed.
• Final-source Windows real Google tests above: five passed. The two existing lifecycle/policy tests took 25.31 s and the three incremental tests took 138.90 s. Concurrent compilation/resource contention was present; these elapsed suite times are not application search benchmarks.
• Final-source native Linux actual Google integration: the same five tests passed; existing lifecycle/policy suite 49.77 s, incremental suite 173.59 s. Ubuntu 24.04 WSL x86-64 with two allocated CPUs, one inference thread, isolated verified models/normal auto worker and pinned native FFmpeg helpers. Cold Clippy compilation ran concurrently; these are not latency comparisons.

Remaining gates

Actual process termination during a multi-batch source, simultaneous real-model source mutation, wider policy changes, near-limit pruning, minimum-ABI/older-CPU checks and first-query/foreground latency during indexing remain open. Full file hash validation currently precedes ANN construction; direct file mutation during that construction before watcher reconciliation needs a later validation callback. See INCREMENTAL-SEMANTIC.md.