127 lines
7.2 KiB
Markdown
127 lines
7.2 KiB
Markdown
# Process recovery and prepared-reader handoff
|
|
|
|
2026-10-13 development validation. Production release gates remain open.
|
|
|
|
## Abrupt process termination
|
|
|
|
`crates/searchi-cli/tests/crash_recovery.rs` invokes the real installed-layout
|
|
CLI adapter with verified Google EmbeddingGemma 2 assets, one CPU thread and
|
|
actual FFmpeg helpers. It creates two document/code source groups and a
|
|
65-second PCM WAV with three normal audio windows. It polls SQLite for both
|
|
completed document groups plus a committed, unowned chunk belonging to the
|
|
running media task, then terminates only its owned process tree.
|
|
|
|
Windows uses PID-scoped `taskkill /T /F`; Unix assigns a dedicated process group
|
|
and kills that group. Cleanup does not kill processes by name and does not panic
|
|
during test unwinding. No user app/settings/index is used.
|
|
|
|
Assertions verify that the previous active generation and completed vector
|
|
bytes remain intact. Restart resumes the same generation, makes zero document
|
|
encoder calls, re-encodes all three windows of the unfinished audio source,
|
|
and publishes five unique chunks without leaked partial vectors. A real
|
|
Google CodeRetrieval query checks the current source line citation.
|
|
|
|
Windows and native Linux actual runs passed in 157.86 s and 233.65 s. These are suite elapsed times with compiler
|
|
contention, not a search benchmark. Power loss, arbitrary batch boundaries and
|
|
all crash scenarios are not established by one process-termination case.
|
|
|
|
## Prepared graphs and warmed query model
|
|
|
|
The desktop already used separate background and query model workers, but
|
|
publication could send the next foreground request through blob reading,
|
|
SHA verification and ANN deserialization. It also withdrew readiness and
|
|
repeated query warmup while holding the retained session mutex.
|
|
|
|
The publisher now shares its immutable prepared `Arc<usearch::Index>` before
|
|
SQL activation through a process-local `PreparedGraphs` handle. That handle
|
|
binds to the canonical database and vector dimension; entries match generation,
|
|
full policy fingerprint and persisted ANN checksum. At most two registry entries
|
|
are retained. Failed SQL activation removes only its unpublished entry; readers
|
|
retain their own Arc when an old registry entry is evicted.
|
|
|
|
The desktop query worker keeps its model and query-vector LRU ready throughout
|
|
publication. Detailed traces include `semantic.prepared_graph_lookup`; the new
|
|
active graph is a cache hit instead of foreground blob loading. Existing source
|
|
eligibility, strict filtering and generation snapshot checks still apply.
|
|
|
|
Storage regressions cover identity/dimension/checksum mismatches, bounded
|
|
eviction, retained-reader lifetime and SQL activation rejection/retry. The real
|
|
Google CPU test drops the background worker, searches the updated document and
|
|
asserts no blob-read/checksum/deserialization spans. It verifies both a retained
|
|
query-vector hit and an unseen query with actual neural inference.
|
|
|
|
Windows one-document fixture: retained query 4.389 ms, unseen query 111.086 ms,
|
|
29.05 s suite elapsed. These measure the shared engine request in debug mode,
|
|
not native UI first input, application startup, a representative corpus or p95.
|
|
The same native Linux fixture passed: retained query 2.107 ms, unseen query
|
|
79.614 ms, suite 21.98 s. Different cache/contention conditions prevent treating
|
|
these numbers as Windows-versus-Linux comparative performance.
|
|
|
|
## Windows package media provenance
|
|
|
|
The packager now verifies the pinned original decoder ZIP size and checksum,
|
|
then compares the staged manifest, helpers and notices with bounded archive
|
|
members. Valid PE headers and a freshly recorded package checksum are not
|
|
accepted as decoder provenance. Fifteen packaging tests cover changed helpers,
|
|
notices, manifests, archive members and changes during ZIP writing; failure
|
|
preserves the previous output. Media staging was refreshed from the verified
|
|
cached archive before packaging.
|
|
|
|
Current embedded-frontend Windows release built successfully. Its 145,265,448-byte unsigned ZIP
|
|
contains 39 verified files; optimized and baseline-only packaged CPU inference
|
|
and decoder startup passed. Archive SHA-256:
|
|
`cf7953827be90255f182932fe53c16faf76701098d2636c8860b6ed4e0eb6ec4`.
|
|
Both worker smoke tests request auto with the GPU provider disabled only in the
|
|
child process. Actual GPU, native UI interaction and clean-machine installation
|
|
are not certified by those smoke tests.
|
|
|
|
## Remaining performance/release work
|
|
|
|
Background and query workers now share inference admission and foreground
|
|
priority; see [PERFORMANCE.md](PERFORMANCE.md#shared-inference-admission).
|
|
Publication no longer charges
|
|
foreground graph deserialization, but eligibility refresh, indexing I/O and
|
|
CPU contention need representative measurements. Native compositor/input timing,
|
|
large-corpus p95, older CPUs, actual GPU/device-loss, power-loss durability,
|
|
installers/upgrades/signatures and macOS packaging remain open.
|
|
|
|
## Executed checks
|
|
|
|
The scheduling change at `d85746d` passed 110 Windows workspace tests (19
|
|
explicitly ignored), 20 desktop unit tests and 39 frontend tests. Workspace and
|
|
desktop strict Clippy, Rust formatting, frontend production build and Prettier
|
|
checks passed. The real Google concurrent-inference regression passed again
|
|
after scoped-thread cleanup; its measured scope is documented in
|
|
[PERFORMANCE.md](PERFORMANCE.md#shared-inference-admission).
|
|
|
|
The updated Windows embedded-frontend Release executable is 23,523,328 bytes,
|
|
SHA-256 `6efc81b653c7bd37ecbe63bbe538b126440daa5b62419dd1994c4a3696799d52`.
|
|
Its unsigned development ZIP is 145,262,551 bytes, SHA-256
|
|
`c1859d6ead7628f02965568e1c509cd52484e22ec397a86f62bcea108487de16`.
|
|
The extracted archive passed integrity checks for 39 files, genuine 768-dimensional
|
|
normalized Google code-query inference and decoder startup. It requested automatic
|
|
device selection with Vulkan disabled only in the smoke-test child, used the
|
|
optimized CPU module and loaded no Vulkan provider. This does not verify the
|
|
updated desktop UI, clean-machine installation or actual GPU behavior.
|
|
|
|
Five targeted scheduler tests also passed, including lowering admission below
|
|
a session's worker weight: rejection invokes no worker, clears prior diagnostics,
|
|
closes the failure trace and releases priority reservations. This is scheduler
|
|
failure testing; semantic quality is covered by the separate real Google fixture.
|
|
|
|
The following checks describe the earlier prepared-reader/recovery milestone,
|
|
not the updated Linux desktop or updated Windows release artifacts:
|
|
|
|
- Windows and native Linux workspace tests: 106 passed each, 18 ignored.
|
|
- Windows native desktop unit suite: 17 passed.
|
|
- Strict Windows/Linux workspace and Windows desktop Clippy plus both formatting checks passed.
|
|
- Real Google CPU process-kill and prepared-reader regressions passed on both OSes.
|
|
- Windows packager tests: 15 passed. Current embedded-frontend release build,
|
|
archive integrity, packaged optimized/baseline CPU inference and decoder startup
|
|
passed; no accelerator was used.
|
|
|
|
The prepared graph handle is process-local: foreign CLI/MCP index writers cannot
|
|
share its native Arc. Cold CLI initialization and cross-process reader refresh
|
|
remain separate costs. Representative concurrent-indexing/foreground measurements
|
|
across decoder, extraction and ANN work remain the next performance milestone.
|