101 lines
5.6 KiB
Markdown
101 lines
5.6 KiB
Markdown
# Process recovery and prepared-reader handoff
|
|
|
|
2026-10-13 development validation. Production release gates remain open.
|
|
|
|
## Abrupt process termination
|
|
|
|
`crates/searchi-cli/tests/crash_recovery.rs` invokes the real installed-layout
|
|
CLI adapter with verified Google EmbeddingGemma 2 assets, one CPU thread and
|
|
actual FFmpeg helpers. It creates two document/code source groups and a
|
|
65-second PCM WAV with three normal audio windows. It polls SQLite for both
|
|
completed document groups plus a committed, unowned chunk belonging to the
|
|
running media task, then terminates only its owned process tree.
|
|
|
|
Windows uses PID-scoped `taskkill /T /F`; Unix assigns a dedicated process group
|
|
and kills that group. Cleanup does not kill processes by name and does not panic
|
|
during test unwinding. No user app/settings/index is used.
|
|
|
|
Assertions verify that the previous active generation and completed vector
|
|
bytes remain intact. Restart resumes the same generation, makes zero document
|
|
encoder calls, re-encodes all three windows of the unfinished audio source,
|
|
and publishes five unique chunks without leaked partial vectors. A real
|
|
Google CodeRetrieval query checks the current source line citation.
|
|
|
|
Windows and native Linux actual runs passed in 157.86 s and 233.65 s. These are suite elapsed times with compiler
|
|
contention, not a search benchmark. Power loss, arbitrary batch boundaries and
|
|
all crash scenarios are not established by one process-termination case.
|
|
|
|
## Prepared graphs and warmed query model
|
|
|
|
The desktop already used separate background and query model workers, but
|
|
publication could send the next foreground request through blob reading,
|
|
SHA verification and ANN deserialization. It also withdrew readiness and
|
|
repeated query warmup while holding the retained session mutex.
|
|
|
|
The publisher now shares its immutable prepared `Arc<usearch::Index>` before
|
|
SQL activation through a process-local `PreparedGraphs` handle. That handle
|
|
binds to the canonical database and vector dimension; entries match generation,
|
|
full policy fingerprint and persisted ANN checksum. At most two registry entries
|
|
are retained. Failed SQL activation removes only its unpublished entry; readers
|
|
retain their own Arc when an old registry entry is evicted.
|
|
|
|
The desktop query worker keeps its model and query-vector LRU ready throughout
|
|
publication. Detailed traces include `semantic.prepared_graph_lookup`; the new
|
|
active graph is a cache hit instead of foreground blob loading. Existing source
|
|
eligibility, strict filtering and generation snapshot checks still apply.
|
|
|
|
Storage regressions cover identity/dimension/checksum mismatches, bounded
|
|
eviction, retained-reader lifetime and SQL activation rejection/retry. The real
|
|
Google CPU test drops the background worker, searches the updated document and
|
|
asserts no blob-read/checksum/deserialization spans. It verifies both a retained
|
|
query-vector hit and an unseen query with actual neural inference.
|
|
|
|
Windows one-document fixture: retained query 4.389 ms, unseen query 111.086 ms,
|
|
29.05 s suite elapsed. These measure the shared engine request in debug mode,
|
|
not native UI first input, application startup, a representative corpus or p95.
|
|
The same native Linux fixture passed: retained query 2.107 ms, unseen query
|
|
79.614 ms, suite 21.98 s. Different cache/contention conditions prevent treating
|
|
these numbers as Windows-versus-Linux comparative performance.
|
|
|
|
## Windows package media provenance
|
|
|
|
The packager now verifies the pinned original decoder ZIP size and checksum,
|
|
then compares the staged manifest, helpers and notices with bounded archive
|
|
members. Valid PE headers and a freshly recorded package checksum are not
|
|
accepted as decoder provenance. Fifteen packaging tests cover changed helpers,
|
|
notices, manifests, archive members and changes during ZIP writing; failure
|
|
preserves the previous output. Media staging was refreshed from the verified
|
|
cached archive before packaging.
|
|
|
|
Current embedded-frontend Windows release built successfully. Its 145,265,448-byte unsigned ZIP
|
|
contains 39 verified files; optimized and baseline-only packaged CPU inference
|
|
and decoder startup passed. Archive SHA-256:
|
|
`cf7953827be90255f182932fe53c16faf76701098d2636c8860b6ed4e0eb6ec4`.
|
|
Both worker smoke tests request auto with the GPU provider disabled only in the
|
|
child process. Actual GPU, native UI interaction and clean-machine installation
|
|
are not certified by those smoke tests.
|
|
|
|
## Remaining performance/release work
|
|
|
|
Background and query workers still receive independent thread budgets; shared
|
|
CPU admission and foreground priority are open. Publication no longer charges
|
|
foreground graph deserialization, but eligibility refresh, indexing I/O and
|
|
CPU contention need representative measurements. Native compositor/input timing,
|
|
large-corpus p95, older CPUs, actual GPU/device-loss, power-loss durability,
|
|
installers/upgrades/signatures and macOS packaging remain open.
|
|
|
|
## Executed checks
|
|
|
|
- Windows and native Linux workspace tests: 106 passed each, 18 ignored.
|
|
- Windows native desktop unit suite: 17 passed.
|
|
- Strict Windows/Linux workspace and Windows desktop Clippy plus both formatting checks passed.
|
|
- Real Google CPU process-kill and prepared-reader regressions passed on both OSes.
|
|
- Windows packager tests: 15 passed. Current embedded-frontend release build,
|
|
archive integrity, packaged optimized/baseline CPU inference and decoder startup
|
|
passed; no accelerator was used.
|
|
|
|
The prepared graph handle is process-local: foreign CLI/MCP index writers cannot
|
|
share its native Arc. Cold CLI initialization and cross-process reader refresh
|
|
remain separate costs. Representative concurrent-indexing/foreground measurements
|
|
and shared CPU thread admission remain the next performance milestone.
|