feat: add persistent CPU search engine and CLI with lifecycle tests

docs/ARCHITECTURE.md

@@ -0,0 +1,38 @@
+# Architecture and boundaries
+
+The Rust engine owns storage and search. Its callers do not construct SQL.
+The CLI and desktop shell share the engine; the Tauri bridge runs filesystem
+work outside the UI thread using independent database connections.
+
+The current lexical index is a foundation. It uses SQLite WAL and FTS5 with
+exact-name priority and source snippets. Reconciliation is transactional:
+incomplete traversal must not delete previously indexed files. A root that
+cannot be read produces an error, not an empty successful reconciliation.
+
+The next storage design separates metadata discovery, persistent extraction
+jobs and passage publication. Bounded workers publish small batches so search
+can see newly indexed names before a complete root finishes. A generation
+marker only removes stale entries after a successful walk. Watchers enqueue
+changes; reconciliation catches lost events. Overlapping roots need explicit
+ownership before root removal is implemented.
+
+Semantic inference is a replaceable CPU adapter with explicit model artifact
+download, pinned checksums, normalization and versioned chunk vectors. Search
+must never download a model implicitly. No synthetic embeddings qualify as
+semantic search. An exact lexical match keeps priority during hybrid fusion.
+
+Document extractors eventually run in isolated workers with input size,
+timeout and output limits. PDF pages, code lines and media timestamps are
+stored as source locations, not inferred by the frontend.
+
+Indexes contain private extracted text. No telemetry or remote search is
+enabled. Model downloads and release checks must be explicit operations.
+
+## Known foundation limits
+
+- Substring filename search scans the metadata table; the million-file target
+  requires an additional substring index and measured query planning.
+- Initial indexing publishes after the root transaction finishes.
+- File extraction currently covers bounded UTF-8 text, not PDF/DOCX/media.
+- No automatic watcher, persistent extraction queue or ANN index yet.
+- Native CI and signed package validation require actual platform runners.

docs/ROADMAP.md

@@ -0,0 +1,50 @@
+# Production readiness roadmap
+
+Status: implementation started; no production release yet. Windows VM tests use CPU only.
+
+## 1. Search foundation
+- [x] Rust workspace, reusable engine, CLI, versioned SQLite storage.
+- [ ] Filename/path and ranked full-text retrieval with strict filters.
+- [ ] Safe folder discovery, exclusions, bounded extraction, passage locations.
+- [ ] Incremental updates, rename/deletion reconciliation, crash recovery.
+- [x] Persistence and filesystem lifecycle integration tests (Windows CPU).
+
+## 2. Document and semantic retrieval
+- [ ] Text, Markdown, code, PDF and DOCX extraction with fixtures.
+- [ ] Validate actual embedding model artifacts, license, CPU runtime and quantization.
+- [ ] Versioned embeddings, ANN index, exact-match priority and hybrid fusion.
+- [ ] English/German relevance corpus and measured top-10 recall.
+- [ ] Atomic model/index replacement and resumable persistent work queues.
+
+## 3. Desktop
+- [ ] Tauri shell, accessible keyboard search, progressive results and previews.
+- [ ] Folder selection, exclusion controls, resource budgets and indexing status.
+- [ ] Global shortcut, opening/revealing source files, PDF page navigation.
+- [ ] Watchers and reconciliation across Windows, macOS and Linux.
+
+## 4. Media
+- [ ] CPU OCR, image retrieval, audio transcription, video segment locations.
+- [ ] Isolated extraction workers with timeouts and resource ceilings.
+
+## 5. Release gates
+- [ ] Formatting, lint, tests and security/dependency review.
+- [ ] CPU benchmarks: filename p95 <50 ms/1M records; text <150 ms/100K documents; semantic <500 ms on declared hardware.
+- [ ] Measure cold start, indexing throughput, RAM, disk and idle CPU.
+- [ ] Test migrations, interrupted work, invalid documents, inaccessible/offline roots and cloud placeholders.
+- [ ] Automated native build matrix and documented supported cross-compilation paths.
+- [ ] Execute Linux/macOS platform tests; create Windows/macOS/Linux installers.
+- [ ] Signing/notarization configuration, upgrade/rollback validation, release instructions.
+
+Completion requires recorded evidence; targets are not measured results.
+
+## Completed foundation details
+
+- SQLite FTS5 with exact filename priority, text snippets and strict filters.
+- Default directory exclusions and protection against indexing the index itself.
+- Metadata reuse, atomic root reconciliation and schema-v1 migration coverage.
+- Actual CLI process integration tests and independent-reader concurrency test.
+- TypeScript desktop frontend build and dependency audit passed; native validation underway.
+- Initial README commit pushed to Gitea main via protected Infisical injection.
+
+The partial features above do not close broader production milestones such as
+crash-resumable jobs, fuzzy retrieval, watcher reliability or performance targets.

docs/VALIDATION.md

@@ -0,0 +1,18 @@
+# Validation evidence
+
+Development host: Windows VM, x86_64 MSVC, Rust stable 1.99.0.
+All local tests run on CPU; no GPU provider is configured.
+
+Initial foundation checks (2026-10-12):
+- `cargo test --workspace`: CLI parser/executable checks and engine retrieval,
+  lifecycle, exclusions, filters, migration and concurrent-reader checks passed.
+- `cargo clippy --workspace --all-targets -- -D warnings`: passed.
+- Rust formatting applied; final formatting verification follows integration.
+- Desktop TypeScript/Vite build passed; npm audit reported zero vulnerabilities.
+
+Not yet established: production latency targets, semantic inference, desktop
+runtime behavior, Linux/macOS execution, installer signing or upgrades.
+CI configuration is source configuration, not evidence that remote jobs ran.
+Gitea workflows require matching Windows, Linux and macOS runners; runner
+availability has not been established. GitHub workflows are also supplied for
+future mirroring, but do not run merely because code was pushed to Gitea.