searchi/docs/ARCHITECTURE.md

# Architecture and boundaries

The Rust engine owns storage and search. Its callers do not construct SQL.
The CLI and desktop shell share the engine; the Tauri bridge runs filesystem
work outside the UI thread using independent database connections.

The current lexical index is a foundation. It uses SQLite WAL and FTS5 with
exact-name priority and source snippets. Reconciliation is transactional:
incomplete traversal must not delete previously indexed files. A root that
cannot be read produces an error, not an empty successful reconciliation.

The next storage design separates metadata discovery, persistent extraction
jobs and passage publication. Bounded workers publish small batches so search
can see newly indexed names before a complete root finishes. A generation
marker only removes stale entries after a successful walk. Watchers enqueue
changes; reconciliation catches lost events. Overlapping roots need explicit
ownership before root removal is implemented.

Semantic inference is a replaceable CPU adapter with explicit model artifact
download, pinned checksums, normalization and versioned chunk vectors. Search
must never download a model implicitly. No synthetic embeddings qualify as
semantic search. An exact lexical match keeps priority during hybrid fusion.

Document extractors eventually run in isolated workers with input size,
timeout and output limits. PDF pages, code lines and media timestamps are
stored as source locations, not inferred by the frontend.

Indexes contain private extracted text. No telemetry or remote search is
enabled. Model downloads and release checks must be explicit operations.

## Known foundation limits

- Substring filename search scans the metadata table; the million-file target
  requires an additional substring index and measured query planning.
- Initial indexing publishes after the root transaction finishes.
- File extraction covers bounded UTF-8, PDF text and DOCX paragraphs. OCR and
  media extraction remain upcoming work; parser workers are not isolated yet.
- No automatic watcher, persistent extraction queue or ANN index yet.
- Native CI and signed package validation require actual platform runners.