feat: add persistent CPU search engine and CLI with lifecycle tests

crates/searchi-core/README.md

@@ -0,0 +1,49 @@
+# Searchi core
+
+The core has no desktop or inference dependencies. Its current retrieval engine
+combines literal filename matching with SQLite FTS5 BM25 keyword search. It does
+not implement semantic search.
+
+`SearchIndex::open` creates or migrates schema version 2. Current-schema opens do
+not acquire a write lock. WAL allows independent readers to use the committed
+index while reconciliation runs. All indexing changes are committed atomically;
+a traversal or supported-file read error rolls back the entire reconciliation.
+
+`index_folder` applies `IndexOptions::default()`: directories named `.git`,
+`node_modules`, and `target` are pruned. `index_folder_with_options` lets callers
+replace those names. Matching directory names are ASCII case insensitive. An
+explicit root is always traversed, even if its name is excluded. Symlinks are not
+followed. The index's own database, WAL, and shared-memory files are excluded.
+
+Metadata includes byte size and modification time at nanosecond resolution.
+Unchanged files bypass extraction. `IndexReport.indexed` counts newly extracted
+or updated records; `unchanged` counts reused records; `skipped` counts new or
+changed records whose content could not be extracted because of format, encoding,
+or size. Unsupported files remain searchable by filename. Content reads are
+limited to 2 MiB. Reconciliation tracks seen files in SQLite rather than retaining
+every path in process memory.
+
+`search_with_options` supports canonical folder boundaries, extension, inclusive
+size bounds, and inclusive Unix millisecond modification-time bounds. Filters
+apply before limiting results. `search` retains the unfiltered API. Query text is
+escaped as literal FTS terms, not interpreted as operators. Results are capped at
+1,000 and queries at 4,096 bytes.
+
+## Known limits
+
+- Filename substring lookup uses `instr` and scans metadata. A dedicated substring
+  index and benchmarks are needed before claiming million-file latency targets.
+- Files changed without changing size or modification time require a future
+  forced refresh/content-fingerprint option. Timestamp precision depends on the
+  filesystem. Indexing does not yet run a filesystem watcher.
+- Metadata modification times before the Unix epoch or beyond the nanosecond
+  integer range are explicit errors. Unicode filename case folding is currently
+  limited by SQLite's built-in `lower` function.
+- Lossy path conversion does not preserve non-Unicode native filenames. This must
+  be addressed before Linux production support for arbitrary byte filenames.
+- The `files.root` ownership model does not yet support overlapping selected
+  roots independently. Use disjoint roots until root memberships are implemented.
+- Extraction runs in the caller and a large reconciliation holds one transaction.
+  Bounded worker isolation, persistent jobs, and crash-resumable scans remain
+  production milestones.
+- PDF, DOCX, OCR, media extraction, and real model inference remain unimplemented.