karl
f77112db97
Core CI / core (ubuntu-latest) (push) Successful in 1m0s
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
39 lines
2.1 KiB
Markdown
39 lines
2.1 KiB
Markdown
# Architecture and boundaries
|
|
|
|
The Rust engine owns storage and search. Its callers do not construct SQL.
|
|
The CLI and desktop shell share the engine; the Tauri bridge runs filesystem
|
|
work outside the UI thread using independent database connections.
|
|
|
|
The current lexical index is a foundation. It uses SQLite WAL and FTS5 with
|
|
exact-name priority and source snippets. Reconciliation is transactional:
|
|
incomplete traversal must not delete previously indexed files. A root that
|
|
cannot be read produces an error, not an empty successful reconciliation.
|
|
|
|
The next storage design separates metadata discovery, persistent extraction
|
|
jobs and passage publication. Bounded workers publish small batches so search
|
|
can see newly indexed names before a complete root finishes. A generation
|
|
marker only removes stale entries after a successful walk. Watchers enqueue
|
|
changes; reconciliation catches lost events. Overlapping roots need explicit
|
|
ownership before root removal is implemented.
|
|
|
|
Semantic inference is a replaceable CPU adapter with explicit model artifact
|
|
download, pinned checksums, normalization and versioned chunk vectors. Search
|
|
must never download a model implicitly. No synthetic embeddings qualify as
|
|
semantic search. An exact lexical match keeps priority during hybrid fusion.
|
|
|
|
Document extractors eventually run in isolated workers with input size,
|
|
timeout and output limits. PDF pages, code lines and media timestamps are
|
|
stored as source locations, not inferred by the frontend.
|
|
|
|
Indexes contain private extracted text. No telemetry or remote search is
|
|
enabled. Model downloads and release checks must be explicit operations.
|
|
|
|
## Known foundation limits
|
|
|
|
- Substring filename search scans the metadata table; the million-file target
|
|
requires an additional substring index and measured query planning.
|
|
- Initial indexing publishes after the root transaction finishes.
|
|
- File extraction currently covers bounded UTF-8 text, not PDF/DOCX/media.
|
|
- No automatic watcher, persistent extraction queue or ANN index yet.
|
|
- Native CI and signed package validation require actual platform runners.
|