searchi/docs/ROADMAP.md

# Production readiness roadmap

Status: implementation started; no production release yet. Windows VM tests use CPU only.

## Required product scope (user clarification)

- Desktop UI is primary; semantic setup/indexing/search must require no terminal.
- Google EmbeddingGemma 2 is the required production model, matching DigUp.
- Local document, image, audio and video search are required product milestones.
- A separate CLI-only build supports headless Linux without desktop dependencies.
- Production automatically uses available hardware acceleration with CPU fallback;
  this Windows VM is tested in forced-CPU mode only.
- Installer ships the native worker. First launch automatically downloads/verifies
  missing model files and manages indexing in the UI; no terminal setup is required.
- Earlier MiniLM proof is an experiment and does not satisfy the model requirement.
- First interaction must be responsive: prepare indexes and a real Google query
  in the background, retain ready sessions, and show immediate keyword results.
  Model download and initialization must have honest progress and retry states.

## 1. Search foundation
- [x] Rust workspace, reusable engine, CLI, versioned SQLite storage.
- [ ] Filename/path and ranked full-text retrieval with strict filters.
- [ ] Safe folder discovery, exclusions, bounded extraction, passage locations.
- [ ] Incremental updates, rename/deletion reconciliation, crash recovery.
- [x] Persistence and filesystem lifecycle integration tests (Windows CPU).

## 2. Document and semantic retrieval
- [x] Text, Markdown, code, PDF and DOCX extraction with real fixtures (Windows CPU).
- [ ] Validate actual embedding model artifacts, license, CPU runtime and quantization.
- [ ] Versioned embeddings, ANN index, exact-match priority and hybrid fusion.
- [ ] English/German relevance corpus and measured top-10 recall.
- [ ] Atomic model/index replacement and resumable persistent work queues.

## 3. Desktop
- [ ] Tauri shell, accessible keyboard search, progressive results and previews (shell built; UI interaction verification pending).
- [ ] Folder selection, exclusion controls, resource budgets and indexing status.
- [ ] Global shortcut, opening/revealing source files, PDF page navigation.
- [ ] Watchers and reconciliation across Windows, macOS and Linux.

## 4. Media
- [x] Genuine Google image, audio and video embeddings integrated with the shared
  search service and CPU-tested FFmpeg decoding; music tags and duration filters.
- [x] Frozen independent media corpus with retrieval, timestamp, negative-query
  and indexing-cost measurements (see MEDIA-QUALITY.md).
- [ ] Improve precise spoken-passage localization, music hard negatives and
  calibrated handling of queries without a relevant match.
- [ ] Package and verify decoders on every release OS; test production acceleration.
- [ ] CPU OCR, image retrieval, audio transcription, video segment locations.
- [ ] Isolated extraction workers with timeouts and resource ceilings.

## 5. Release gates
- [x] Shared monotonic Basic/Debug search metrics, granular bounded engine traces,
  actual worker query diagnostics, and headless JSON performance output.
- [ ] Verify detailed performance UI interaction on native desktop platforms;
  repeat overhead measurements on representative libraries and hardware.
- [ ] Formatting, lint, tests and security/dependency review.
- [ ] CPU benchmarks: filename p95 <50 ms/1M records; text <150 ms/100K documents; semantic <500 ms on declared hardware.
- [ ] Measure cold start, indexing throughput, RAM, disk and idle CPU.
- [ ] Record the very first foreground query and distinct unseen queries after
  production background preparation; report initialization time separately.
  Warm p95 alone does not satisfy the first-use requirement.
- [ ] Test migrations, interrupted work, invalid documents, inaccessible/offline roots and cloud placeholders.
- [ ] Automated native build matrix and documented supported cross-compilation paths.
- [ ] Execute Linux/macOS platform tests; create Windows/macOS/Linux installers.
- [ ] Signing/notarization configuration, upgrade/rollback validation, release instructions.

Completion requires recorded evidence; targets are not measured results.

## Completed foundation details

- SQLite FTS5 with exact filename priority, text snippets and strict filters.
- Default directory exclusions and protection against indexing the index itself.
- Metadata reuse, atomic root reconciliation and schema-v1 migration coverage.
- Actual CLI process integration tests and independent-reader concurrency test.
- TypeScript desktop frontend build and dependency audit passed; native validation underway.
- Initial README commit pushed to Gitea main via protected Infisical injection.
- Native desktop build and guarded opening tests passed on Windows.
- Genuine multilingual CPU model proof: 5/6 top-1, 6/6 recall@3; quality gate remains open.
- Native Rust CPU embedding adapter verifies pinned model/tokenizer assets;
  integration with the search index and platform runtime packaging remain open.
- PDF page and DOCX paragraph matches persist; corrupt/encrypted/over-limit
  documents remain filename-searchable and report extraction errors.
- CLI exposes folder/type/size/date filters and additional directory exclusions.

The partial features above do not close broader production milestones such as
crash-resumable jobs, fuzzy retrieval, watcher reliability or performance targets.