karl/searchi
1
0 Fork 0
Code Issues Pull Requests Packages Projects Releases 10 Wiki Activity

feat: add multimodal search and granular performance diagnostics
Core CI / core (ubuntu-latest) (push) Failing after 3m31s
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
Core CI / desktop (ubuntu-latest) (push) Successful in 5m39s
Core CI / desktop (macos-latest) (push) Canceled after 0s
Core CI / desktop (windows-latest) (push) Canceled after 0s

Browse Source
This commit is contained in:
karl committed 2026-10-12 19:16:23 +02:00
1 parent d3bb8c7618
commit b8400c4fbf
110 files changed
+79395 -250
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+118 -63
View File
@@ -2,89 +2,144 @@
**Find what matters. Locally.**
Local desktop search for Windows, macOS and Linux. Under active development;
production readiness is tracked in [the roadmap](docs/ROADMAP.md).
Local desktop search for Windows, macOS and Linux, under active development.
Production readiness is tracked in [the roadmap](docs/ROADMAP.md).
## Architecture
## What works
`crates/searchi-core` owns persistent indexing and retrieval. `crates/searchi-cli`
provides a headless interface. `apps/desktop` hosts the Tauri desktop app.
Search combines indexed filenames, SQLite FTS5 keyword retrieval, and real
Google EmbeddingGemma 2 semantic search with a persistent HNSW index. PDF pages
and DOCX paragraphs retain source locations. Image, audio, and video search remain
planned; the provisioned media projector is not yet used. Granola is not required.
The desktop UI is primary; the CLI builds separately without GUI dependencies.
Search combines instant indexed filenames, SQLite FTS5 content retrieval, and
**real Google EmbeddingGemma 2** vectors in a persisted HNSW index. PDF pages,
DOCX paragraphs and text passages retain their source locations. The same
shared Rust service now indexes **images, video frames, movie soundtracks,
audio and music**: actual pixels and sound enter Google's media projector in
the same 768-dimensional space as text queries. No generated captions,
simulated embeddings, replacement model, or Granola service is required.
Existing artist/title tags contribute keyword matches.
Production uses supported hardware acceleration with CPU fallback. All testing
in the Windows development VM uses CPU only. Indexed content stays
local. Treat the index as sensitive: it contains extracted document text.
The desktop is the primary interface. It automatically prepares verified local
models, indexes selected folders, and prewarms its query worker. Image previews
and audio/video controls use narrowly scoped local assets; timestamp results
seek inside the preview without autoplay. Open file launches the default
application, whose exact opening position is not controlled by Searchi.
The CLI provides the same retrieval engine without a GUI.
The desktop automatically downloads and verifies missing Google artifacts
(approximately 865 MB), reuses its local cache, and warms the query session in
the background. Development builds must build and stage the native worker first;
production installers and signing remain release gates. First-query performance
is measured separately from repeated-query performance.
**Media search is experimental.** File retrieval works on the independent seed
corpus, but precise spoken-topic timestamps, music hard negatives, and no-answer
handling still need improvement. Nearest-neighbor results are not calibrated
confidence. Video frames are sampled across the movie; every fleeting scene is
not guaranteed. Audio windows cover the primary stream within explicit resource
limits. See [actual media quality and failures](docs/MEDIA-QUALITY.md).
Production supports compiled hardware acceleration with CPU fallback. All
Windows VM tests use CPU only. Files, extracted text, metadata and embeddings
stay local; treat the index as sensitive. Initial verified model provisioning
requires approximately 865 MB of downloads. Search performs no remote inference.
## Copy-and-paste setup prompt
Paste this into your coding agent:
```text
Set up Searchi from https://tea.karlcom.de/karl/searchi.git in a new searchi
directory, or reuse the existing checkout without overwriting local changes.
Read AGENTS.md, SKILLS.md, docs/ROADMAP.md, and apps/desktop/README.md first.
Inspect the OS and installed tools. Install missing supported prerequisites:
Git, stable Rust, Node.js 24+, Python 3, CMake, native C/C++ tools and Tauri's
platform dependencies. Preserve existing credentials and never print secrets.
From the repository root, build and stage the pinned Google runtime:
folder, or reuse the checkout without overwriting local changes. Read AGENTS.md,
SKILLS.md, docs/ROADMAP.md, docs/BUILDING.md and apps/desktop/README.md first.
Inspect the OS/tools. Install supported missing prerequisites: Git, stable Rust,
Node.js 24 LTS, Python 3.11+, CMake, native C/C++ tools and Tauri dependencies.
Preserve credentials and never print secrets. Prefer established Rust crates
and portable native tools; keep engine APIs shared by CLI and desktop.
For Karl's Windows VM set SEARCHI_FORCE_CPU=1 and
WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS='--disable-gpu --disable-accelerated-video-decode'
before tests or launch. Never enable a GPU provider in that VM.
From the repository root:
python tools/embeddinggemma2/build_runtime.py --backend cpu --jobs 2
python tools/embeddinggemma2/stage_runtime.py --backend cpu
Run cargo test --workspace --locked, cargo fmt --all -- --check, and
python tools/media-runtime/provision.py --stage
The decoder manifest supports Windows/Linux x86-64 and ARM64; macOS decoder
packaging is still a gate. On unsupported hosts report the real blocker and
use explicitly verified native tools only when their provenance is known.
Run cargo test --workspace --locked, cargo fmt --all --check and
cargo clippy --workspace --all-targets --locked -- -D warnings.
Run cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml --locked.
In apps/desktop run npm ci, npm test, npm run format:check, and npm run build.
For Karl's Windows VM set SEARCHI_FORCE_CPU=1 and
WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS=--disable-gpu before tests or launch.
Launch the real app using npm run tauri dev from apps/desktop. Allow automatic
verified Google model setup; never substitute another model or mock results.
Ask which folders I want indexed; do not index my whole home automatically.
Verify filename and remembered-content searches on my selected files, including
persistence after restart. Report actual checks and blockers. Do not publish a
release or claim production readiness, GPU validation, or untested OS coverage.
In apps/desktop run npm ci, npm test, npm run format:check and npm run build.
Launch the real app with npm run tauri -- dev. Allow automatic verified Google
model setup and retained-worker warmup. Never substitute another model or mocks.
Ask which folders I want indexed; don't index my whole home automatically.
Verify filename/content/image/music/video searches and persistence on selected
files. Inspect media failure reports and actual preview seeking. Read
MEDIA-QUALITY.md before claiming precise speech timestamps or reliable absence.
Report actual checks and blockers. Do not publish a release or claim production
readiness, GPU validation, native visual QA, or untested OS coverage.
```
This is developer setup, not an installer. See the [desktop guide](apps/desktop/README.md)
and [official Tauri prerequisites](https://v2.tauri.app/start/prerequisites/).
The worker builder also supports `auto`, `metal`, `vulkan`, and `cuda`; those
require platform SDKs and separate hardware validation.
This is developer setup, not an installer. The worker builder also accepts
`auto`, `metal`, `vulkan` and `cuda`; each requires its platform SDK and separate
hardware validation. Standalone FFmpeg/FFprobe and notices are staged for the
desktop; end users should not install tools or configure decoder paths.
## Development
## Headless setup
Install stable Rust and the platform's C/C++ build prerequisites. Run:
Build with `cargo build --release --locked -p searchi-cli`. No Node, Tauri,
WebView or display server is needed. Build the native Google worker separately,
and provision verified assets and FFmpeg tools:
```sh
cargo test --workspace
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
python tools/embeddinggemma2/build_runtime.py --backend cpu --jobs 2
python tools/media-runtime/provision.py
cargo run -p searchi-cli -- setup --model-dir .searchi/models/embeddinggemma2
cargo run -p searchi-cli -- index --db searchi.sqlite --root /path/to/files
```
On native Linux, unified document/media indexing and hybrid search use:
```sh
./target/release/searchi embed --db searchi.sqlite --model-dir .searchi/models/embeddinggemma2 --embedding-worker .searchi/runtime/cpu/searchi-gemma-worker --ffmpeg .searchi/media-runtime/bin/ffmpeg --ffprobe .searchi/media-runtime/bin/ffprobe --cpu-threads 1
./target/release/searchi search --db searchi.sqlite 'the recording about orange soil' --model-dir .searchi/models/embeddinggemma2 --embedding-worker .searchi/runtime/cpu/searchi-gemma-worker --cpu-threads 1
```
These are current interfaces, not executed Linux release evidence. Windows uses
`Release/searchi-gemma-worker.exe` and `ffmpeg.exe`/`ffprobe.exe` instead.
`embed` includes media by default and reports failures. Use `--text-only`
explicitly for document-only vectors. `setup` provisions Google assets, not
worker/decoder executables. Installed release CLIs discover decoders under
`media-runtime/bin` beside the CLI; explicit paired paths or
`SEARCHI_FFMPEG_PATH`/`SEARCHI_FFPROBE_PATH` override discovery. Search never
silently downloads missing helpers. CLI invocations load their own worker;
the desktop retains a prewarmed worker for interactive queries.
## Search performance
Normal searches show result counts, measured search time and time until results
are displayed. Enable **Performance details** in desktop settings to
inspect UI scheduling, native queue/lock time, keyword retrieval, actual Google
query inference, ANN/cache/filter work and ranking. Export the current trace as
local JSON for comparison; tracing sends no telemetry and excludes search text
and file paths. Nested stages and parallel retrieval lanes overlap.
For headless diagnostics, append `--performance --json` for basic metrics or
`--debug-performance --json` for the detailed response. Plain `--json` retains
the existing results-array format. See [timing boundaries, trace schema and
reproducible measurements](docs/PERFORMANCE.md).
## Development and release
The Rust engine, service, media adapter and CLI live under `crates/`; the Tauri
shell is `apps/desktop`. SQLite, image/jpeg-decoder, USearch/NumKong, and Tauri
provide established storage, decoding, ANN/SIMD and desktop facilities.
Extraction and inference use bounded isolated work with explicit errors.
```sh
cargo test --workspace --locked
cargo fmt --all --check
cargo clippy --workspace --all-targets --locked -- -D warnings
cargo run -p searchi-cli -- --help
```
Desktop development instructions live in `apps/desktop/README.md`.
Agent constraints and workflows live in [AGENTS.md](AGENTS.md) and
[SKILLS.md](SKILLS.md). Use Node.js 24+ for frontend state tests.
The approved visual identity and desktop direction live in [the brand guide](docs/BRAND.md).
See [desktop setup](apps/desktop/README.md), [build/platform constraints](docs/BUILDING.md),
[semantic bounds](docs/SEMANTIC.md), [executed validation](docs/VALIDATION.md),
and [brand direction](docs/BRAND.md). Native visual QA, operating-system builds,
macOS decoder packaging, installers, licensing distribution, signatures,
upgrades and representative production quality remain release gates.
Cross-platform source support does not mean signed macOS installers can be
built on Windows. CI declarations are not evidence of executed platform jobs.
For a headless build, run `cargo build --release --locked -p searchi-cli`.
No Node, Tauri, or WebView is needed. Semantic commands additionally need the
native worker and Google assets; `searchi setup` provisions the model cache.
Platform and CI details live in [the build guide](docs/BUILDING.md).
Native CI builds must pass on all supported systems before release. Cross-platform
source support does not imply signed macOS installers can be built on Windows.
## Release policy
Do not publish until the roadmap's release gates have evidence. Record benchmark
hardware, corpus size, cold/warm state and p95 latency. Signing credentials are
supplied through CI secrets, never source control.
Licensed under [MIT](LICENSE). Repository author: Karl.
Record benchmark hardware, corpus, cold/warm state and p95 latency. Never ship
signing credentials in source control. Licensed under [MIT](LICENSE).
Repository author: Karl.