docs: rewrite README for searchi users
Core CI / core (ubuntu-latest) (push) Failing after 3m48s
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
Core CI / desktop (ubuntu-latest) (push) Successful in 5m53s
Core CI / desktop (macos-latest) (push) Canceled after 0s
Core CI / desktop (windows-latest) (push) Canceled after 0s
Core CI / core (ubuntu-latest) (push) Failing after 3m48s
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
Core CI / desktop (ubuntu-latest) (push) Successful in 5m53s
Core CI / desktop (macos-latest) (push) Canceled after 0s
Core CI / desktop (windows-latest) (push) Canceled after 0s
This commit is contained in:
karl committed
1 parent
79553ce921
commit
70746ef428
1 file changed
+106
-146
@@ -1,183 +1,143 @@
|
|||||||
# searchi
|
<p align="center">
|
||||||
|
<img src="assets/brand/searchi-app.png" alt="searchi app icon" width="96">
|
||||||
|
</p>
|
||||||
|
|
||||||
**Find what matters. Locally.**
|
<h1 align="center">searchi</h1>
|
||||||
|
<p align="center"><strong>Find what matters. Locally.</strong></p>
|
||||||
|
<p align="center">Your documents, photos, music and movies. One place to search.</p>
|
||||||
|
|
||||||
Local desktop search for Windows, macOS and Linux, under active development.
|
You remember what was in the file. searchi helps you find it—even when you
|
||||||
Production readiness is tracked in [the roadmap](docs/ROADMAP.md).
|
don't remember what it was called.
|
||||||
|
|
||||||
|
Search a filename, a phrase, or describe what you're looking for in your own
|
||||||
|
words. See the relevant passage or media preview alongside your results,
|
||||||
|
then open the file. Your files stay on your device.
|
||||||
|
|
||||||
*The production UI searching real fixture files with Google EmbeddingGemma 2.
|
|
||||||
Captured on Windows with CPU inference in the read-only local browser capture harness;
|
|
||||||
native window chrome is omitted.*
|
|
||||||
|
|
||||||
## What works
|
*“cancel the supplier partnership” finds the agreement and brings its
|
||||||
|
termination passage into view.*
|
||||||
|
|
||||||
Search combines instant indexed filenames, SQLite FTS5 content retrieval, and
|
## Find it your way
|
||||||
**real Google EmbeddingGemma 2** vectors in a persisted HNSW index. PDF pages,
|
|
||||||
DOCX paragraphs and text passages retain their source locations. The same
|
|
||||||
shared Rust service now indexes **images, video frames, movie soundtracks,
|
|
||||||
audio and music**: actual pixels and sound enter Google's media projector in
|
|
||||||
the same 768-dimensional space as text queries. No generated captions,
|
|
||||||
simulated embeddings, replacement model, or Granola service is required.
|
|
||||||
Existing artist/title tags contribute keyword matches.
|
|
||||||
|
|
||||||
The desktop is the primary interface. It automatically prepares verified local
|
| What you remember | What to try |
|
||||||
models, indexes selected folders, and prewarms its query worker. Image previews
|
| --- | --- |
|
||||||
and audio/video controls use narrowly scoped local assets; timestamp results
|
| The file's name | `Supplier agreement.pdf` |
|
||||||
seek inside the preview without autoplay. Open file launches the default
|
| Words inside a document | `sixty days written notice` |
|
||||||
application, whose exact opening position is not controlled by Searchi.
|
| What a document is about | `cancel the supplier partnership` |
|
||||||
The CLI provides the same retrieval engine without a GUI.
|
| A photo's subject | `mountains reflected in calm water` |
|
||||||
|
| An artist or song title | `Nina Simone` or `Feeling Good`, if present in your music tags |
|
||||||
|
| A scene or sound | Try `rain and thunder` or `a boat crossing a lake` in experimental media search |
|
||||||
|
|
||||||
**Media search is experimental.** File retrieval works on the independent seed
|
**Names, words and meaning work together.** Exact filenames help you get
|
||||||
corpus, but precise spoken-topic timestamps, music hard negatives, and no-answer
|
straight to a known file. Content search finds the words inside it. Google
|
||||||
handling still need improvement. Nearest-neighbor results are not calibrated
|
EmbeddingGemma 2 adds local semantic search for related meaning, images and
|
||||||
confidence. Video frames are sampled across the movie; every fleeting scene is
|
sound.
|
||||||
not guaranteed. Audio windows cover the primary stream within explicit resource
|
|
||||||
limits. See [actual media quality and failures](docs/MEDIA-QUALITY.md).
|
|
||||||
|
|
||||||
Production supports compiled hardware acceleration with CPU fallback. All
|
Use **All files** to narrow results by folder, file extension, modification
|
||||||
Windows VM tests use CPU only. Files, extracted text, metadata and embeddings
|
date, file size or media duration. Choose a result to inspect it before opening.
|
||||||
stay local; treat the index as sensitive. Initial verified model provisioning
|
|
||||||
requires approximately 865 MB of downloads. Search performs no remote inference.
|
|
||||||
|
|
||||||
## See searchi working
|
## A little less searching. A little more finding.
|
||||||
|
|
||||||
### A familiar dark theme
|
Matching words stand out in the results and document preview. PDF results
|
||||||
|
retain their page number; text results show the relevant passage. If you'd
|
||||||
|
like an explanation, expand the small **Why this result?** detail.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Light and dark themes follow the operating system preference. The selected
|
Light and dark themes follow your system preference. Keyboard navigation,
|
||||||
result retains its source location and extracted passage. Matching query words
|
short transitions and reduced-motion support keep the interface familiar
|
||||||
are highlighted in results and previews. A small, collapsed **Why this result?**
|
and comfortable.
|
||||||
detail is available when you want the explanation.
|
|
||||||
|
|
||||||
### Find a photo from what you remember
|
|
||||||
|
|
||||||
|
*“mountains reflected in calm water” searches the pictures themselves,
|
||||||
|
so a useful filename isn't required.*
|
||||||
|
|
||||||
Actual image pixels are indexed by the Google model. The selected photo is
|
Images open in the preview. Audio and video have playback controls, and
|
||||||
loaded from the indexed fixture file for preview.
|
matching media segments can jump to their approximate location. Nothing
|
||||||
|
autoplays when you select a result.
|
||||||
|
|
||||||
### Understand where the time goes
|
## Get searchi
|
||||||
|
|
||||||
|
**searchi is in active development. Ready-to-install downloads have not been
|
||||||
|
published yet.** Windows, macOS and Linux are the intended desktop platforms;
|
||||||
|
packaged releases and platform testing are still being completed.
|
||||||
|
|
||||||
Normal searches show a compact timing summary. Optional performance details
|
Check the [releases page](https://tea.karlcom.de/karl/searchi/releases) for
|
||||||
break down the real request into measured stages, including worker inference,
|
published downloads. If you want to try the current source version, follow
|
||||||
retrieval, filtering and ranking.
|
the [build and launch instructions](docs/BUILDING.md). Building it currently
|
||||||
|
requires development tools.
|
||||||
|
|
||||||
The [capture workflow](tools/desktop-capture/README.md) and
|
### Your first search
|
||||||
[capture manifest](docs/screenshots/capture-manifest.json) record the queries,
|
|
||||||
runtime, measured responses and image hashes. The photo sources and licenses
|
|
||||||
are listed in [the media fixture documentation](fixtures/media/README.md).
|
|
||||||
These browser captures supplement the open native desktop QA gates.
|
|
||||||
|
|
||||||
## Copy-and-paste setup prompt
|
Once searchi is running:
|
||||||
|
|
||||||
```text
|
1. Choose **Add folder**, browse to the folder you want to search, then choose
|
||||||
Set up Searchi from https://tea.karlcom.de/karl/searchi.git in a new searchi
|
**Index folder**.
|
||||||
folder, or reuse the checkout without overwriting local changes. Read AGENTS.md,
|
2. Start with a filename or words you remember. These searches become
|
||||||
SKILLS.md, docs/ROADMAP.md, docs/BUILDING.md and apps/desktop/README.md first.
|
available as files are indexed.
|
||||||
Inspect the OS/tools. Install supported missing prerequisites: Git, stable Rust,
|
3. Let the local search model finish preparing in the background. First
|
||||||
Node.js 24 LTS, Python 3.11+, CMake, native C/C++ tools and Tauri dependencies.
|
setup downloads approximately **865 MB** and needs an internet connection.
|
||||||
Preserve credentials and never print secrets. Prefer established Rust crates
|
The app shows its progress; semantic search becomes available when setup
|
||||||
and portable native tools; keep engine APIs shared by CLI and desktop.
|
and indexing finish.
|
||||||
For Karl's Windows VM set SEARCHI_FORCE_CPU=1 and
|
4. Describe what you're looking for, select a result and inspect the preview.
|
||||||
WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS='--disable-gpu --disable-accelerated-video-decode'
|
Press **Enter** to open the selected file in its usual application.
|
||||||
before tests or launch. Never enable a GPU provider in that VM.
|
|
||||||
From the repository root:
|
|
||||||
python tools/embeddinggemma2/build_runtime.py --backend cpu --jobs 2
|
|
||||||
python tools/embeddinggemma2/stage_runtime.py --backend cpu
|
|
||||||
python tools/media-runtime/provision.py --stage
|
|
||||||
The decoder manifest supports Windows/Linux x86-64 and ARM64; macOS decoder
|
|
||||||
packaging is still a gate. On unsupported hosts report the real blocker and
|
|
||||||
use explicitly verified native tools only when their provenance is known.
|
|
||||||
Run cargo test --workspace --locked, cargo fmt --all --check and
|
|
||||||
cargo clippy --workspace --all-targets --locked -- -D warnings.
|
|
||||||
Run cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml --locked.
|
|
||||||
In apps/desktop run npm ci, npm test, npm run format:check and npm run build.
|
|
||||||
Launch the real app with npm run tauri -- dev. Allow automatic verified Google
|
|
||||||
model setup and retained-worker warmup. Never substitute another model or mocks.
|
|
||||||
Ask which folders I want indexed; don't index my whole home automatically.
|
|
||||||
Verify filename/content/image/music/video searches and persistence on selected
|
|
||||||
files. Inspect media failure reports and actual preview seeking. Read
|
|
||||||
MEDIA-QUALITY.md before claiming precise speech timestamps or reliable absence.
|
|
||||||
Report actual checks and blockers. Do not publish a release or claim production
|
|
||||||
readiness, GPU validation, native visual QA, or untested OS coverage.
|
|
||||||
```
|
|
||||||
|
|
||||||
This is developer setup, not an installer. The worker builder also accepts
|
How quickly a folder becomes searchable depends on its size and contents.
|
||||||
`auto`, `metal`, `vulkan` and `cuda`; each requires its platform SDK and separate
|
The desktop keeps its search model warm for subsequent searches.
|
||||||
hardware validation. Standalone FFmpeg/FFprobe and notices are staged for the
|
Use **Reindex** after adding or changing files in an indexed folder.
|
||||||
desktop; end users should not install tools or configure decoder paths.
|
|
||||||
|
|
||||||
## Headless setup
|
## Private by design
|
||||||
|
|
||||||
Build with `cargo build --release --locked -p searchi-cli`. No Node, Tauri,
|
You choose the folders to index. File contents, previews and search processing
|
||||||
WebView or display server is needed. Build the native Google worker separately,
|
stay local. Your searches aren't sent to a cloud AI service.
|
||||||
and provision verified assets and FFmpeg tools:
|
|
||||||
|
|
||||||
```sh
|
The model download happens during setup. Once it's ready, semantic inference
|
||||||
python tools/embeddinggemma2/build_runtime.py --backend cpu --jobs 2
|
runs on your device. The local search index contains information from your
|
||||||
python tools/media-runtime/provision.py
|
files, so protect it as you would the files themselves.
|
||||||
cargo run -p searchi-cli -- setup --model-dir .searchi/models/embeddinggemma2
|
|
||||||
cargo run -p searchi-cli -- index --db searchi.sqlite --root /path/to/files
|
|
||||||
```
|
|
||||||
|
|
||||||
On native Linux, unified document/media indexing and hybrid search use:
|
## Documents and media
|
||||||
|
|
||||||
```sh
|
| Files | Current support |
|
||||||
./target/release/searchi embed --db searchi.sqlite --model-dir .searchi/models/embeddinggemma2 --embedding-worker .searchi/runtime/cpu/searchi-gemma-worker --ffmpeg .searchi/media-runtime/bin/ffmpeg --ffprobe .searchi/media-runtime/bin/ffprobe --cpu-threads 1
|
| --- | --- |
|
||||||
./target/release/searchi search --db searchi.sqlite 'the recording about orange soil' --model-dir .searchi/models/embeddinggemma2 --embedding-worker .searchi/runtime/cpu/searchi-gemma-worker --cpu-threads 1
|
| Documents | PDF, DOCX, plain text, Markdown and source code |
|
||||||
```
|
| Photos | JPEG, PNG, WebP and BMP |
|
||||||
|
| Music and audio | MP3, FLAC, WAV, M4A, OGG, Opus, AAC and AIFF |
|
||||||
|
| Movies and video | MP4, MKV, MOV, AVI, WebM, M4V and MPEG |
|
||||||
|
|
||||||
These are current interfaces, not executed Linux release evidence. Windows uses
|
**Media search is experimental.** Image, audio and video search are implemented,
|
||||||
`Release/searchi-gemma-worker.exe` and `ffmpeg.exe`/`ffprobe.exe` instead.
|
but relevance still needs improvement. Spoken topics can be missed, music
|
||||||
`embed` includes media by default and reports failures. Use `--text-only`
|
matches can be too broad, and video sampling can miss a brief scene. Media
|
||||||
explicitly for document-only vectors. `setup` provisions Google assets, not
|
timestamps identify approximate segments, rather than an exact spoken word.
|
||||||
worker/decoder executables. Installed release CLIs discover decoders under
|
Playback also depends on the formats your system supports; you can open a
|
||||||
`media-runtime/bin` beside the CLI; explicit paired paths or
|
file in its usual player if the preview cannot play it.
|
||||||
`SEARCHI_FFMPEG_PATH`/`SEARCHI_FFPROBE_PATH` override discovery. Search never
|
|
||||||
silently downloads missing helpers. CLI invocations load their own worker;
|
|
||||||
the desktop retains a prewarmed worker for interactive queries.
|
|
||||||
|
|
||||||
## Search performance
|
## Fast should feel fast
|
||||||
|
|
||||||
Normal searches show result counts, measured search time and time until results
|
Each search shows a compact result count and timing summary. Want to look
|
||||||
are displayed. Enable **Performance details** in desktop settings to
|
closer? Enable **Performance details** in settings for a breakdown of where
|
||||||
inspect UI scheduling, native queue/lock time, keyword retrieval, actual Google
|
the time went. Copied timing reports exclude your search text and file paths.
|
||||||
query inference, ANN/cache/filter work and ranking. Export the current trace as
|
|
||||||
local JSON for comparison; tracing sends no telemetry and excludes search text
|
|
||||||
and file paths. Nested stages and parallel retrieval lanes overlap.
|
|
||||||
|
|
||||||
For headless diagnostics, append `--performance --json` for basic metrics or
|
<details>
|
||||||
`--debug-performance --json` for the detailed response. Plain `--json` retains
|
<summary>See the optional performance view</summary>
|
||||||
the existing results-array format. See [timing boundaries, trace schema and
|
|
||||||
reproducible measurements](docs/PERFORMANCE.md).
|
|
||||||
|
|
||||||
## Development and release
|
|
||||||
|
|
||||||
The Rust engine, service, media adapter and CLI live under `crates/`; the Tauri
|
</details>
|
||||||
shell is `apps/desktop`. SQLite, image/jpeg-decoder, USearch/NumKong, and Tauri
|
|
||||||
provide established storage, decoding, ANN/SIMD and desktop facilities.
|
|
||||||
Extraction and inference use bounded isolated work with explicit errors.
|
|
||||||
|
|
||||||
```sh
|
## Help shape searchi
|
||||||
cargo test --workspace --locked
|
|
||||||
cargo fmt --all --check
|
|
||||||
cargo clippy --workspace --all-targets --locked -- -D warnings
|
|
||||||
cargo run -p searchi-cli -- --help
|
|
||||||
```
|
|
||||||
|
|
||||||
See [desktop setup](apps/desktop/README.md), [build/platform constraints](docs/BUILDING.md),
|
Found a problem, or searched for something that should have been easy to find?
|
||||||
[semantic bounds](docs/SEMANTIC.md), [executed validation](docs/VALIDATION.md),
|
[Report it or suggest an improvement](https://tea.karlcom.de/karl/searchi/issues).
|
||||||
and [brand direction](docs/BRAND.md). Native visual QA, operating-system builds,
|
Describe what you expected and what happened; use made-up examples if your
|
||||||
macOS decoder packaging, installers, licensing distribution, signatures,
|
files contain private information.
|
||||||
upgrades and representative production quality remain release gates.
|
|
||||||
Cross-platform source support does not mean signed macOS installers can be
|
|
||||||
built on Windows. CI declarations are not evidence of executed platform jobs.
|
|
||||||
|
|
||||||
Record benchmark hardware, corpus, cold/warm state and p95 latency. Never ship
|
For contributors: [building searchi and the headless Linux CLI](docs/BUILDING.md)
|
||||||
signing credentials in source control. Licensed under [MIT](LICENSE).
|
· [roadmap](docs/ROADMAP.md) · [search-quality evaluation](docs/MEDIA-QUALITY.md).
|
||||||
Repository author: Karl.
|
|
||||||
|
The screenshots show the working interface searching real example files with
|
||||||
|
the local Google model. They are browser captures of the app interface;
|
||||||
|
operating-system window borders are omitted. [Photo credits](fixtures/media/README.md).
|
||||||
|
|
||||||
|
Made by Karl. Open source under the [MIT license](LICENSE).
|
||||||
Reference in new issue
Block a user