From 169966f74f9fc8233e26fbfe816095e2ac2baf63 Mon Sep 17 00:00:00 2001 From: karl Date: Mon, 12 Oct 2026 17:26:14 +0200 Subject: [PATCH] docs: add agent workflows and copy-paste setup prompt --- AGENTS.md | 22 +++++++++++++++ README.md | 58 +++++++++++++++++++++++++++++++++++--- SKILLS.md | 84 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 160 insertions(+), 4 deletions(-) create mode 100644 SKILLS.md diff --git a/AGENTS.md b/AGENTS.md index 6fd8ca7..142c84d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,9 +1,31 @@ # Searchi development +Read README.md, SKILLS.md, and docs/ROADMAP.md before starting or resuming work. +Inspect Git status and preserve existing changes. Do not claim production +readiness without completed release gates and recorded evidence. + - All tests in the Windows VM must use CPU only. Never enable GPU providers here. +- Production uses available GPU/hardware acceleration with reliable CPU fallback; + the CPU-only rule is a VM testing constraint, not a product restriction. - Keep the Rust engine independent of the desktop shell; CLI and UI share its API. - Use bounded work, explicit errors, versioned storage, and offline defaults. - Never substitute simulated embeddings for working semantic search. +- Production semantic search must use Google EmbeddingGemma 2 as in DigUp. + MiniLM proof artifacts are historical experiments, not an accepted backend. +- The desktop UI is the primary product, including local model setup and semantic + indexing/search. Keep a separate CLI-only build for headless Linux with no GUI dependencies. +- Desktop setup must work out of the box: ship the native worker, automatically + download/verify missing Google assets, reuse cached assets, and expose progress/retry in the UI. - Add meaningful tests for retrieval, persistence, and filesystem lifecycle changes. - Keep docs/ROADMAP.md current. Unexecuted platform checks remain unchecked. - Agents own assigned paths and must coordinate shared interface changes. Do not commit other agents' unfinished work. +- Prepare indexes and the real Google query session in the background. Retain + warmed readers and keep keyword search usable during setup. Measure startup, + actual first input, and warm p95 separately. +- UI should belong to each OS: native typography/chrome/dialogs, themes, + keyboard access, reduced motion and high contrast. Brief transitions must not + delay requests. Preserve selection/focus as progressive results arrive. +- Follow SKILLS.md for validation and handoffs. Leave actual checks, blockers, + and concrete next tasks. Keep unexecuted OS and GPU checks explicitly open. +- Use karl as Git author with no agent co-author trailers. Keep credentials, + model binaries, generated build assets, and local indexes out of Git. diff --git a/README.md b/README.md index ea780ca..d52f6c7 100644 --- a/README.md +++ b/README.md @@ -7,13 +7,55 @@ production readiness is tracked in [the roadmap](docs/ROADMAP.md). `crates/searchi-core` owns persistent indexing and retrieval. `crates/searchi-cli` provides a headless interface. `apps/desktop` hosts the Tauri desktop app. -The initial implementation provides lexical search over filenames, bounded text, -PDF pages and DOCX paragraphs. Semantic and media search are explicit upcoming -milestones; a genuine CPU model proof is included separately. +Search combines indexed filenames, SQLite FTS5 keyword retrieval, and real +Google EmbeddingGemma 2 semantic search with a persistent HNSW index. PDF pages +and DOCX paragraphs retain source locations. Image, audio, and video search remain +planned; the provisioned media projector is not yet used. Granola is not required. +The desktop UI is primary; the CLI builds separately without GUI dependencies. -All testing in the Windows development VM uses CPU only. Indexed content stays +Production uses supported hardware acceleration with CPU fallback. All testing +in the Windows development VM uses CPU only. Indexed content stays local. Treat the index as sensitive: it contains extracted document text. +The desktop automatically downloads and verifies missing Google artifacts +(approximately 865 MB), reuses its local cache, and warms the query session in +the background. Development builds must build and stage the native worker first; +production installers and signing remain release gates. First-query performance +is measured separately from repeated-query performance. + +## Copy-and-paste setup prompt + +Paste this into your coding agent: + +```text +Set up Searchi from https://tea.karlcom.de/karl/searchi.git in a new searchi +directory, or reuse the existing checkout without overwriting local changes. +Read AGENTS.md, SKILLS.md, docs/ROADMAP.md, and apps/desktop/README.md first. +Inspect the OS and installed tools. Install missing supported prerequisites: +Git, stable Rust, Node.js 24+, Python 3, CMake, native C/C++ tools and Tauri's +platform dependencies. Preserve existing credentials and never print secrets. +From the repository root, build and stage the pinned Google runtime: +python tools/embeddinggemma2/build_runtime.py --backend cpu --jobs 2 +python tools/embeddinggemma2/stage_runtime.py --backend cpu +Run cargo test --workspace --locked, cargo fmt --all -- --check, and +cargo clippy --workspace --all-targets --locked -- -D warnings. +Run cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml --locked. +In apps/desktop run npm ci, npm test, npm run format:check, and npm run build. +For Karl's Windows VM set SEARCHI_FORCE_CPU=1 and +WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS=--disable-gpu before tests or launch. +Launch the real app using npm run tauri dev from apps/desktop. Allow automatic +verified Google model setup; never substitute another model or mock results. +Ask which folders I want indexed; do not index my whole home automatically. +Verify filename and remembered-content searches on my selected files, including +persistence after restart. Report actual checks and blockers. Do not publish a +release or claim production readiness, GPU validation, or untested OS coverage. +``` + +This is developer setup, not an installer. See the [desktop guide](apps/desktop/README.md) +and [official Tauri prerequisites](https://v2.tauri.app/start/prerequisites/). +The worker builder also supports `auto`, `metal`, `vulkan`, and `cuda`; those +require platform SDKs and separate hardware validation. + ## Development Install stable Rust and the platform's C/C++ build prerequisites. Run: @@ -26,6 +68,12 @@ cargo run -p searchi-cli -- --help ``` Desktop development instructions live in `apps/desktop/README.md`. +Agent constraints and workflows live in [AGENTS.md](AGENTS.md) and +[SKILLS.md](SKILLS.md). Use Node.js 24+ for frontend state tests. + +For a headless build, run `cargo build --release --locked -p searchi-cli`. +No Node, Tauri, or WebView is needed. Semantic commands additionally need the +native worker and Google assets; `searchi setup` provisions the model cache. Platform and CI details live in [the build guide](docs/BUILDING.md). Native CI builds must pass on all supported systems before release. Cross-platform source support does not imply signed macOS installers can be built on Windows. @@ -35,3 +83,5 @@ source support does not imply signed macOS installers can be built on Windows. Do not publish until the roadmap's release gates have evidence. Record benchmark hardware, corpus size, cold/warm state and p95 latency. Signing credentials are supplied through CI secrets, never source control. + +Licensed under [MIT](LICENSE). Repository author: Karl. diff --git a/SKILLS.md b/SKILLS.md new file mode 100644 index 0000000..e056e64 --- /dev/null +++ b/SKILLS.md @@ -0,0 +1,84 @@ +# Searchi agent workflows + +This repository guide does not automatically install a Codex skill or plugin. +Read AGENTS.md, README.md, and docs/ROADMAP.md before working. + +## Resume and implement + +Inspect Git status and recent commits, preserve local changes, and choose a +concrete roadmap milestone with acceptance criteria. Read relevant code and tests +first. Assign paths for authorized parallel work and coordinate shared APIs. +Keep the production goal active until all required gates have recorded evidence. + +## Search and indexing + +Preserve strict filters before candidate limits, exact identifier priority, +real Google vectors, source locations, atomic semantic generation activation, +and stale/deleted source suppression. Test updates, renames, deletion, restart, +corrupt documents, extraction limits, and interrupted work. Never silently drop +searchable content or hide slow first inputs behind warm percentiles. + +## Runtime and setup + +Use tools/embeddinggemma2/model-manifest.json and its pinned runtime builder. +Build/stage CPU workers for VM tests. Production acceleration requires separate +hardware evidence and reliable CPU fallback. Verify size/SHA-256, cache reuse, +interrupted downloads, errors, and retries. Keep inference local. Never substitute +MiniLM, simulated vectors, or hosted inference. Ship the worker with the app. + +## Desktop design + +The desktop UI is primary. Use native typography, chrome, dialogs, shortcuts, +light/dark themes, reduced motion, and high contrast. Keep search dominant and +previews readable; setup status must be honest. Preserve selection, focus, and +scroll when semantic results arrive. Brief transitions must not delay requests +or block interaction. The selected direction is the first generated concept, +with Raycast as a reference: compact search, result rows, and a passage preview. +Generated mockups do not establish functional PDF navigation or visual QA. + +## Run checks + +Use stable Rust and Node.js 24+. Before Windows VM tests or launch: + +```powershell +$env:SEARCHI_FORCE_CPU = '1' +$env:WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS = '--disable-gpu' +``` + +From the repository root: + +```sh +cargo fmt --all -- --check +cargo test --workspace --locked +cargo clippy --workspace --all-targets --locked -- -D warnings +cargo fmt --manifest-path apps/desktop/src-tauri/Cargo.toml -- --check +cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml --locked +``` + +From apps/desktop: + +```sh +npm ci +npm test +npm run format:check +npm run build +``` + +Real-model tests require verified assets and the native CPU worker. Set +SEARCHI_GEMMA_MODEL_DIR and SEARCHI_GEMMA_WORKER, then run: + +```sh +cargo test -p searchi-semantic --test gemma_cpu -- --ignored --nocapture +``` + +Record actual outputs, hardware, corpus size, background preparation, first-input +latency and warm latency in the relevant validation report. Unexecuted platform +checks stay open. Compilation alone is not UI interaction verification. + +## Commit and hand off + +Review staged diffs for secrets, model binaries, generated files, and unfinished +parallel work. Use Karl as author with no agent co-author trailers. Push only when +authorized. Karl's infrastructure uses selected Infisical injection; never print +credentials or commit them. Leave changed paths, checks, blockers, and a concrete +next task. Mark roadmap items complete only with recorded evidence.