Files
searchi/docs/BUILDING.md
T

168 lines
9.0 KiB
Markdown
Raw Normal View History

# Build and release targets
Use stable Rust, Node 24 LTS, Python 3.11+, Git, CMake, and a native C/C++
toolchain. Python provisions the pinned Google runtime and model; inference
runs in a native process. Default Rust tests do not download model assets.
Windows needs Visual Studio C++ build tools, a Windows SDK, and WebView2.
Linux desktop builds need WebKitGTK 4.1 development libraries; macOS needs
Xcode command-line tools. Follow the [Tauri prerequisites](https://v2.tauri.app/start/prerequisites/).
Headless Linux CLI/worker builds do not need WebKitGTK or a display server.
They still need a C/C++ compiler, linker, CMake, and Rust: SQLite, USearch,
NumKong, and the Google worker contain native code.
## Windows CPU development
Run from the repository root. Building the worker clones a pinned llama.cpp
revision and verifies/applies its memory patch. Model provisioning explicitly
uses the network and verifies both Google assets, approximately 865 MB total.
The Python CMake package is needed only when CMake is absent from PATH.
```powershell
python -m pip install cmake
python tools/embeddinggemma2/build_runtime.py --backend cpu --jobs 2
python tools/embeddinggemma2/stage_runtime.py --backend cpu
python tools/media-runtime/provision.py --stage
python tools/embeddinggemma2/download.py
$env:SEARCHI_FORCE_CPU = '1'
$env:SEARCHI_GEMMA_MODEL_DIR = (Resolve-Path .searchi/models/embeddinggemma2).Path
$env:SEARCHI_GEMMA_WORKER = (Resolve-Path .searchi/runtime/cpu/Release/searchi-gemma-worker.exe).Path
cargo fmt --all --check
cargo clippy --workspace --all-targets --locked -- -D warnings
cargo test --workspace --locked
cargo test -p searchi-semantic --test gemma_cpu -- --ignored --nocapture
cargo test -p searchi-service --test google_cpu -- --ignored --nocapture
cargo test -p searchi-service --lib google_cpu_preserves -- --ignored --nocapture
```
The worker is statically linked with the MSVC CRT on Windows. Staging copies
it and the llama.cpp license into the desktop resources directory. Add
`--bundle-model` to staging for an offline model bundle after downloading;
otherwise desktop setup provisions verified models on first launch. Search
itself performs no network inference or downloads. The worker now uses the verified media projector for genuine image/audio embeddings. The FFmpeg script stages verified standalone decoders and their notices.
The desktop is a standalone Cargo workspace and needs separate validation:
```powershell
cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml --locked
Set-Location apps/desktop
npm ci
npm run format:check
npm test
npm run build
$env:WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS = '--disable-gpu --disable-accelerated-video-decode'
npm run tauri -- dev
```
Every Windows VM inference/rendering test must stay CPU only. Production
supports `--backend auto|metal|vulkan|cuda` builds with CPU fallback; do not
exercise those acceleration backends in this VM. Their execution and failure
recovery require validation on the corresponding release hardware.
## Headless Linux Google search
These are the current script and CLI interfaces, not evidence of an executed
Linux release. On a native Linux build host, run from the repository root:
```sh
python3 tools/embeddinggemma2/build_runtime.py --backend cpu --jobs 2
python3 tools/embeddinggemma2/download.py
python3 tools/media-runtime/provision.py
export SEARCHI_FORCE_CPU=1
cargo build --release -p searchi-cli --locked
./target/release/searchi index --db searchi.sqlite --root /path/to/documents
./target/release/searchi embed --db searchi.sqlite --model-dir .searchi/models/embeddinggemma2 --embedding-worker .searchi/runtime/cpu/searchi-gemma-worker --ffmpeg .searchi/media-runtime/bin/ffmpeg --ffprobe .searchi/media-runtime/bin/ffprobe --cpu-threads 1
./target/release/searchi search --db searchi.sqlite 'supplier cancellation terms' --model-dir .searchi/models/embeddinggemma2 --embedding-worker .searchi/runtime/cpu/searchi-gemma-worker --cpu-threads 1
```
`embed` builds document, image, video and audio/music vectors by default. Use
`--text-only` explicitly to omit media. Default media indexing needs paired
FFmpeg/FFprobe paths or packaged helpers beside the installed CLI.
`searchi setup --model-dir PATH` is an alternative verified model-provisioning
command. It provisions model assets, not the native worker. Package the worker
separately and pass its path to headless embedding/search. Each short-lived CLI
search starts a model process; the desktop retains a prewarmed query worker.
## Cross-compilation and native release matrix
Normal distribution worker builds and staging default to `auto`: Metal on
macOS, Vulkan on Windows/Linux, with CPU fallback in the same application.
Users do not choose a CPU/GPU edition or rebuild for their graphics card.
Windows/Linux build machines need the Vulkan SDK including `glslc`. Windows
x64 provisioning now downloads the pinned, checksum-verified official SDK into
the workspace using copy-only mode; it does not install drivers or change the
system environment. Linux builders provision their distro's Vulkan development
tools. Missing required tools fail release preparation rather than omitting GPU.
The compilation SDK is not an end-user requirement. Explicit `--backend cpu`
commands in the VM development sections are internal test configurations.
The shared Rust engine/service has no GUI dependency. Install a destination
Rust target with `rustup target add TARGET`, then build using
`cargo build -p searchi-cli --target TARGET --locked`. This also requires the
target C/C++ toolchain, linker, and SDK for bundled SQLite and the vector index.
The separate native Google worker must be compiled for the same destination;
installing a Rust target does not cross-compile it. The current Python runtime
script configures the host toolchain; it does not expose a CMake cross-toolchain
file interface. Record each verified host/target pair before claiming support.
| Release host | Intended artifacts | Remaining validation |
| --- | --- | --- |
| Windows | Windows CLI, worker, desktop installer | Installer, upgrade, signing, GPU hardware |
| macOS | Apple Silicon/Intel CLI, worker, desktop bundle | Native builds, architecture parity, Metal, notarization |
| Linux | CLI, worker, desktop packages | Native builds, distribution ABI, webview, acceleration |
Use native runners for dependable desktop releases. Platform webviews,
packaging tools, SDKs, and signing impose additional constraints. macOS
signing/notarization requires Apple tooling and credentials. See the
[Tauri Windows installer cross-build caveats](https://v2.tauri.app/distribute/windows-installer/).
## CI and packaging gates
### Windows archive for source installations
After staging the CPU worker and media helpers, build an embedded-frontend
desktop binary and collect its runtime files:
```powershell
Set-Location apps/desktop
npm run tauri -- build --no-bundle
Set-Location ../..
python tools/package_windows.py
```
The generated `.searchi/packages/searchi-windows.zip` contains the desktop,
matching-architecture Google worker, FFmpeg/FFprobe and notices. It excludes
model caches, local settings and indexes. `PACKAGE.json` records SHA-256 for
each included file. Extract the entire archive before launching `searchi.exe`.
First launch uses normal verified model setup. WebView2 and the applicable
Microsoft C++ runtime remain Windows prerequisites. The Windows archive includes
the verified Vulkan loader DLL and its license; GPU availability is detected at
runtime, with CPU fallback. The SDK is used only to compile the app.
This is an unsigned development archive, not a production installer. Native
launch, clean-machine dependencies, upgrades and signatures must still be
verified. The packager rejects mixed architectures and preserves a previous
archive if validation fails. It does not certify GPU dependency packaging.
### CI status and installer gates
Gitea Actions are disabled for this repository at the owner's request after
repeated failure emails (2026-10-12). Existing workflows remain as references:
GitHub defines native OS matrices; Gitea targets its available Ubuntu runner.
A workflow declaration does not prove
a platform was tested; Linux/macOS gates remain open until actual job evidence
exists. Current workflow checks do not certify the separately built Google
worker or execute downloaded-model tests automatically.
Desktop bundles remain disabled during development. Before enabling release
installers, stage the target worker, retain dependency/model notices, verify
clean-machine launch and setup, run native Google smoke tests, and complete
upgrade/migration, signatures, and rollback checks. Optional offline bundles
need both verified model files. See [semantic bounds and evidence](SEMANTIC.md)
for current corpus, localization, and performance limitations. Keep signing keys,
repository tokens, and model credentials outside source control.
FFmpeg provisioning supports manifest-pinned Windows/Linux x86-64 and ARM64 artifacts;
only Windows x86-64 has executed here. macOS decoder packaging remains a gate.
See [media runtime provenance](../tools/media-runtime/README.md) and
[measured media limitations](MEDIA-QUALITY.md).