# CPU semantic inference proof

**Historical experiment, not the accepted Searchi backend.** The user requires
Google EmbeddingGemma 2 as used by DigUp. MiniLM results below establish only
runtime plumbing and must not be presented as completed product semantic search.
The implementation is being corrected to use the required Google model.

This milestone runs genuine neural embeddings locally. It does not yet connect
semantic vectors to the SQLite search index or desktop interface. Production
retrieval quality, large-corpus performance, and Linux/macOS runtime packaging
remain unverified.

## Model and runtime decision

The first working baseline is the quantized ONNX export of
`sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2`, published by
`Xenova`. The upstream model card describes 384-dimensional text embeddings,
attention-mask mean pooling, a 128-token maximum, and Apache-2.0 licensing.
It covers multiple languages. This is a text-only baseline; it does not claim
EmbeddingGemma 2's multimodal features or compatibility.

Sources reviewed on 2026-10-12:

- [Original model card and pooling architecture](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2)
- [ONNX export publisher](https://huggingface.co/Xenova/paraphrase-multilingual-MiniLM-L12-v2)
- [ONNX Runtime Python session API](https://onnxruntime.ai/docs/api/python/api_summary.html)
- [Rust ort dynamic-library initialization](https://docs.rs/ort/2.0.0-rc.13/ort/environment/fn.init_from.html)
- [Rust ort session configuration](https://docs.rs/ort/2.0.0-rc.13/ort/session/builder/struct.SessionBuilder.html)

Inference explicitly selects `CPUExecutionProvider`. Python uses the CPU
`onnxruntime` wheel, never `onnxruntime-gpu`. The native Rust crate disables ort
default features: there are no binary downloads, GPU provider features, or model
HTTP client features. It loads an explicitly supplied local runtime library and
disables runtime telemetry. Tokenization is sequential; ONNX sessions use
sequential graph execution with bounded intra-op threads and one inter-op thread.

## Reproduce on Windows

Run these commands from the repository root. Provisioning commands intentionally
use the network; the inference command does not provision anything.

```powershell
python -m venv tools/semantic-proof/.venv
& tools/semantic-proof/.venv/Scripts/python.exe -m pip install --only-binary=:all: -r tools/semantic-proof/requirements.txt
& tools/semantic-proof/.venv/Scripts/python.exe tools/semantic-proof/download.py
& tools/semantic-proof/.venv/Scripts/python.exe tools/semantic-proof/proof.py --threads 1 --output tools/semantic-proof/windows-cpu-result.json
```

On Linux/macOS, use `.venv/bin/python` instead of `.venv/Scripts/python.exe`.
These instructions have only been executed on Windows. Wheel availability and
native library distribution must be checked on other platforms.

`download.py` downloads exactly two revision-pinned artifacts, streams them to a
temporary file, rejects oversized downloads, checks byte length and SHA256, then
atomically replaces the local asset. Assets occupy approximately 136 MB in
decimal units and are ignored by Git. Inference validates both files again and
fails clearly when either is missing or damaged. It never contacts the model
hub. The Python proof sets offline environment flags and rejects Python socket
connect, name-resolution, and bind audit events throughout import and inference.
This guard checks Python socket activity; it is not an operating-system firewall
against arbitrary native libraries.

The proof intentionally exits nonzero if any fixture query misses top-1. Its
JSON report is written before the quality gate, preserving failures for review.

## Pinned artifacts and licensing

Revision: `2c4055b12046f11709e9df2c122e59ffbdc2f900`.
The manifest in `tools/semantic-proof/model-manifest.json` records the same
model contract and hashes used by the native adapter.

| Artifact | Bytes | SHA256 |
| --- | ---: | --- |
| `onnx/model_quantized.onnx` | 118,308,126 | `66fc00f5f29afcaff34092e1bdd20008ca3918265a82fb9695a551e510cc4ebc` |
| `tokenizer.json` | 17,082,913 | `b60b6b43406a48bf3638526314f3d232d97058bc93472ff2de930d43686fa441` |

The base model card specifies Apache-2.0. The export card identifies that base
model but does not display its own license declaration. Before redistributing
the ONNX export in installers, confirm export licensing and preserve the base
license, attribution, and applicable notices. Current code explicitly downloads
user-local assets and does not ship model weights or runtime DLLs in Git.
ONNX Runtime uses MIT licensing; ort uses MIT or Apache-2.0 and tokenizers uses
Apache-2.0. Release packaging must retain third-party notices and validate
platform-specific runtime hashes independently; a model hash does not attest
to the runtime library.

The tested Windows ONNX Runtime 1.31.0 DLL is 19,288,888 bytes with SHA256
`90497e3d406c09f4d59d364c696a2f93dc02f6ef13741b3b89eec95bdb4a3806`.
The adapter currently trusts the explicitly supplied runtime path; it validates
model hashes, not runtime-library hashes. A release distribution must add a
platform runtime manifest before shipping. `requirements-lock-windows.txt`
captures the exact Python environment used for this proof.

## Measured Windows VM result

`tools/semantic-proof/windows-cpu-result.json` contains the full measurement and
rankings. The run used Windows 11 AMD64, Python 3.14.8, ONNX Runtime 1.31.0,
four visible logical CPUs, and **one inference thread with only the CPU provider**.

| Metric | Measurement |
| --- | ---: |
| Fixture documents / queries | 8 / 6 |
| Timed warm query runs | 60 |
| Artifact verification plus runtime/model load | approximately 3.4 seconds |
| Corpus batch embedding | approximately 145 ms |
| Warm query p50 / p95 | approximately 3.1 / 11.9 ms |
| Top-1 accuracy | 5/6 (83.3%) |
| Recall at 3 | 6/6 (100%) |

The corpus includes English and German documents, same-language semantic
queries, and cross-language queries. A question about a postponed product
release ranks the supplier contract above the expected meeting notes. That
failure remains in the fixture and blocks the strict top-1 quality gate.
This small demonstration supports the feasibility of CPU inference; it does
not establish large-corpus latency, robust multilingual quality, or production
readiness. Quantization and batch shape can affect similarity scores.

## Native integration contract

`searchi-semantic` exposes:

```rust
CpuEmbedder::load(model_dir, runtime_library, threads) -> Result<CpuEmbedder>
CpuEmbedder::encode(&mut self, texts: &[&str]) -> Result<Vec<Vec<f32>>>
DIMENSIONS // 384
MODEL_FINGERPRINT // revision + quantization + pooling + normalization + token cap
```

Inputs are bounded to eight texts per batch, 64 KiB UTF-8 per text, and 128
tokens. Outputs are finite, normalized `f32` vectors. The runtime library is
process-global; adapter initialization rejects attempts to switch libraries or
silently accept an environment initialized outside the adapter. Supply a known,
trusted runtime before calling any other ort APIs in the process.

Default unit tests need no model or runtime. The real-artifact test is opt-in:

```powershell
$env:SEARCHI_MODEL_DIR = (Resolve-Path tools/semantic-proof/model).Path
$env:SEARCHI_ORT_LIBRARY = (Resolve-Path tools/semantic-proof/.venv/Lib/site-packages/onnxruntime/capi/onnxruntime.dll).Path
cargo test -p searchi-semantic --test cpu_model -- --ignored --nocapture
```

The smoke test checks actual local inference, vector normalization, English and
German retrieval, recall at three, and the measured five-of-six top-1 baseline.
It does not replace the strict production quality gate.

The native real-artifact smoke was executed successfully on the Windows VM,
using that CPU-only DLL. It reproduced five-of-six top-1 and six-of-six recall
at three. Three default unit tests passed; targeted Clippy passed with warnings
denied. Linux and macOS native checks have not been executed.

The next integration should read bounded `SearchIndex::passage_batch` rows,
split long passages into overlapping tokenizer windows, preserve source page or
paragraph locations, and persist each vector with a content fingerprint and
`MODEL_FINGERPRINT`. Passage IDs are not immutable content identities: changes
must invalidate stale vectors. Queries should combine lexical and semantic
retrieval while preserving exact filename priority. Model upgrades build a
separate vector generation before an atomic swap. Cancellation, persistent
embedding jobs, ANN retrieval, representative quality corpora, and signed
cross-platform runtime packaging are still required.