searchi/docs/QUERY-CONTEXT.md

Foreground query context

The normal hardware-detecting Google worker now accepts --role query. That role allocates 512 tokens for n_ctx, n_batch and n_ubatch, matching searchi's existing query token limit. The default indexing role retains all 2048 tokens and document/image/audio operations. This is one binary, not a separate CPU distribution. Hardware selection and CPU fallback are unchanged.

Desktop foreground sessions, CLI semantic searches and the retained MCP search session use HybridSearch::open_query. Background document/media indexing continues to use open. Role is retained when a failed GPU process is restarted as CPU. A query-role worker rejects indexing operations before Rust verifies a projector or decodes images, and the native protocol independently rejects them. Invalid oversized queries leave the session usable.

The handshake reports role and context_tokens. A legacy worker cannot silently claim the smaller allocation: update/rebuild the worker with the app. media_supported still describes the installed binary's compiled capability; it remains true for query sessions so desktop status does not incorrectly hide media indexing. It does not mean the foreground session loads the projector. Query vectors and model/index fingerprints remain unchanged.

Actual Windows CPU evidence, 2026-10-13

Normal Windows x64 CPU/Vulkan worker, pinned llama b11461-no-logits-v1, verified Google Q8_0 backbone, CPU forced and no Vulkan module loaded. Windows 11 build 26300 VM, 8 logical processors, one inference thread. Six fresh worker processes, three per role, alternating order; cached filesystem/model bytes. These are small paired allocation measurements, not cold-disk, UI or p95 proof.

Median measurement	Full 2048	Query 512
Startup to ready	716.69 ms	710.10 ms
Ready private commitment	421,359,616 bytes	238,362,624 bytes
Ready resident working set	118,394,880 bytes	118,276,096 bytes
After first query private commitment	422,735,872 bytes	239,734,784 bytes
After first query resident working set	274,632,704 bytes	273,989,632 bytes
First query IPC/inference	106.57 ms	103.20 ms

Private commitment decreased by about 183 MB (43.4%). Resident memory and startup were effectively unchanged in this sample. Do not advertise an 183 MB RSS saving or a search-speed improvement from these results. Model verification took 195.86 ms once before the process trials; startup excludes Rust verification and desktop background preparation. Working sets include mapped shared model pages.

The measured worker SHA256 was 3bd88fdb26f5d62d400ed101277f637af028c757c619547f6333b2912137c60b. Raw samples are retained locally at .searchi/qa/query-context-windows.json.

The actual Google integration query_context_cpu passed in 27.39 s: English/German remembered-document ranking unchanged across both contexts; CodeRetrieval vector parity for English/German/Arabic/Bengali; actual tokenizer count exactly 512 with vector parity, rejection at 513 followed by a successful query; foreground indexing/media guards and full-context long-document windows. Three bilingual ranking vectors had cosine 0.99999999 or better. This tiny regression is correctness proof, not independent retrieval-quality evaluation. The benchmark additionally sends a direct native document operation to the query role, verifies rejection is nonfatal, and then performs CodeRetrieval.

Final worker rebuild after consolidating the query-size constant has SHA256 1272797af94c21db1ccabaa3ad996ab57d64f1bf9d33280c61d65c51360deab7. Six actual CPU integration tests passed with this final binary: two code fixtures (16.52 s), two text/debug/reference fixtures (59.18 s), image/audio reference parity (52.20 s), and the query-context test again (26.05 s). Image reference cosines were 0.99994150, 0.99977568 and 0.99917617; audio reference cosine was 0.99975052. The full indexing/media role remains usable. Native startup/memory values above belong to the explicitly identified earlier binary, before this equivalent constant cleanup; they are not installer checks.

An actual CLI/MCP boundary regression also passed (36.36 s): the CLI indexed three real document/code sources with the full role, CLI semantic search returned the expected supplier file, and the retained MCP session reported query role, 512-token context and CPU. Both remembered-document and code-only searches returned source handles whose guarded excerpts matched original text/line data. Compiled media support remained visible to clients. This is a tiny integration fixture, not a representative agent-library quality or performance benchmark.

Reproduce with verified assets and the current normal worker:

$env:SEARCHI_FORCE_CPU = '1'
$env:SEARCHI_GEMMA_MODEL_DIR = 'PATH/TO/VERIFIED/MODELS'
$env:SEARCHI_GEMMA_WORKER = 'PATH/TO/searchi-gemma-worker.exe'
cargo test -p searchi-semantic --test query_context_cpu --locked -- --ignored --nocapture
cargo test -p searchi-cli --test mcp actual_google_cli_and_mcp_use_query_role_with_cited_results --locked -- --ignored --nocapture
python tools/embeddinggemma2/query_context_benchmark.py --worker "$env:SEARCHI_GEMMA_WORKER" --models "$env:SEARCHI_GEMMA_MODEL_DIR" --output .searchi/qa/query-context.json --repeats 3

The benchmark reports Windows private commitment/working set or Linux PSS/private resident pages separately. Actual native Linux query/client checks and six paired measurements also passed; see Linux evidence. Linux resident/PSS memory was effectively unchanged in those samples. Native macOS and actual GPU context allocation/parity, pressure behavior, long code queries, representative query latency and updated installed desktop packaging remain release checks.

Current Windows CPU worker timing, 2026-10-13

The normal current worker (c01cb3ec13cfca722ee768e8350e1399a8b9a4d14a24a35adc3473a5cfc82a73) was measured with CPU explicitly selected and one inference thread. Three paired indexing/query processes alternate order; each retains a separate first request and twenty distinct warmed inputs with native debug timings. Real 768-dimensional vectors were checked. Output preservation now refuses existing reports.

Query-role repeat	Process startup ms	First inference/IPC ms	Warm p95 ms (20 samples)
0	729.27	100.02	71.30
1	662.83	106.18	78.38
2	704.21	100.84	71.75

One-time text-model size/SHA verification took 192.45 ms and is separate from worker startup. Indexing-role startup ranged 624.49–779.14 ms, first query 101.64–110.53 ms, warm p95 73.03–80.47 ms. These are cached-file native-worker measurements, not cold disk, Rust query-cache, ANN, hybrid retrieval, actual UI first input or paint. The installed desktop/query worker remained present; background activity was not controlled, so this is not an idle reference-host acceptance measurement. All test inference used CPU; no GPU provider was loaded.

Full ignored report: .searchi/qa/query-context-warm-current.json, SHA-256 ea7dd0b72eeb1dbd79f06fce92e0475dfdc321d7781e5d06d58fe12755faf0fd. The query-role memory observation after first inference was 239,878,144 bytes private commitment and 274,141,184 bytes resident working set for repeat zero; these are different metrics and must not be summed. Native desktop first-input and complete search latency remain separate open checks.