138 lines
8.2 KiB
Markdown
138 lines
8.2 KiB
Markdown
# Foreground query context
|
||
|
||
The normal hardware-detecting Google worker now accepts `--role query`.
|
||
That role allocates 512 tokens for `n_ctx`, `n_batch` and `n_ubatch`, matching
|
||
searchi's existing query token limit. The default `indexing` role retains all
|
||
2048 tokens and document/image/audio operations. This is one binary, not a
|
||
separate CPU distribution. Hardware selection and CPU fallback are unchanged.
|
||
|
||
Desktop foreground sessions, CLI semantic searches and the retained MCP search
|
||
session use `HybridSearch::open_query`. Background document/media indexing
|
||
continues to use `open`. Role is retained when a failed GPU process is restarted
|
||
as CPU. A query-role worker rejects indexing operations before Rust verifies a
|
||
projector or decodes images, and the native protocol independently rejects them.
|
||
Invalid oversized queries leave the session usable.
|
||
|
||
The handshake reports `role` and `context_tokens`. A legacy worker cannot
|
||
silently claim the smaller allocation: update/rebuild the worker with the app.
|
||
`media_supported` still describes the installed binary's compiled capability;
|
||
it remains true for query sessions so desktop status does not incorrectly hide
|
||
media indexing. It does not mean the foreground session loads the projector.
|
||
Query vectors and model/index fingerprints remain unchanged.
|
||
|
||
## Actual Windows CPU evidence, 2026-10-13
|
||
|
||
Normal Windows x64 CPU/Vulkan worker, pinned llama b11461-no-logits-v1,
|
||
verified Google Q8_0 backbone, CPU forced and no Vulkan module loaded. Windows
|
||
11 build 26300 VM, 8 logical processors, one inference thread. Six fresh worker
|
||
processes, three per role, alternating order; cached filesystem/model bytes.
|
||
These are small paired allocation measurements, not cold-disk, UI or p95 proof.
|
||
|
||
| Median measurement | Full 2048 | Query 512 |
|
||
| --- | ---: | ---: |
|
||
| Startup to ready |716.69 ms|710.10 ms|
|
||
| Ready private commitment |421,359,616 bytes|238,362,624 bytes|
|
||
| Ready resident working set |118,394,880 bytes|118,276,096 bytes|
|
||
| After first query private commitment |422,735,872 bytes|239,734,784 bytes|
|
||
| After first query resident working set |274,632,704 bytes|273,989,632 bytes|
|
||
| First query IPC/inference |106.57 ms|103.20 ms|
|
||
|
||
Private commitment decreased by about 183 MB (43.4%). Resident memory and startup
|
||
were effectively unchanged in this sample. Do not advertise an 183 MB RSS saving
|
||
or a search-speed improvement from these results. Model verification took 195.86 ms
|
||
once before the process trials; startup excludes Rust verification and desktop
|
||
background preparation. Working sets include mapped shared model pages.
|
||
|
||
The measured worker SHA256 was
|
||
`3bd88fdb26f5d62d400ed101277f637af028c757c619547f6333b2912137c60b`.
|
||
Raw samples are retained locally at `.searchi/qa/query-context-windows.json`.
|
||
|
||
The actual Google integration `query_context_cpu` passed in 27.39 s:
|
||
English/German remembered-document ranking unchanged across both contexts;
|
||
CodeRetrieval vector parity for English/German/Arabic/Bengali; actual tokenizer
|
||
count exactly 512 with vector parity, rejection at 513 followed by a successful
|
||
query; foreground indexing/media guards and full-context long-document windows.
|
||
Three bilingual ranking vectors had cosine 0.99999999 or better. This tiny
|
||
regression is correctness proof, not independent retrieval-quality evaluation.
|
||
The benchmark additionally sends a direct native document operation to the
|
||
query role, verifies rejection is nonfatal, and then performs CodeRetrieval.
|
||
|
||
Final worker rebuild after consolidating the query-size constant has SHA256
|
||
`1272797af94c21db1ccabaa3ad996ab57d64f1bf9d33280c61d65c51360deab7`.
|
||
Six actual CPU integration tests passed with this final binary: two code
|
||
fixtures (16.52 s), two text/debug/reference fixtures (59.18 s), image/audio
|
||
reference parity (52.20 s), and the query-context test again (26.05 s).
|
||
Image reference cosines were 0.99994150, 0.99977568 and 0.99917617; audio
|
||
reference cosine was 0.99975052. The full indexing/media role remains usable.
|
||
Native startup/memory values above belong to the explicitly identified earlier
|
||
binary, before this equivalent constant cleanup; they are not installer checks.
|
||
|
||
An actual CLI/MCP boundary regression also passed (36.36 s): the CLI indexed
|
||
three real document/code sources with the full role, CLI semantic search returned
|
||
the expected supplier file, and the retained MCP session reported query role,
|
||
512-token context and CPU. Both remembered-document and code-only searches
|
||
returned source handles whose guarded excerpts matched original text/line data.
|
||
Compiled media support remained visible to clients. This is a tiny integration
|
||
fixture, not a representative agent-library quality or performance benchmark.
|
||
|
||
Reproduce with verified assets and the current normal worker:
|
||
|
||
```powershell
|
||
$env:SEARCHI_FORCE_CPU = '1'
|
||
$env:SEARCHI_GEMMA_MODEL_DIR = 'PATH/TO/VERIFIED/MODELS'
|
||
$env:SEARCHI_GEMMA_WORKER = 'PATH/TO/searchi-gemma-worker.exe'
|
||
cargo test -p searchi-semantic --test query_context_cpu --locked -- --ignored --nocapture
|
||
cargo test -p searchi-cli --test mcp actual_google_cli_and_mcp_use_query_role_with_cited_results --locked -- --ignored --nocapture
|
||
python tools/embeddinggemma2/query_context_benchmark.py --worker "$env:SEARCHI_GEMMA_WORKER" --models "$env:SEARCHI_GEMMA_MODEL_DIR" --output .searchi/qa/query-context.json --repeats 3
|
||
```
|
||
|
||
The benchmark reports Windows private commitment/working set or Linux
|
||
PSS/private resident pages separately. Actual native Linux query/client checks
|
||
and six paired measurements also passed; see
|
||
[Linux evidence](LINUX-DESKTOP-VALIDATION.md). Linux resident/PSS memory was
|
||
effectively unchanged in those samples. Native macOS and actual GPU context
|
||
allocation/parity, pressure behavior, long code queries, representative query
|
||
latency and updated installed desktop packaging remain release checks.
|
||
|
||
## Current Windows CPU worker timing, 2026-10-13
|
||
|
||
The normal current worker (`c01cb3ec13cfca722ee768e8350e1399a8b9a4d14a24a35adc3473a5cfc82a73`)
|
||
was measured with CPU explicitly selected and one inference thread. Three paired
|
||
indexing/query processes alternate order; each retains a separate first request
|
||
and twenty distinct warmed inputs with native debug timings. Real 768-dimensional
|
||
vectors were checked. Output preservation now refuses existing reports.
|
||
|
||
| Query-role repeat | Process startup ms | First inference/IPC ms | Warm p95 ms (20 samples) |
|
||
| --- | ---: | ---: | ---: |
|
||
| 0 | 729.27 | 100.02 | 71.30 |
|
||
| 1 | 662.83 | 106.18 | 78.38 |
|
||
| 2 | 704.21 | 100.84 | 71.75 |
|
||
|
||
One-time text-model size/SHA verification took 192.45 ms and is separate from
|
||
worker startup. Indexing-role startup ranged 624.49–779.14 ms, first query
|
||
101.64–110.53 ms, warm p95 73.03–80.47 ms. These are cached-file native-worker
|
||
measurements, not cold disk, Rust query-cache, ANN, hybrid retrieval, actual UI
|
||
first input or paint. The installed desktop/query worker remained present;
|
||
background activity was not controlled, so this is not an idle reference-host
|
||
acceptance measurement. All test inference used CPU; no GPU provider was loaded.
|
||
|
||
Full ignored report: `.searchi/qa/query-context-warm-current.json`, SHA-256
|
||
`ea7dd0b72eeb1dbd79f06fce92e0475dfdc321d7781e5d06d58fe12755faf0fd`.
|
||
The query-role memory observation after first inference was 239,878,144 bytes
|
||
private commitment and 274,141,184 bytes resident working set for repeat zero;
|
||
these are different metrics and must not be summed. Native desktop first-input
|
||
and complete search latency remain separate open checks.
|
||
|
||
Benchmark contract hardening then replaced Python assertions with explicit
|
||
errors in both shared benchmark and query-context tooling. Model hashes,
|
||
CPU/device state, role/context, normalized vectors, source-window preservation,
|
||
MCP errors/filter behavior and batch parity remain checked under `python -O`.
|
||
Both tools refuse existing output reports. Four targeted invalid-vector/device
|
||
tests passed in normal and optimized modes. A genuine one-pair Google CPU
|
||
query/indexing run also passed under optimization, retaining twenty warmed
|
||
requests per role; this is a contract smoke rather than an additional latency
|
||
acceptance sample. Ignored report SHA-256:
|
||
`e611fd3c705a94a6c06ab6ae6a934e6f604665aacb17d0e86d6d5b162ac59e46`
|
||
(`.searchi/qa/query-context-optimized-contract.json`). Earlier recorded timing
|
||
reports were run without optimization and are unaffected by this correction.
|