93 lines
5.5 KiB
Markdown
93 lines
5.5 KiB
Markdown
# Foreground query context
|
|
|
|
The normal hardware-detecting Google worker now accepts `--role query`.
|
|
That role allocates 512 tokens for `n_ctx`, `n_batch` and `n_ubatch`, matching
|
|
searchi's existing query token limit. The default `indexing` role retains all
|
|
2048 tokens and document/image/audio operations. This is one binary, not a
|
|
separate CPU distribution. Hardware selection and CPU fallback are unchanged.
|
|
|
|
Desktop foreground sessions, CLI semantic searches and the retained MCP search
|
|
session use `HybridSearch::open_query`. Background document/media indexing
|
|
continues to use `open`. Role is retained when a failed GPU process is restarted
|
|
as CPU. A query-role worker rejects indexing operations before Rust verifies a
|
|
projector or decodes images, and the native protocol independently rejects them.
|
|
Invalid oversized queries leave the session usable.
|
|
|
|
The handshake reports `role` and `context_tokens`. A legacy worker cannot
|
|
silently claim the smaller allocation: update/rebuild the worker with the app.
|
|
`media_supported` still describes the installed binary's compiled capability;
|
|
it remains true for query sessions so desktop status does not incorrectly hide
|
|
media indexing. It does not mean the foreground session loads the projector.
|
|
Query vectors and model/index fingerprints remain unchanged.
|
|
|
|
## Actual Windows CPU evidence, 2026-10-13
|
|
|
|
Normal Windows x64 CPU/Vulkan worker, pinned llama b11461-no-logits-v1,
|
|
verified Google Q8_0 backbone, CPU forced and no Vulkan module loaded. Windows
|
|
11 build 26300 VM, 8 logical processors, one inference thread. Six fresh worker
|
|
processes, three per role, alternating order; cached filesystem/model bytes.
|
|
These are small paired allocation measurements, not cold-disk, UI or p95 proof.
|
|
|
|
| Median measurement | Full 2048 | Query 512 |
|
|
| --- | ---: | ---: |
|
|
| Startup to ready |716.69 ms|710.10 ms|
|
|
| Ready private commitment |421,359,616 bytes|238,362,624 bytes|
|
|
| Ready resident working set |118,394,880 bytes|118,276,096 bytes|
|
|
| After first query private commitment |422,735,872 bytes|239,734,784 bytes|
|
|
| After first query resident working set |274,632,704 bytes|273,989,632 bytes|
|
|
| First query IPC/inference |106.57 ms|103.20 ms|
|
|
|
|
Private commitment decreased by about 183 MB (43.4%). Resident memory and startup
|
|
were effectively unchanged in this sample. Do not advertise an 183 MB RSS saving
|
|
or a search-speed improvement from these results. Model verification took 195.86 ms
|
|
once before the process trials; startup excludes Rust verification and desktop
|
|
background preparation. Working sets include mapped shared model pages.
|
|
|
|
The measured worker SHA256 was
|
|
`3bd88fdb26f5d62d400ed101277f637af028c757c619547f6333b2912137c60b`.
|
|
Raw samples are retained locally at `.searchi/qa/query-context-windows.json`.
|
|
|
|
The actual Google integration `query_context_cpu` passed in 27.39 s:
|
|
English/German remembered-document ranking unchanged across both contexts;
|
|
CodeRetrieval vector parity for English/German/Arabic/Bengali; actual tokenizer
|
|
count exactly 512 with vector parity, rejection at 513 followed by a successful
|
|
query; foreground indexing/media guards and full-context long-document windows.
|
|
Three bilingual ranking vectors had cosine 0.99999999 or better. This tiny
|
|
regression is correctness proof, not independent retrieval-quality evaluation.
|
|
The benchmark additionally sends a direct native document operation to the
|
|
query role, verifies rejection is nonfatal, and then performs CodeRetrieval.
|
|
|
|
Final worker rebuild after consolidating the query-size constant has SHA256
|
|
`1272797af94c21db1ccabaa3ad996ab57d64f1bf9d33280c61d65c51360deab7`.
|
|
Six actual CPU integration tests passed with this final binary: two code
|
|
fixtures (16.52 s), two text/debug/reference fixtures (59.18 s), image/audio
|
|
reference parity (52.20 s), and the query-context test again (26.05 s).
|
|
Image reference cosines were 0.99994150, 0.99977568 and 0.99917617; audio
|
|
reference cosine was 0.99975052. The full indexing/media role remains usable.
|
|
Native startup/memory values above belong to the explicitly identified earlier
|
|
binary, before this equivalent constant cleanup; they are not installer checks.
|
|
|
|
An actual CLI/MCP boundary regression also passed (36.36 s): the CLI indexed
|
|
three real document/code sources with the full role, CLI semantic search returned
|
|
the expected supplier file, and the retained MCP session reported query role,
|
|
512-token context and CPU. Both remembered-document and code-only searches
|
|
returned source handles whose guarded excerpts matched original text/line data.
|
|
Compiled media support remained visible to clients. This is a tiny integration
|
|
fixture, not a representative agent-library quality or performance benchmark.
|
|
|
|
Reproduce with verified assets and the current normal worker:
|
|
|
|
```powershell
|
|
$env:SEARCHI_FORCE_CPU = '1'
|
|
$env:SEARCHI_GEMMA_MODEL_DIR = 'PATH/TO/VERIFIED/MODELS'
|
|
$env:SEARCHI_GEMMA_WORKER = 'PATH/TO/searchi-gemma-worker.exe'
|
|
cargo test -p searchi-semantic --test query_context_cpu --locked -- --ignored --nocapture
|
|
cargo test -p searchi-cli --test mcp actual_google_cli_and_mcp_use_query_role_with_cited_results --locked -- --ignored --nocapture
|
|
python tools/embeddinggemma2/query_context_benchmark.py --worker "$env:SEARCHI_GEMMA_WORKER" --models "$env:SEARCHI_GEMMA_MODEL_DIR" --output .searchi/qa/query-context.json --repeats 3
|
|
```
|
|
|
|
The benchmark reports Windows private commitment/working set or Linux
|
|
PSS/private resident pages separately. Native Linux/macOS and actual GPU context
|
|
allocation/parity, pressure behavior, long code queries, representative query
|
|
latency and updated installed desktop packaging remain release checks.
|