Measure genuine CPU query startup first input and warm latency

This commit is contained in:
karl committed 2026-10-13 13:53:03 +02:00
1 parent 6e7616b6d3
commit d568bdb2b1
2 files changed
+43 -1

No files matched your search

+29
View File
@@ -93,3 +93,32 @@ and six paired measurements also passed; see
effectively unchanged in those samples. Native macOS and actual GPU context
allocation/parity, pressure behavior, long code queries, representative query
latency and updated installed desktop packaging remain release checks.
## Current Windows CPU worker timing, 2026-10-13
The normal current worker (`c01cb3ec13cfca722ee768e8350e1399a8b9a4d14a24a35adc3473a5cfc82a73`)
was measured with CPU explicitly selected and one inference thread. Three paired
indexing/query processes alternate order; each retains a separate first request
and twenty distinct warmed inputs with native debug timings. Real 768-dimensional
vectors were checked. Output preservation now refuses existing reports.
| Query-role repeat | Process startup ms | First inference/IPC ms | Warm p95 ms (20 samples) |
| --- | ---: | ---: | ---: |
| 0 | 729.27 | 100.02 | 71.30 |
| 1 | 662.83 | 106.18 | 78.38 |
| 2 | 704.21 | 100.84 | 71.75 |
One-time text-model size/SHA verification took 192.45 ms and is separate from
worker startup. Indexing-role startup ranged 624.49–779.14 ms, first query
101.64–110.53 ms, warm p95 73.03–80.47 ms. These are cached-file native-worker
measurements, not cold disk, Rust query-cache, ANN, hybrid retrieval, actual UI
first input or paint. The installed desktop/query worker remained present;
background activity was not controlled, so this is not an idle reference-host
acceptance measurement. All test inference used CPU; no GPU provider was loaded.
Full ignored report: `.searchi/qa/query-context-warm-current.json`, SHA-256
`ea7dd0b72eeb1dbd79f06fce92e0475dfdc321d7781e5d06d58fe12755faf0fd`.
The query-role memory observation after first inference was 239,878,144 bytes
private commitment and 274,141,184 bytes resident working set for repeat zero;
these are different metrics and must not be summed. Native desktop first-input
and complete search latency remain separate open checks.