Coordinate foreground search and background inference CPU budgets
This commit is contained in:
1 parent
6b78c8bb7f
commit
d85746db5b
12 files changed
+858
-60
No files matched your search
+28
-1
@@ -181,4 +181,31 @@ The actual CPU regression checks both a repeated cached query and an unseen
|
||||
query after publication; the unseen input must record query_cache_hit zero.
|
||||
See [RECOVERY-AND-HANDOFF.md](RECOVERY-AND-HANDOFF.md). Its single-document
|
||||
request times do not establish representative-corpus p95, native first input
|
||||
or foreground priority during indexing. Shared CPU admission remains open.
|
||||
or foreground priority during indexing.
|
||||
|
||||
## Shared inference admission
|
||||
|
||||
The desktop now shares one CPU thread budget across its persistent query and
|
||||
indexing workers. The default is two threads, clamped to available logical CPUs
|
||||
and a maximum of eight. Indexing uses one thread; the query worker uses the
|
||||
remaining threads (at least one). Pending searches have priority over new
|
||||
background encodes, and document indexing yields between individual encodes.
|
||||
Cached query vectors do not acquire an inference permit.
|
||||
|
||||
Settings version two stores the total budget. Version-one settings counted
|
||||
threads per worker; migration adds one background thread, capped at eight.
|
||||
An explicit version-two budget of one remains one. Resource changes are rejected
|
||||
during active semantic work; a failed settings save preserves the warm session.
|
||||
|
||||
Debug traces record `semantic.inference.admission_wait` separately from native
|
||||
inference. Admission waits are bounded at 30 seconds. Already executing native
|
||||
inference cannot be preempted: a one-thread budget or single-core machine can
|
||||
still delay a search during a long media encode. This budget covers admitted
|
||||
inference worker threads, not extraction, decoding, ANN construction, rendering
|
||||
or total operating-system CPU usage.
|
||||
|
||||
The real Google CPU regression `budget_google_cpu` confirmed overlapping query
|
||||
and background inference with a two-thread budget: one query thread, one indexing
|
||||
thread, six background document calls, an unseen query taking 103.407 ms including
|
||||
0.034 ms admission in the final scoped-cleanup run. These are Windows VM debug measurements on a small fixture,
|
||||
not representative-corpus p95, native UI latency, GPU or scale evidence.
|
||||
@@ -77,8 +77,9 @@ are not certified by those smoke tests.
|
||||
|
||||
## Remaining performance/release work
|
||||
|
||||
Background and query workers still receive independent thread budgets; shared
|
||||
CPU admission and foreground priority are open. Publication no longer charges
|
||||
Background and query workers now share inference admission and foreground
|
||||
priority; see [PERFORMANCE.md](PERFORMANCE.md#shared-inference-admission).
|
||||
Publication no longer charges
|
||||
foreground graph deserialization, but eligibility refresh, indexing I/O and
|
||||
CPU contention need representative measurements. Native compositor/input timing,
|
||||
large-corpus p95, older CPUs, actual GPU/device-loss, power-loss durability,
|
||||
@@ -97,4 +98,4 @@ installers/upgrades/signatures and macOS packaging remain open.
|
||||
The prepared graph handle is process-local: foreign CLI/MCP index writers cannot
|
||||
share its native Arc. Cold CLI initialization and cross-process reader refresh
|
||||
remain separate costs. Representative concurrent-indexing/foreground measurements
|
||||
and shared CPU thread admission remain the next performance milestone.
|
||||
across decoder, extraction and ANN work remain the next performance milestone.
|
||||
@@ -164,6 +164,11 @@ Audit and measured priorities: [EMBEDDINGGEMMA2-AUDIT.md](EMBEDDINGGEMMA2-AUDIT.
|
||||
See [INCREMENTAL-SEMANTIC.md](INCREMENTAL-SEMANTIC.md).
|
||||
- [ ] Benchmark genuine token-budgeted multi-sequence batching and isolation.
|
||||
- [ ] Implement and measure shared foreground/indexer resource budgets/priority.
|
||||
- [x] Shared inference admission, foreground priority between background
|
||||
encodes, total-thread settings migration and retained warm sessions on save
|
||||
failure; real Google CPU overlap regression passed on Windows.
|
||||
- [ ] Measure representative mixed-media contention, single-thread latency,
|
||||
extraction/decoder/ANN work and native UI p95 before closing this gate.
|
||||
- [ ] Attribute startup cost; measure smaller query context and selective encoders.
|
||||
- [ ] Buffered bounded media IPC and linear UTF-8 source-window offset processing.
|
||||
- [ ] Evaluate 256d MRL and content-specific vision budgets on independent data.
|
||||
|
||||
Reference in new issue
Block a user