Coordinate foreground search and background inference CPU budgets

This commit is contained in:
karl committed 2026-10-13 01:56:03 +02:00
1 parent 6b78c8bb7f
commit d85746db5b
12 files changed
+858 -60

No files matched your search

+28 -1
View File
@@ -181,4 +181,31 @@ The actual CPU regression checks both a repeated cached query and an unseen
query after publication; the unseen input must record query_cache_hit zero.
See [RECOVERY-AND-HANDOFF.md](RECOVERY-AND-HANDOFF.md). Its single-document
request times do not establish representative-corpus p95, native first input
or foreground priority during indexing. Shared CPU admission remains open.
or foreground priority during indexing.
## Shared inference admission
The desktop now shares one CPU thread budget across its persistent query and
indexing workers. The default is two threads, clamped to available logical CPUs
and a maximum of eight. Indexing uses one thread; the query worker uses the
remaining threads (at least one). Pending searches have priority over new
background encodes, and document indexing yields between individual encodes.
Cached query vectors do not acquire an inference permit.
Settings version two stores the total budget. Version-one settings counted
threads per worker; migration adds one background thread, capped at eight.
An explicit version-two budget of one remains one. Resource changes are rejected
during active semantic work; a failed settings save preserves the warm session.
Debug traces record `semantic.inference.admission_wait` separately from native
inference. Admission waits are bounded at 30 seconds. Already executing native
inference cannot be preempted: a one-thread budget or single-core machine can
still delay a search during a long media encode. This budget covers admitted
inference worker threads, not extraction, decoding, ANN construction, rendering
or total operating-system CPU usage.
The real Google CPU regression `budget_google_cpu` confirmed overlapping query
and background inference with a two-thread budget: one query thread, one indexing
thread, six background document calls, an unseen query taking 103.407 ms including
0.034 ms admission in the final scoped-cleanup run. These are Windows VM debug measurements on a small fixture,
not representative-corpus p95, native UI latency, GPU or scale evidence.