Coordinate foreground search and background inference CPU budgets
This commit is contained in:
1 parent
6b78c8bb7f
commit
d85746db5b
12 files changed
+858
-60
No files matched your search
+28
-1
@@ -181,4 +181,31 @@ The actual CPU regression checks both a repeated cached query and an unseen
|
||||
query after publication; the unseen input must record query_cache_hit zero.
|
||||
See [RECOVERY-AND-HANDOFF.md](RECOVERY-AND-HANDOFF.md). Its single-document
|
||||
request times do not establish representative-corpus p95, native first input
|
||||
or foreground priority during indexing. Shared CPU admission remains open.
|
||||
or foreground priority during indexing.
|
||||
|
||||
## Shared inference admission
|
||||
|
||||
The desktop now shares one CPU thread budget across its persistent query and
|
||||
indexing workers. The default is two threads, clamped to available logical CPUs
|
||||
and a maximum of eight. Indexing uses one thread; the query worker uses the
|
||||
remaining threads (at least one). Pending searches have priority over new
|
||||
background encodes, and document indexing yields between individual encodes.
|
||||
Cached query vectors do not acquire an inference permit.
|
||||
|
||||
Settings version two stores the total budget. Version-one settings counted
|
||||
threads per worker; migration adds one background thread, capped at eight.
|
||||
An explicit version-two budget of one remains one. Resource changes are rejected
|
||||
during active semantic work; a failed settings save preserves the warm session.
|
||||
|
||||
Debug traces record `semantic.inference.admission_wait` separately from native
|
||||
inference. Admission waits are bounded at 30 seconds. Already executing native
|
||||
inference cannot be preempted: a one-thread budget or single-core machine can
|
||||
still delay a search during a long media encode. This budget covers admitted
|
||||
inference worker threads, not extraction, decoding, ANN construction, rendering
|
||||
or total operating-system CPU usage.
|
||||
|
||||
The real Google CPU regression `budget_google_cpu` confirmed overlapping query
|
||||
and background inference with a two-thread budget: one query thread, one indexing
|
||||
thread, six background document calls, an unseen query taking 103.407 ms including
|
||||
0.034 ms admission in the final scoped-cleanup run. These are Windows VM debug measurements on a small fixture,
|
||||
not representative-corpus p95, native UI latency, GPU or scale evidence.
|
||||
Reference in new issue
Block a user