Keep real model benchmark validity checks active under optimization
This commit is contained in:
1 parent
d568bdb2b1
commit
66bf7d7b47
4 files changed
+79
-27
No files matched your search
@@ -122,3 +122,16 @@ The query-role memory observation after first inference was 239,878,144 bytes
|
||||
private commitment and 274,141,184 bytes resident working set for repeat zero;
|
||||
these are different metrics and must not be summed. Native desktop first-input
|
||||
and complete search latency remain separate open checks.
|
||||
|
||||
Benchmark contract hardening then replaced Python assertions with explicit
|
||||
errors in both shared benchmark and query-context tooling. Model hashes,
|
||||
CPU/device state, role/context, normalized vectors, source-window preservation,
|
||||
MCP errors/filter behavior and batch parity remain checked under `python -O`.
|
||||
Both tools refuse existing output reports. Four targeted invalid-vector/device
|
||||
tests passed in normal and optimized modes. A genuine one-pair Google CPU
|
||||
query/indexing run also passed under optimization, retaining twenty warmed
|
||||
requests per role; this is a contract smoke rather than an additional latency
|
||||
acceptance sample. Ignored report SHA-256:
|
||||
`e611fd3c705a94a6c06ab6ae6a934e6f604665aacb17d0e86d6d5b162ac59e46`
|
||||
(`.searchi/qa/query-context-optimized-contract.json`). Earlier recorded timing
|
||||
reports were run without optimization and are unaffected by this correction.
|
||||
Reference in new issue
Block a user