Preserve explicit code candidates and expose verified model asset milestones
This commit is contained in:
1 parent
6e05227f53
commit
e28e20af38
9 files changed
+475
-16
No files matched your search
@@ -219,3 +219,56 @@ process swap; their private `/dev/dxg` was the null device (1:3). Owned shutdown
|
||||
ended the app group with SIGTERM and Xvfb with status zero. Subsequent process
|
||||
inspection found no remaining app/worker/WebKit/Xvfb/DBus/clipboard helpers,
|
||||
and the outer `/dev/dxg` remained the real device (hex a:102).
|
||||
|
||||
## Native query-role validation: `5d9cfc4`
|
||||
|
||||
Executed on the same provisioned Ubuntu 24.04 x86-64 WSL host. The isolated
|
||||
normal `auto` worker was rebuilt with one job, retaining Vulkan and CPU modules.
|
||||
All inference used `SEARCHI_FORCE_CPU=1`; the benchmark additionally requested
|
||||
`--device cpu` and checked that no Vulkan module loaded. Windows staged resources
|
||||
were not changed. Worker SHA-256:
|
||||
`b52d50a1e676bd4bf64124daff1c4ce83d86a8bde01e4eaf09838cbb24a4c28f`.
|
||||
|
||||
Actual Google checks passed:
|
||||
|
||||
- `query_context_cpu`: one passed in **42.88 s**, covering query role/context
|
||||
negotiation, unchanged real vectors/ranking, multilingual CodeRetrieval,
|
||||
exact 512-token boundary, nonfatal oversized rejection, indexing/media guards
|
||||
and full-context document windows. Compilation took 1m 06s.
|
||||
- CLI/MCP `actual_google_cli_and_mcp_use_query_role_with_cited_results`: one
|
||||
passed in **38.73 s**, including real indexing with the full role, foreground
|
||||
CLI search, retained MCP query role/context and guarded source excerpts.
|
||||
|
||||
Logs remain in the native cache as `query-role-native-build.log`,
|
||||
`query-context-linux-test.log`, and `query-context-linux-mcp.log`. These tiny
|
||||
fixtures establish native role/client correctness, not representative relevance
|
||||
or a release package gate.
|
||||
|
||||
After correctness tests and other agents' downloads were terminal, a paired
|
||||
benchmark ran six fresh workers, alternating order, three per role, one inference
|
||||
thread. Cached model bytes were verified once in **241.48 ms** before process
|
||||
trials; process startup excludes that verification and desktop preparation.
|
||||
|
||||
| Median observed measurement | Indexing 2048 | Query 512 |
|
||||
| --- | ---: | ---: |
|
||||
| Startup to ready | 717.08 ms | 564.02 ms |
|
||||
| First query IPC/inference | 67.13 ms | 66.89 ms |
|
||||
| Ready PSS | 405,303,296 bytes | 405,278,720 bytes |
|
||||
| Ready RSS | 406,806,528 bytes | 406,773,760 bytes |
|
||||
| Ready private dirty resident pages | 103,739,392 bytes | 103,735,296 bytes |
|
||||
| After query PSS | 410,157,056 bytes | 410,001,408 bytes |
|
||||
|
||||
Linux resident/PSS observations were effectively unchanged. They must not be
|
||||
reported as the Windows private commitment saving; this benchmark does not
|
||||
measure Linux virtual commitment. The startup samples are small cached-process
|
||||
observations, not cold disk, actual first UI input, pressure behavior or warm p95
|
||||
proof. Mapped model pages contribute to resident/PSS figures; do not sum them
|
||||
as independent model allocations across workers. Raw paired samples are retained
|
||||
locally in `.searchi/qa/query-context-linux.json`; native benchmark output is
|
||||
`query-context-linux-benchmark.log`. The benchmark itself also checked nonfatal
|
||||
rejection of a direct indexing operation followed by a successful code query.
|
||||
|
||||
All owned test and benchmark handles ended successfully. Subsequent process
|
||||
inspection found no surviving Google workers, Cargo or Rust compiler processes.
|
||||
Native desktop packaging with this new worker, actual GPU, macOS and
|
||||
representative performance remain open.
|
||||
@@ -105,8 +105,10 @@ restart or substituted model was used. Reproduce with the
|
||||
|
||||
## Open quality gates
|
||||
|
||||
- Correct the documented All-files audio-intent suppression without hiding
|
||||
legitimate media results; validate candidate preservation before ranking.
|
||||
- Validate the subsequent explicit-code routing fix on broader mixed queries
|
||||
without hiding legitimate media results. A tiny real-model candidate-preservation
|
||||
regression passed; see [routing evidence](RETRIEVAL-ROUTING.md). The frozen
|
||||
baseline recorded here remains the pre-fix measurement.
|
||||
- Evaluate general/code query task fusion on a new independently labelled set,
|
||||
with bounded retrieval and measured cache/filter costs.
|
||||
- Improve music discrimination and precise speech timestamps.
|
||||
|
||||
@@ -87,6 +87,9 @@ python tools/embeddinggemma2/query_context_benchmark.py --worker "$env:SEARCHI_G
|
||||
```
|
||||
|
||||
The benchmark reports Windows private commitment/working set or Linux
|
||||
PSS/private resident pages separately. Native Linux/macOS and actual GPU context
|
||||
PSS/private resident pages separately. Actual native Linux query/client checks
|
||||
and six paired measurements also passed; see
|
||||
[Linux evidence](LINUX-DESKTOP-VALIDATION.md). Linux resident/PSS memory was
|
||||
effectively unchanged in those samples. Native macOS and actual GPU context
|
||||
allocation/parity, pressure behavior, long code queries, representative query
|
||||
latency and updated installed desktop packaging remain release checks.
|
||||
@@ -55,3 +55,25 @@ The existing `semantic.eligibility` and
|
||||
`semantic.eligibility_filtered_scan_and_hash` traces expose eligibility cost.
|
||||
Use an independently reviewed held-out corpus for acceptance; the frozen
|
||||
diagnostic alone must not become the optimization target.
|
||||
|
||||
## Narrow suppression fix and actual regression
|
||||
|
||||
After the frozen baseline confirmed the English speech-related code failure,
|
||||
explicit code/function/implementation cues now veto automatic audio preference,
|
||||
alongside the existing document/image cues. English and German cues are covered
|
||||
by the routing unit test. This changes candidate eligibility through routing;
|
||||
it does not add CodeRetrieval to All files or implement soft multi-lane fusion.
|
||||
|
||||
The ignored integration `routing_google_cpu` passed on Windows CPU in **38.40 s**
|
||||
using the verified normal worker and Google model, unchanged frozen routing
|
||||
source bytes and the licensed real roaring-creature sound. The same remembered
|
||||
code query produced qualifying audio candidates through the explicit Audio
|
||||
diagnostic lane. Updated default semantic routing also returned the source
|
||||
implementation, proving the exclusive suppression is removed in this fixture.
|
||||
The test asserts candidate preservation, not top-1 superiority or broad quality.
|
||||
The full mixed baseline above remains the unchanged pre-fix measurement.
|
||||
|
||||
With this fix and the verified-asset callback, the Windows CPU workspace suite
|
||||
passed 130 tests (23 explicitly ignored integration/resource checks), desktop
|
||||
Rust passed 28, and both strict all-target Clippy checks and formatting passed.
|
||||
The ignored actual routing regression was executed separately as described above.
|
||||
+9
-1
@@ -186,9 +186,15 @@ Audit and measured priorities: [EMBEDDINGGEMMA2-AUDIT.md](EMBEDDINGGEMMA2-AUDIT.
|
||||
integration and actual Windows CPU parity/boundary/reference tests.
|
||||
Paired samples reduced private commitment by 183 MB; resident memory and
|
||||
cached startup were essentially unchanged. See [QUERY-CONTEXT.md](QUERY-CONTEXT.md).
|
||||
- [ ] Verify native Linux/macOS, GPU parity and updated installed packages.
|
||||
- [x] Native Linux query-context and CLI/MCP cited-result checks, with six
|
||||
paired CPU process measurements; resident/PSS memory effectively unchanged.
|
||||
- [ ] Verify native macOS, GPU parity and updated installed packages.
|
||||
- [ ] Evaluate text readiness before the larger media projector finishes
|
||||
downloading; preserve verified assets and honest media setup status.
|
||||
- [x] Typed backbone/projector verification callbacks with full-pair setup
|
||||
semantics preserved; 16 asset tests passed, including gated media download.
|
||||
- [ ] Integrate explicit text/media coverage, retained query readiness and
|
||||
media retry status across engine/desktop; verify actual first use.
|
||||
- [ ] Buffered bounded media IPC and linear UTF-8 source-window offset processing.
|
||||
- [ ] Evaluate 256d MRL and content-specific vision budgets on independent data.
|
||||
- [ ] Validate GPU precision, synchronization-aware timings, parity and fallback
|
||||
@@ -219,6 +225,8 @@ Completion requires recorded evidence; targets are not measured results.
|
||||
returned candidates. See [mixed quality](MIXED-QUALITY.md).
|
||||
- [ ] Measure task fusion and cache cost against independently reviewed
|
||||
held-out data, preserving the frozen diagnostic for regression comparison.
|
||||
- [x] Explicit English/German code intent vetoes exclusive audio routing;
|
||||
actual Google CPU regression preserves code among qualifying audio candidates.
|
||||
- [ ] Validate code search UI interaction on native Windows/macOS/Linux and
|
||||
actual GPU hardware. Tiny CPU fixtures do not close those release gates.
|
||||
|
||||
|
||||
@@ -6,8 +6,9 @@ Google EmbeddingGemma 2 remains the production semantic model.
|
||||
|
||||
A bounded [CPU diagnostic harness](../tools/speech-eval/README.md) is now prepared.
|
||||
Seven structural tests passed on Windows, including actual owned subprocess
|
||||
failure, timeout, output overflow and retained diagnostics. No ASR model has been
|
||||
downloaded or run for this proof; these tests establish no speech quality.
|
||||
failure, timeout, output overflow and retained diagnostics. Both candidate models
|
||||
have now been downloaded and their exact bytes/SHA verified. Neither has been
|
||||
loaded or evaluated; these tests establish no speech quality.
|
||||
|
||||
## Why add a transcript lane
|
||||
|
||||
@@ -60,9 +61,13 @@ Model metadata was read from the Hugging Face blob API at repository revision
|
||||
|
||||
Use immutable download URLs of the form
|
||||
`https://huggingface.co/ggerganov/whisper.cpp/resolve/5359861c739e955e79d9a303bcbc70fb988958b1/ggml-base.bin`.
|
||||
These are **publisher metadata**, not hashes verified from downloaded bytes yet.
|
||||
Commit a manifest only after a genuine download verifies exact size/SHA, model
|
||||
loading and provenance. Never accept upstream `for-tests-*` empty models.
|
||||
Both downloads were independently checked against these exact bytes/SHA values
|
||||
on 2026-10-13. The clean runtime checkout resolves to the pinned commit; model
|
||||
URLs, verified hashes and MIT notices are retained under ignored
|
||||
`.searchi/speech/provisioning.json`. This is provisioning evidence only: no
|
||||
runtime build, model loading or inference has yet been performed. Commit an
|
||||
accepted distribution manifest only after model loading and runtime provenance
|
||||
checks. Never accept upstream `for-tests-*` empty models.
|
||||
[Upstream model provenance](https://github.com/ggml-org/whisper.cpp/blob/master/models/README.md),
|
||||
[blob metadata](https://huggingface.co/api/models/ggerganov/whisper.cpp?blobs=true).
|
||||
|
||||
|
||||
Reference in new issue
Block a user