feat: add Google code retrieval across desktop CLI and MCP
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
Core CI / desktop (macos-latest) (push) Canceled after 0s
Core CI / desktop (ubuntu-latest) (push) Canceled after 0s
Core CI / desktop (windows-latest) (push) Canceled after 0s
Core CI / core (ubuntu-latest) (push) Canceled after 2m23s
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
Core CI / desktop (macos-latest) (push) Canceled after 0s
Core CI / desktop (ubuntu-latest) (push) Canceled after 0s
Core CI / desktop (windows-latest) (push) Canceled after 0s
Core CI / core (ubuntu-latest) (push) Canceled after 2m23s
This commit is contained in:
1 parent
8b9b079469
commit
81846b3e98
32 files changed
+1303
-59
No files matched your search
@@ -0,0 +1,78 @@
|
||||
# Source-code relevance checkpoint
|
||||
|
||||
Recorded 2026-10-12 using the actual verified Google EmbeddingGemma 2 Q8_0
|
||||
model, rebuilt native worker, shared hybrid service and MCP protocol. Windows
|
||||
VM, CPU only, one inference thread. No simulated provider or injected vectors.
|
||||
|
||||
Google describes local code retrieval in its
|
||||
[launch blog](https://blog.google/innovation-and-ai/technology/developers-tools/embeddinggemma-2/).
|
||||
Its [model card](https://ai.google.dev/gemma/docs/embeddinggemma/model_card_2)
|
||||
specifies `task: code retrieval | query: {query}` for queries and
|
||||
`title: {filename} | text: {code}` for indexed source. searchi uses that pairing
|
||||
in Code scope. Google's published benchmark measures their full-precision
|
||||
checkpoint; it is not a searchi benchmark.
|
||||
|
||||
## Frozen small regression
|
||||
|
||||
[The evaluator](../tools/code-eval/evaluate.py) fixes the implementations,
|
||||
queries and expected files before inference. Eight short source files use
|
||||
generic filenames; a ninth ordinary document deliberately contains search
|
||||
terms. All nine files are indexed, so code filtering must exclude that document
|
||||
before ranking. The expected code excerpts are checked byte-for-byte through
|
||||
MCP `read_source`, including line locations. Fixture hashes and every returned
|
||||
rank are recorded in [raw CPU evidence](evidence/code-search-cpu.json).
|
||||
|
||||
| Query intent | Source | Lexical rank | Hybrid rank |
|
||||
| --- | --- | --- | --- |
|
||||
| Streaming file SHA256 | Python | absent | 1 |
|
||||
| Lower bound in sorted numbers | C++ | absent | 1 |
|
||||
| Sum amounts by category | TypeScript | absent | 1 |
|
||||
| Retry requests with exponential backoff | Python | absent | 1 |
|
||||
| Release semaphore after async failure | C# | absent | 1 |
|
||||
| Escape unsafe HTML characters | PHP | absent | 1 |
|
||||
| Ten most expensive records | Python | absent | 1 |
|
||||
| File verification without full memory load, German | Python | absent | 1 |
|
||||
| Exact `refreshAccessToken` symbol | Rust | 1 | 1 |
|
||||
|
||||
Top-1 success: **9/9 hybrid, 1/9 lexical**. Every expected hybrid source was
|
||||
verified and no ordinary document leaked into Code scope. Measured MCP
|
||||
roundtrips for these nine uncached code queries ranged from **63.88–228.21 ms**,
|
||||
median **110.76 ms**. Those timings exclude process/model startup and describe
|
||||
this tiny corpus only; they are not desktop first-use or large-project targets.
|
||||
|
||||
## Defect discovered and fixed
|
||||
|
||||
The first actual indexing attempt failed on TypeScript: Google's tokenizer
|
||||
normalized source characters, invalidating a detokenization-based byte-offset
|
||||
assumption. The worker now retains original UTF-8 source, checks the complete
|
||||
prefixed token budget, and splits normalized long inputs on original character
|
||||
boundaries. It returns exact original slices with bounded overlap and complete
|
||||
tail coverage. Short code that fits uses a direct fast path. A separate actual
|
||||
CPU regression covers the failing TypeScript plus roughly 38 KB of repeated
|
||||
code with German, Japanese and emoji. Fixtures were not edited to hide failure.
|
||||
|
||||
Additional checks cover strict filtering before lexical/ANN candidate limits,
|
||||
named build files, overlapping routine context, schema upgrades, task-specific
|
||||
query caching, actual code/general vector separation and debug-vector parity.
|
||||
|
||||
## Reproduce and limits
|
||||
|
||||
Build the CLI and provision the pinned CPU worker/model as documented in
|
||||
[BUILDING.md](BUILDING.md) and [SEMANTIC.md](SEMANTIC.md), then run:
|
||||
|
||||
```powershell
|
||||
$env:SEARCHI_FORCE_CPU = '1'
|
||||
python tools/code-eval/evaluate.py
|
||||
```
|
||||
|
||||
The script uses the repository's generated `.searchi/code-eval` folder and
|
||||
default local runtime/model paths. It does not index private project folders.
|
||||
Outputs stay in that generated folder until deliberately reviewed and copied
|
||||
into documentation evidence.
|
||||
|
||||
These are synthetic, short implementations with one intended answer per query.
|
||||
They do not prove general project accuracy, declaration-first ranking,
|
||||
call-graph understanding or reliable no-answer detection. Extraction is
|
||||
line-based, not an AST parser. Independent real repositories, difficult
|
||||
near-duplicates, more languages, native UI interaction, other operating systems
|
||||
and actual GPU execution remain open evaluation gates.
|
||||
Reference in new issue
Block a user