feat: add multimodal search and granular performance diagnostics
Core CI / core (ubuntu-latest) (push) Failing after 3m31s
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
Core CI / desktop (ubuntu-latest) (push) Successful in 5m39s
Core CI / desktop (macos-latest) (push) Canceled after 0s
Core CI / desktop (windows-latest) (push) Canceled after 0s
Core CI / core (ubuntu-latest) (push) Failing after 3m31s
Core CI / core (macos-latest) (push) Canceled after 0s
Core CI / core (windows-latest) (push) Canceled after 0s
Core CI / desktop (ubuntu-latest) (push) Successful in 5m39s
Core CI / desktop (macos-latest) (push) Canceled after 0s
Core CI / desktop (windows-latest) (push) Canceled after 0s
This commit is contained in:
1 parent
d3bb8c7618
commit
b8400c4fbf
110 files changed
+79395
-250
No files matched your search
+84
-54
@@ -1,64 +1,94 @@
|
||||
# Validation evidence
|
||||
|
||||
Development host: Windows VM, x86_64 MSVC, Rust stable 1.99.0.
|
||||
All local tests run on CPU; no GPU provider is configured.
|
||||
Development host: Windows VM, x86_64 MSVC, stable Rust 1.99.0. All local
|
||||
inference, FFmpeg decoding and WebView tests use CPU only. UI launches set
|
||||
`WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS='--disable-gpu --disable-accelerated-video-decode'`
|
||||
and `SEARCHI_FORCE_CPU=1`.
|
||||
|
||||
## Current integration checkpoint (2026-10-12)
|
||||
|
||||
- `cargo test --workspace --locked`: 41 default tests passed after final core integration. Four asset-dependent
|
||||
or explicit benchmark tests remain opt-in; a default pass does not run them.
|
||||
- Native desktop tests: 7 passed with SEARCHI_FORCE_CPU=1 and WebView2 GPU disabled.
|
||||
- Frontend: 7 async-state tests passed, Prettier passed, TypeScript/Vite build passed.
|
||||
- Genuine Google CPU reference/window test passed separately: five DigUp reference
|
||||
vectors had cosine similarity above 0.999; actual multilingual retrieval and
|
||||
tokenizer tail-window coverage passed. MiniLM evidence below is historical.
|
||||
- Repository description/topics were read back after update; the branded avatar
|
||||
upload returned HTTP 204 and the repository API returned its avatar URL.
|
||||
- New desktop styling is in progress. `design-qa.md` records the visual gate as
|
||||
blocked because browser bootstrap and native screen capture are unavailable.
|
||||
No visual match or complete UI interaction verification is claimed.
|
||||
- Native desktop: **9/9 tests passed**, including structured trace/basic responses.
|
||||
Post-generation cached query warmup is implemented but its interactive first-query gate remains open.
|
||||
Frontend: **18/18 tests passed**, TypeScript/Vite production build and
|
||||
formatting passed. Preview policy tests cover fractional timestamp seeking,
|
||||
image labels, one active/latest pending request, stale replies and Escape.
|
||||
- Semantic adapter: **8 default tests passed**; the explicit 48 MP resource test
|
||||
remains opt-in. Strict targeted all-target Clippy and formatting passed.
|
||||
Protocol bounds, invalid vectors, artifact size/checksum mismatch, old-worker
|
||||
capability defaults and oversized JPEG rejection are covered.
|
||||
- The CPU native helper rebuilt and staged successfully with genuine Google
|
||||
MTMD image/audio support. Its handshake reports `media_supported:true`;
|
||||
older text workers default false rather than claiming media readiness.
|
||||
- Actual Google CPU media parity passed separately in **52.85 seconds**:
|
||||
screenshot 280 cosine 0.99994194, photo 140: 0.99974583, frame 70: 0.99921928 and
|
||||
speech: 0.99983318 against pinned MIT DigUp vectors. All exceed the 0.999 gate.
|
||||
This verifies runtime parity, not broad retrieval quality.
|
||||
- The release resource test encodes/decodes an actual 8000x6000 JPEG with EXIF
|
||||
rotation and verifies region colors after scaled decoding/resizing. Decode
|
||||
plus resize took 176–220 ms. A sampled whole-test peak was 146.86 MiB, including
|
||||
source fixture generation; it is not decoder-only RSS or neural quality.
|
||||
- The independent production media pipeline has run on 20 real assets/65 frozen
|
||||
queries using actual Google inference, FFmpeg, metadata, persistence and HNSW.
|
||||
It encoded 20/20 files with no failures in the recorded 30-second-window run.
|
||||
Broad file recall is promising; precise speech timestamps, music confusion
|
||||
and no-answer behavior remain failed quality gates. Preserve the earlier
|
||||
direct harness failure separately. [Raw results and limitations](MEDIA-QUALITY.md).
|
||||
- Actual document seed regression passed 24/24 semantic/hybrid queries. This is
|
||||
a small synthetic regression, not a production relevance corpus.
|
||||
- The explicit 100,000-vector release ANN diagnostic passed 98.5% recall@10,
|
||||
HNSW p95 1.37 ms and warm service p95 6.40 ms with zero warm passage hashes.
|
||||
Vectors are synthetic index benchmark data, never production embeddings.
|
||||
- The fixed one-million-file first-input report records 40 inputs below 50 ms in
|
||||
that run: [recorded benchmark](../crates/searchi-core/benchmarks/results/windows-cpu-full-candidate-first-final.json).
|
||||
These measurements do not establish every query or concurrent-indexing goal.
|
||||
|
||||
Final UI integration: the branded compact launcher, local Inter/Fluent assets,
|
||||
themes and motion implementation passed frontend build, formatting, and seven
|
||||
state tests. Native launch succeeded; accessibility text showed real controls
|
||||
and a warmed Google CPU worker. UI input and screenshots remain blocked by
|
||||
Windows access/capture errors, as recorded in design-qa.md. Workspace formatting
|
||||
and strict all-target Clippy passed. The fixed full-corpus first-input report
|
||||
is in crates/searchi-core/benchmarks/results/windows-cpu-full-candidate-first-final.json;
|
||||
all 40 measured filename inputs were below 50 ms in that run.
|
||||
The integrated CPU workspace checkpoint passed **62 default tests** (8 assets, 6 CLI, 23 core, 4 media, 8 semantic, 13 service), with **10 explicit opt-in tests ignored**. Workspace and native all-target Clippy passed with warnings denied. The later semantic CPU-restart counter change passed workspace Clippy and the actual CPU query diagnostics test. Earlier aggregate counts are historical.
|
||||
|
||||
The active parallel development branches of work may add tests or change counts
|
||||
after this checkpoint. Run the documented checks again after integration.
|
||||
The updated pinned C++ helper compiled in an independent CPU build directory and
|
||||
was staged for desktop resources without interrupting media evaluation. Actual
|
||||
Google query diagnostics passed separately in 18.09 seconds including model
|
||||
initialization and two queries: 13 tokens, tokenization 149 us, batch preparation
|
||||
22 us, inference 112914 us, pooling/normalization 17 us. Rust worker wait/response
|
||||
decode measured 113960 us. Debug/normal embedding cosine was 0.99999999. These
|
||||
are one diagnostic sample under VM contention, not production latency evidence.
|
||||
See [performance boundaries and raw overhead samples](PERFORMANCE.md).
|
||||
|
||||
## Historical foundation evidence
|
||||
## Native UI and platform limits
|
||||
|
||||
Initial foundation checks (2026-10-12):
|
||||
- `cargo test --workspace`: CLI parser/executable checks and engine retrieval,
|
||||
lifecycle, exclusions, filters, migration and concurrent-reader checks passed.
|
||||
- `cargo clippy --workspace --all-targets -- -D warnings`: passed.
|
||||
- Rust formatting applied; final formatting verification follows integration.
|
||||
- Desktop TypeScript/Vite build passed; npm audit reported zero vulnerabilities.
|
||||
- Initial Windows desktop native check, embedded app build, strict Clippy and
|
||||
two desktop tests passed (index gate recovery and guarded source opening).
|
||||
- Desktop launched with WebView2 `--disable-gpu`. UI automation could not capture
|
||||
the window: `GetCursorPos` access denied, then monitor capture error
|
||||
`0x80070057`. Visual layout and end-to-end UI interactions remain unverified.
|
||||
- Real Python CPU ONNX embedding proof completed on 8 documents / 6 queries:
|
||||
5/6 top-1 accuracy, 6/6 recall@3. The strict top-1 quality gate is not met.
|
||||
See `tools/semantic-proof/windows-cpu-result.json` for raw timings and rankings.
|
||||
This tiny fixture is not evidence of large-corpus performance.
|
||||
- Native Rust CPU embedding adapter compiled, passed its three unit tests and
|
||||
the explicit local-model smoke test. The smoke reproduced the same 5/6 top-1
|
||||
limitation. Default CI omits this asset-dependent test; it must be executed
|
||||
separately with provisioned local artifacts.
|
||||
- Integrated workspace validation: 19 default tests passed; 1 local-model test
|
||||
is opt-in. Rust formatting and strict Clippy passed after lockfile refresh.
|
||||
- PDF/DOCX fixture tests include matching source locations, corrupt replacement,
|
||||
encrypted PDF behavior and compressed-input limits.
|
||||
The branded launcher launched natively. Accessibility inspection showed real
|
||||
controls and a warmed Google CPU worker. Native screenshots, input automation
|
||||
and complete image/audio/video playback interactions remain blocked by Windows
|
||||
access/capture errors recorded in [design QA](../design-qa.md). Passing pure
|
||||
preview policy tests does not establish real WebView codec playback or visual
|
||||
fidelity. Default-application opening is not exact timestamp navigation.
|
||||
|
||||
Not yet established: production latency targets, semantic inference, desktop
|
||||
runtime behavior, Linux/macOS execution, installer signing or upgrades.
|
||||
CI configuration is source configuration, not evidence that remote jobs ran.
|
||||
Gitea workflows require matching Windows, Linux and macOS runners; runner
|
||||
availability has not been established. GitHub workflows are also supplied for
|
||||
future mirroring, but do not run merely because code was pushed to Gitea.
|
||||
Windows x86-64 verified decoder provisioning/staging has executed. Manifest
|
||||
entries for Linux and ARM64 are configuration, not evidence of native execution.
|
||||
A vetted macOS decoder package remains missing. Linux/macOS worker/desktop
|
||||
execution, production GPU parity/fallback, installer signing, upgrade/rollback,
|
||||
licenses/source distribution, large libraries and calibrated absence handling
|
||||
remain unexecuted release gates. FFmpeg subprocess memory is not sandbox-capped;
|
||||
bounded outputs/timeouts are not an absolute RSS guarantee.
|
||||
|
||||
Gitea needs matching registered runners. GitHub workflows are supplied for
|
||||
mirroring; they do not run merely because code is pushed to Gitea. The current
|
||||
workflow checks do not automatically certify downloaded Google assets or the
|
||||
separately built worker. CI configuration is not remote job evidence.
|
||||
|
||||
## Historical evidence
|
||||
|
||||
Initial engine/CLI checks covered retrieval, persistence, filesystem changes,
|
||||
exclusions/filters, migration, concurrent readers, PDF pages, DOCX paragraphs,
|
||||
corrupt replacement, encrypted PDF errors and bounded compressed input.
|
||||
The first actual Google text adapter matched five pinned reference vectors
|
||||
above 0.999, including multilingual queries and long-document window coverage.
|
||||
|
||||
The rejected MiniLM/ONNX experiment achieved 5/6 top-1 on its six-query fixture;
|
||||
it is retained under the opt-in historical proof only and is not a production
|
||||
fallback. [Historical experiment](../tools/semantic-proof/HISTORICAL.md).
|
||||
Repository metadata/avatar updates were read back successfully. Historical
|
||||
foundation/default test totals changed as implementation expanded and must not
|
||||
be presented as the current aggregate.
|
||||
|
||||
Reproduction commands and exact interfaces are in [BUILDING.md](BUILDING.md),
|
||||
[SEMANTIC.md](SEMANTIC.md), and [MEDIA-QUALITY.md](MEDIA-QUALITY.md).
|
||||
Reference in new issue
Block a user