2026-10-12 15:51:39 +02:00
# Production readiness roadmap
2026-10-14 11:44:15 +02:00
## Current bounded goal — 2026-10-14
Karl replaces the broad production goal with two deliverables. Work stays local;
there are no intermediate pushes or releases. Publish only the final checked version.
- [ ] Verify native Windows/Linux interaction: launcher shortcut, tray, hide/show,
keyboard search, selection, preview and media seeking. Record unavailable UI
automation honestly rather than substituting headless tests.
- [x] Test Windows uninstall and reinstall, preserving existing user data and
using a fresh isolated application profile. Reinstalling on the development
VM does not prove missing prerequisites on a truly clean Windows image.
- [x] Establish bounded media baseline and improve practical image, music,
audio/video relevance, absent-content results and speech timestamps. Use real
Google EmbeddingGemma 2, retain source citations and report regressions.
2026-10-14 11:51:15 +02:00
- [x] Run meaningful changed-code checks, build/install the final package,
2026-10-14 11:44:15 +02:00
update user documentation and publish that final version once.
Perfect media recognition, macOS, real GPU qualification and the broader gates
below are outside this renewed scope. Stop experimentation when evidence shows
the practical limits; explain those limits and ship the verified improvement.
Additional user steering in this same final local version:
- [x] Replace the normal desktop window with a centered floating search bar;
reveal results/preview on input and keep settings behind a small control.
- [x] Automatically index accessible local files in the background, with
system/cache/index exclusions and a settings switch; no first-use folder gate.
- [x] Search installed applications and launch catalogued results from the bar.
Bounded evidence: uninstall/reinstall and isolated-profile Windows checks passed;
actual CPU Google media tests and the documented music/timing limits are recorded
in MEDIA-FINAL-QUALITY.md and SPEECH-FINAL-QUALITY.md. Launcher/app discovery and
automatic-indexing code passed Windows/Linux desktop tests. Current software
screenshots show actual retrieval and source highlights. Native final-build
hotkey/tray/glass interaction remains unchecked after capture/input failures;
do not infer it from unit tests or software screenshots. A final automatic-start
check exposed a schema writer conflict. Its correction passed held-writer
storage/warmup regressions and actual Google CPU retrieval. An empty Windows
profile with automatic indexing enabled then exposed 27 Desktop files and the
owned marker, with a resident CPU query-role worker and no schema-lock error.
2026-10-14 11:51:15 +02:00
Final bounded delivery: [searchi 0.1.8 is published ](https://tea.karlcom.de/karl/searchi/releases/tag/v0.1.8 ).
Windows desktop 0.1.8 is installed and its corrected installer verified;
Ubuntu 24.04 desktop 0.1.8 passed packaged CPU startup. Both public downloads
were fully hashed and matched the installed/tested artifacts. These are the
final release artifacts. README includes current working-search
2026-10-14 11:44:15 +02:00
images and a user-facing agent installation prompt. Headless Linux/Windows CLI
0.1.6 remain available through older releases. The native interaction checkbox
above stays open because final capture/input was denied; clean-machine, macOS
and real GPU gates below remain deferred. No further feature work belongs to
this bounded delivery. Gitea Actions remain disabled.
2026-10-12 15:51:39 +02:00
2026-10-13 11:20:23 +02:00
Karl's release priority, 2026-10-13: publish a usable first Windows version and
install it on DEV-WIN11 now. Extensive platform/GPU, clean-machine, signing and
large-library checks are deferred follow-up, not blockers for this initial
release. Keep their evidence and unchecked production gates intact.
2026-10-13 18:06:22 +02:00
Karl's current improvement priority: music relevance, precise spoken-passage
timestamps and irrelevant results when the requested content is absent. The
million-record lexical benchmark is deferred while this quality work proceeds.
Google EmbeddingGemma 2 remains the main text/image/audio/video search model.
2026-10-13 22:27:24 +02:00
Karl's Windows handoff request: stop extending the broader build scope after
the usable Windows delivery. Version 0.1.7 is installed, starts hidden at login,
and supports Ctrl+Shift+Space, Escape/close-to-hide and tray Show/Quit.
See [handoff evidence ](WINDOWS-0.1.7.md ). Unexecuted production gates below remain
open and deferred; do not continue expanding them without a renewed request.
2026-10-13 18:42:28 +02:00
- [x] Run frozen actual Google music metadata diagnostics: separate fields improved
known two-file music memories from 2/4 to 4/4; no-answer remained 5/5 failing.
This is hypothesis evidence, not independent production validation.
See [music evidence ](MUSIC-METADATA.md ).
2026-10-13 19:05:49 +02:00
- [x] Validate the tag hypothesis on frozen unseen licensed music plus fixed mixed
distractors: 8/14 → 11/14 with original tags, one genuine regression; misleading
tags fell to 5/14. Missing tags exactly preserved media-only results. No-answer
remained 4/4 failing. See fixtures/music-validation-v1/README.md.
- [ ] Improve and integrate metadata evidence with source citations/guards and full
mixed regression; unconditional field-max policy is not accepted as universal.
- [x] Evaluate pinned compact multilingual local reranker: quantized serial/batch
parity failed; F32 parity passed but six-pair warm cost missed the 200 ms target.
No suppression/default integration. See LOCAL-RERANKING.md.
2026-10-13 18:42:28 +02:00
- [x] Implement exact artist/title constraints in shared storage, lexical/ANN
eligibility, CLI/MCP and desktop controls; metadata-only search skips inference.
Unit/lifecycle evidence is recorded separately from actual model validation.
- [x] Actual Google CPU constraint integration passed with genuine tagged WAVs,
semantic/hybrid limit-one retrieval, empty absent identities, no-inference
metadata lookup and warmed lifecycle refresh; see [evidence ](MEDIA-CONSTRAINTS.md ).
2026-10-13 19:05:49 +02:00
- [x] Publish and install Windows 0.1.5 with exact music filters, verified public
bytes, source/runtime provenance and actual CPU startup/upgrade; see
[release evidence ](WINDOWS-0.1.5.md ). Native interaction and other OS gates stay open.
- [x] Evaluate finer source-preserving local ASR timing with actual Google retrieval:
the frozen VAD + DTW combination completed with all five sources retained and
improved strict start points from 4/8 to 6/8. Rejected ordinary-token and ungated DTW runs remain
preserved; no partial-pool scores were accepted. See SPEECH-DTW.md.
- [ ] Validate independent speech/languages and integrate accepted alignment with
model setup, packaging, cancellation, source guards and shared engine clients.
2026-10-13 20:18:44 +02:00
- [x] Implement isolated local recognition, independent raw/speech completion,
verified excerpts, desktop asset setup and CLI/MCP support; structural and
actual decoder-clock checks passed. See [integration ](SPEECH-INTEGRATION.md ).
- [x] Current CPU worker processed real English/German recordings from Unicode
paths and reused loaded recognition contexts; this is functional evidence,
not independent retrieval or word-timing accuracy.
2026-10-13 21:23:23 +02:00
- [x] Current Rust speech client unit and actual subprocess lifecycle checks
passed on Ubuntu 24.04 WSL; headless client dependency compilation passed.
[Evidence ](LINUX-SPEECH-0.1.6.md ); native Linux ASR and updated packages remain open.
2026-10-13 21:38:31 +02:00
- [x] Current Linux headless release compilation and extraction provenance;
add verified speech-runtime archive support with setup instructions and
21 packaging tests. Native runtime and public Linux speech delivery remain open.
2026-10-13 21:50:25 +02:00
- [x] Normal Linux speech runtime compilation, ELF closure, relocated self-test
and genuine forced-CPU recognition/context reuse from Unicode paths passed.
[Evidence ](LINUX-SPEECH-0.1.6.md ); packaged shared-service retrieval, GPU and
public speech archive delivery remain open.
2026-10-13 22:05:31 +02:00
- [x] Genuine packaged Linux Google/speech retrieval and public headless archive
delivery; full-byte public download verified. See [evidence ](LINUX-SPEECH-0.1.6.md ).
2026-10-13 22:42:06 +02:00
- [x] Rotate the exposed origin protection value using protected Infisical
storage and coordinated Cloudflare/NPM updates; old value rejected on all
three affected services, public access verified, no values committed.
2026-10-13 20:26:04 +02:00
- [x] Genuine shared-service ASR/Google lifecycle passed: verified speech
excerpts, zero raw reencodes through policy failure/retry, and retained
query reader; [proof ](SPEECH-INTEGRATION.md ).
2026-10-13 21:23:23 +02:00
- [x] Run genuine shared-service Google lifecycle and frozen independent EN/DE
query controls; stage and deliver the normal Windows hardware-detecting runtime.
See [speech evidence ](SPEECH-INTEGRATION.md ) and [release ](WINDOWS-0.1.6.md ).
Small positive fixture passed; absent-content controls failed. This does not
close broader recognition, timing or other-platform gates.
2026-10-13 18:06:22 +02:00
- [ ] Validate richer no-answer evidence on new independent positive/negative data; do not retune the failed universal cosine cutoff on old held-out results.
- [ ] Integrate accepted improvements into shared engine/UI/headless clients with verified setup, lifecycle and CPU performance evidence. Diagnostics alone do not raise the media quality rating.
2026-10-13 11:28:24 +02:00
- [x] Publish v0.1.0 Windows installer/checksum, verify public download bytes,
install on DEV-WIN11 and leave the app running with a desktop launcher.
2026-10-13 12:14:48 +02:00
- [x] Publish Windows desktop v0.1.1 with single-instance handling; verify public
bytes and isolated genuine Google CPU launch. Existing user installation is
not automatically upgraded. See [release evidence ](WINDOWS-0.1.1.md ).
2026-10-13 14:07:15 +02:00
- [x] Publish Windows desktop v0.1.2 with incomplete AI-answer rejection;
verify public installer/checksum bytes and isolated genuine CPU package launch.
See [release evidence ](WINDOWS-0.1.2.md ); user installation not auto-upgraded.
2026-10-13 15:38:34 +02:00
- [x] Publish Windows desktop v0.1.3 with isolated document processing and bounded media previews; installer helper and isolated CPU startup verified. See [evidence ](WINDOWS-0.1.3.md ). Native UI playback and upgrade checks stay open.
2026-10-13 12:48:02 +02:00
- [x] Publish the Linux x86-64 headless preview with current query-role runtime,
verified public bytes and genuine document/code/audio CPU archive checks on
Ubuntu 24.04 WSL. See [headless evidence ](LINUX-HEADLESS.md ); desktop, GPU and
broader distribution compatibility remain open.
2026-10-13 11:20:23 +02:00
2026-10-12 17:44:56 +02:00
## Required product scope (user clarification)
- Desktop UI is primary; semantic setup/indexing/search must require no terminal.
- Google EmbeddingGemma 2 is the required production model, matching DigUp.
- Local document, image, audio and video search are required product milestones.
- A separate CLI-only build supports headless Linux without desktop dependencies.
- Production automatically uses available hardware acceleration with CPU fallback;
this Windows VM is tested in forced-CPU mode only.
- Installer ships the native worker. First launch automatically downloads/verifies
missing model files and manages indexing in the UI; no terminal setup is required.
- Earlier MiniLM proof is an experiment and does not satisfy the model requirement.
- First interaction must be responsive: prepare indexes and a real Google query
in the background, retain ready sessions, and show immediate keyword results.
Model download and initialization must have honest progress and retry states.
2026-10-12 15:51:39 +02:00
## 1. Search foundation
- [x] Rust workspace, reusable engine, CLI, versioned SQLite storage.
- [ ] Filename/path and ranked full-text retrieval with strict filters.
- [ ] Safe folder discovery, exclusions, bounded extraction, passage locations.
- [ ] Incremental updates, rename/deletion reconciliation, crash recovery.
- [x] Persistence and filesystem lifecycle integration tests (Windows CPU).
## 2. Document and semantic retrieval
2026-10-12 16:07:53 +02:00
- [x] Text, Markdown, code, PDF and DOCX extraction with real fixtures (Windows CPU).
2026-10-12 15:51:39 +02:00
- [ ] Validate actual embedding model artifacts, license, CPU runtime and quantization.
- [ ] Versioned embeddings, ANN index, exact-match priority and hybrid fusion.
- [ ] English/German relevance corpus and measured top-10 recall.
- [ ] Atomic model/index replacement and resumable persistent work queues.
2026-10-13 01:36:34 +02:00
- [x] Completed-source manifests with title/model/window policy keys and
2026-10-13 00:43:20 +02:00
citation remapping; reuse only verified complete document/media groups.
- [ ] Durable generation tasks, crash resume and final corpus reconciliation;
see [incremental semantic design ](INCREMENTAL-SEMANTIC.md ).
2026-10-12 15:51:39 +02:00
## 3. Desktop
2026-10-13 14:10:45 +02:00
- [x] Image preview pixel ceilings use actual decoded format; a real PNG renamed
`.jpg` cannot receive the larger JPEG allowance (Windows CPU regression).
2026-10-13 14:22:19 +02:00
- [x] Replace lifetime-growing native media asset grants with bounded preview
2026-10-13 14:10:45 +02:00
access, including range limits, stale-source handling and selection/reload tests.
Do not use permanent forbidden patterns as revocation.
2026-10-13 14:22:19 +02:00
- [ ] Validate native custom-protocol image/audio/video playback and seeking on
release platforms; structural Windows CPU tests alone do not close this gate.
See [preview contract and evidence ](MEDIA-PREVIEW.md ).
2026-10-13 14:34:28 +02:00
- [x] Linux WSL native Rust preview tests (9/9) and full desktop suite (45/45)
passed on the immutable implementation snapshot; actual playback remains open.
2026-10-13 11:43:20 +02:00
- [x] Official single-instance transport with isolated explicit profiles;
Windows CPU duplicate launch exits without a second query worker.
See [launch evidence ](SINGLE-INSTANCE.md ); visual focus and other OS checks remain open.
2026-10-13 10:12:25 +02:00
- [x] Persisted solid/glass/frosted appearance and bounded background tint;
opaque reading surfaces, serialized settings updates and high-contrast fallback.
- [ ] Verify native glass visually on Windows 11 22H2+; macOS/Linux currently
use an explicit opaque fallback. Native compositor performance remains open.
2026-10-12 16:07:53 +02:00
- [ ] Tauri shell, accessible keyboard search, progressive results and previews (shell built; UI interaction verification pending).
2026-10-12 15:51:39 +02:00
- [ ] Folder selection, exclusion controls, resource budgets and indexing status.
- [ ] Global shortcut, opening/revealing source files, PDF page navigation.
- [ ] Watchers and reconciliation across Windows, macOS and Linux.
2026-10-13 02:30:38 +02:00
- [x] Native notification coordinator, bounded targeted reconciliation,
2026-10-13 06:30:43 +02:00
periodic verification and CPU-tested Windows/Linux create/update/rename/delete.
2026-10-13 02:30:38 +02:00
- [x] Durable same-stat media hint invalidation, guarded metadata writes and
real Google reindex/reuse regression; owned decoder scratch avoids feedback.
2026-10-13 07:11:27 +02:00
- [x] Native Linux watcher-driven UI refresh, independent refresh timing,
temporary selection restoration and explicit selection override.
- [ ] Native macOS events, Windows/macOS watcher-driven UI refresh and representative storage
2026-10-13 02:30:38 +02:00
failure/contention checks; see [automatic indexing ](AUTOMATIC-INDEXING.md ).
2026-10-12 15:51:39 +02:00
## 4. Media
2026-10-12 19:16:23 +02:00
- [x] Genuine Google image, audio and video embeddings integrated with the shared
search service and CPU-tested FFmpeg decoding; music tags and duration filters.
- [x] Frozen independent media corpus with retrieval, timestamp, negative-query
and indexing-cost measurements (see MEDIA-QUALITY.md).
- [ ] Improve precise spoken-passage localization, music hard negatives and
calibrated handling of queries without a relevant match.
2026-10-13 08:26:01 +02:00
- [x] Prepare bounded CPU speech diagnostics and test process failures,
timestamp units and frozen-label scoring; see [speech proposal ](SPEECH-ALIGNMENT.md ).
2026-10-13 13:39:04 +02:00
- [x] Build a pinned CPU VAD diagnostic with explicit one-thread inference,
native error detection and full probabilities; actual Silero silence/input
contracts passed. Media quality comparison remains open (see SPEECH-VAD.md).
2026-10-13 13:49:23 +02:00
- [x] Complete frozen five-source VAD/base-ASR/Google CPU diagnostic:
component costs decreased, but localization stayed 4/8 point and 6/8 overlap;
no production default selected. Independent held-out media quality remains open.
2026-10-13 08:26:01 +02:00
- [ ] Run genuine local speech candidates, Google transcript retrieval and
held-out English/German/music/no-speech checks before selecting a default.
2026-10-12 19:16:23 +02:00
- [ ] Package and verify decoders on every release OS; test production acceleration.
2026-10-13 00:05:09 +02:00
- [x] Native Linux x86-64 pinned decoder provenance, startup, WAV probe and CPU
decode; broader codec/ABI coverage and desktop packaging remain open.
2026-10-12 15:51:39 +02:00
- [ ] CPU OCR, image retrieval, audio transcription, video segment locations.
- [ ] Isolated extraction workers with timeouts and resource ceilings.
2026-10-13 14:42:40 +02:00
See [implementation subgoals and provenance contract ](EXTRACTION-ISOLATION.md ).
2026-10-12 15:51:39 +02:00
## 5. Release gates
2026-10-12 21:42:31 +02:00
- [x] Normal worker build/staging defaults to hardware-detecting auto with CPU
fallback; missing Vulkan/glslc fails configuration rather than hiding GPU loss.
- [ ] Provision release-build GPU SDKs and verify one hardware-detecting package
per OS/architecture, including missing-driver fallback and dependency closure.
2026-10-12 22:03:16 +02:00
- [x] Windows-x64 official pinned SDK copied into workspace with verified cache
provenance; universal CPU/Vulkan worker and embedded-frontend desktop built.
- [x] Real Windows archive integrity, packaged decoder startup and auto-mode
CPU inference with a controlled no-GPU-provider registry; Google CPU parity.
2026-10-12 22:35:34 +02:00
- [x] Normal Windows x64 archive includes baseline/optimized CPU dispatch,
OS AVX-state guards, shared app-local MSVC CRT and hashed DLL inventory;
real optimized and baseline-only package inference passes in the CPU VM.
2026-10-13 00:43:20 +02:00
- [x] Normal native Linux x86-64 CPU/Vulkan worker build, hashed 19-library
staging and genuine optimized/baseline-only CPU code-query inference outside
the build tree; desktop, ABI/older-hardware and GPU validation remain open.
- [x] Actual Linux automatic CPU fallback with Vulkan loader, ICDs/drivers and
GPU devices absent in a verified private namespace; no GPU provider loaded.
2026-10-12 22:35:34 +02:00
- [ ] Verify CPU dispatch on older physical CPUs and a clean Windows machine;
dependency inspection and baseline execution on a modern VM are insufficient.
2026-10-12 22:03:16 +02:00
- [ ] Actual missing/incompatible GPU drivers, precision, device loss and memory
pressure on release hardware; native GUI and clean-machine archive launch.
2026-10-12 21:36:17 +02:00
- [x] Windows desktop archive packager includes explicit runtime/helper files
and notices, checks architecture and records per-file checksums; excludes
model caches and user data. Controlled packaging regression tests pass.
2026-10-12 23:46:01 +02:00
- [x] Embedded-frontend Windows Release archive launches outside the source tree
with isolated data and a ready real Google CPU semantic generation.
- [ ] Verify native interactions, clean-machine prerequisites, GPU dependencies,
upgrades and signing.
2026-10-12 21:30:10 +02:00
- [x] Preflight native license and the complete offline Google model pair before
modifying staged resources; corrupted/missing assets preserve existing worker.
2026-10-12 23:46:01 +02:00
- [x] Transactional resource staging, verified copied bytes, process locking,
interrupted-copy/swap recovery and backup preservation on failed rollback.
2026-10-13 13:04:58 +02:00
- [x] Normal Windows/Linux source-build provenance gate before staging and
archive preparation; stale inventories preserve existing resources/packages.
Actual normal builds verified. See [runtime provenance ](RUNTIME-PROVENANCE.md ).
2026-10-13 13:26:16 +02:00
- [x] Prepare static macOS Metal inventory and mandatory normal-profile staging,
with embedded-source/dependency/architecture metadata regressions.
- [ ] Execute native macOS runtime build, relocation and genuine CPU/Metal
inference; fixture metadata tests do not satisfy this gate.
2026-10-12 23:46:01 +02:00
- [ ] Verify power-loss durability and release-tool interaction with resource
directory swaps on each OS; fault-injected process interruption is narrower.
2026-10-12 19:16:23 +02:00
- [x] Shared monotonic Basic/Debug search metrics, granular bounded engine traces,
actual worker query diagnostics, and headless JSON performance output.
- [ ] Verify detailed performance UI interaction on native desktop platforms;
repeat overhead measurements on representative libraries and hardware.
2026-10-12 15:51:39 +02:00
- [ ] Formatting, lint, tests and security/dependency review.
2026-10-13 10:35:50 +02:00
- [x] Inventory locked Rust/npm advisories; prepare byte-verified compatible
upstream GLib iterator backport for the Linux desktop dependency graph.
2026-10-13 17:57:03 +02:00
- [x] Execute optimized Linux GLib backport regressions: three tests and the owner-lifetime compile-fail test passed; stored log hash and all 121 upstream source files reverified.
- [x] Native Linux 0.1.4 desktop build and packaged CPU startup use the selected local GLib backport; native graphical interaction remains unchecked.
- [x] Execute cargo-audit 0.22.2 against both current lockfiles with a fresh RustSec database and warnings denied; zero vulnerability-class findings, two tracked maintenance notices and exit 1. See DEPENDENCY-REVIEW.md.
- [ ] Complete native graphical interaction and broader native runtime/source security review. No broad security gate is closed by these narrow checks.
2026-10-12 15:51:39 +02:00
- [ ] CPU benchmarks: filename p95 <50 ms/1M records; text <150 ms/100K documents; semantic <500 ms on declared hardware.
- [ ] Measure cold start, indexing throughput, RAM, disk and idle CPU.
2026-10-12 17:44:56 +02:00
- [ ] Record the very first foreground query and distinct unseen queries after
production background preparation; report initialization time separately.
Warm p95 alone does not satisfy the first-use requirement.
2026-10-12 15:51:39 +02:00
- [ ] Test migrations, interrupted work, invalid documents, inaccessible/offline roots and cloud placeholders.
- [ ] Automated native build matrix and documented supported cross-compilation paths.
2026-10-13 01:36:34 +02:00
- [x] Native Linux headless workspace tests: 106 passed, 18 explicitly ignored.
2026-10-13 00:43:20 +02:00
- [x] Native Linux Google text/code/media reference parity and service
lifecycle/rebuild recovery: seven genuine CPU integration tests passed.
- [x] Unsigned Linux x86-64 headless archive: verify extracted checksums/modes,
mixed document/code/audio indexing, decoder discovery, semantic retrieval,
line citations and strict filters outside the source/build directories.
- [ ] Execute native Linux video/codec coverage and Linux desktop/macOS
platform tests; create Windows/macOS/Linux installers.
2026-10-12 15:51:39 +02:00
- [ ] Signing/notarization configuration, upgrade/rollback validation, release instructions.
2026-10-12 20:28:38 +02:00
## 6. Local agents and optional answers
- [x] Read-only MCP stdio server using the shared search service, persistent
Google worker, structured filters, source citations and measured search stats.
- [x] End-to-end MCP initialization, discovery, search, invalid-input and
source-access tests; no protocol text mixed with diagnostic stdout.
- [x] Installable agent skill and copyable MCP client configuration, with clear
database scope, readiness and untrusted-source handling.
- [x] Optional OpenAI-compatible summaries/questions, disabled by default;
explicit selected-source action, endpoint disclosure, bounded payloads,
credentials kept out of persistent settings, diagnostics and error messages.
- [x] Verify real HTTP wire format against a local test endpoint; reject invalid
settings, stale sources and unauthorized citations; preserve search on failure.
- [ ] Native UI interaction and chosen-provider quality/latency checks before
claiming production-ready answers or compatibility with every provider.
2026-10-12 20:59:10 +02:00
## 7. EmbeddingGemma 2 efficiency
Audit and measured priorities: [EMBEDDINGGEMMA2-AUDIT.md ](EMBEDDINGGEMMA2-AUDIT.md ).
- [x] Audit actual required model, task prefixes, pooling/normalization, native
sequence processing and CPU/runtime policies against primary sources.
- [x] Record real CPU 1/2/4-thread query/first-use/document measurements and
paired repeated-query/filter-change results with unchanged retrieval evidence.
- [x] Bounded provider-scoped actual-query vector LRU with accurate cache traces;
preserve filtering, source freshness, failures and actual warmup probes.
2026-10-13 01:36:34 +02:00
- [x] Bounded shared prepared-ANN handoff before SQL activation; retain query
model/LRU and readiness across publication. Actual CPU first new query has
no foreground graph load; [recovery/handoff evidence ](RECOVERY-AND-HANDOFF.md ).
2026-10-12 20:59:10 +02:00
- [x] Reduce semantic settle delay to 120 ms and retain old ready generation
during rebuilding; test initial-generation and activation readiness boundaries.
2026-10-12 23:46:01 +02:00
- [x] Bound whole IPC write/read deadlines and recover from a nonreading helper;
CPU-only real helper-process regressions cover blocked writes and responses.
- [x] Windows x64 baseline/optimized CPU runtime selection and OS AVX-state
checks, verified with real packaged baseline and optimized inference.
- [ ] Validate CPU selection on older hardware and remaining architectures.
2026-10-13 00:58:57 +02:00
- [x] Completed-source vector reuse in atomic schema-7 generations, verified
with genuine Google CPU document/code/image/audio/video fixtures.
- [x] Resume completed staging sources after failed publication; incomplete
groups are purged under the OS build lock before retry.
2026-10-13 01:14:40 +02:00
- [x] Pre-inference staging pruning with compatible rename remapping and
100,000-chunk storage boundary; late validation after ANN preparation.
- [x] Exact parser-input SHA provenance and same-metadata overwrite detection
on normal rescans; stale/missing agent excerpts fail closed.
- [x] Actual Google CPU same-metadata text mutation before/during inference:
reject publication, rescan without metadata workarounds and reuse unaffected vectors.
2026-10-13 01:36:34 +02:00
- [x] Abrupt owned CLI termination after a committed partial audio chunk;
resume completed documents and reencode unfinished media without duplicates.
Process termination does not prove power-loss durability.
- [ ] Concurrent deletion/media mutation, component-policy reuse and
2026-10-13 01:14:40 +02:00
indexing/foreground measurements.
2026-10-13 00:58:57 +02:00
See [INCREMENTAL-SEMANTIC.md ](INCREMENTAL-SEMANTIC.md ).
2026-10-12 20:59:10 +02:00
- [ ] Benchmark genuine token-budgeted multi-sequence batching and isolation.
- [ ] Implement and measure shared foreground/indexer resource budgets/priority.
2026-10-13 01:56:03 +02:00
- [x] Shared inference admission, foreground priority between background
encodes, total-thread settings migration and retained warm sessions on save
failure; real Google CPU overlap regression passed on Windows.
- [ ] Measure representative mixed-media contention, single-thread latency,
extraction/decoder/ANN work and native UI p95 before closing this gate.
2026-10-12 20:59:10 +02:00
- [ ] Attribute startup cost; measure smaller query context and selective encoders.
2026-10-13 07:30:40 +02:00
- [x] Query-only 512-token context in the normal worker, desktop/CLI/MCP
integration and actual Windows CPU parity/boundary/reference tests.
Paired samples reduced private commitment by 183 MB; resident memory and
cached startup were essentially unchanged. See [QUERY-CONTEXT.md ](QUERY-CONTEXT.md ).
2026-10-13 08:47:41 +02:00
- [x] Native Linux query-context and CLI/MCP cited-result checks, with six
paired CPU process measurements; resident/PSS memory effectively unchanged.
- [ ] Verify native macOS, GPU parity and updated installed packages.
2026-10-13 07:30:40 +02:00
- [ ] Evaluate text readiness before the larger media projector finishes
downloading; preserve verified assets and honest media setup status.
2026-10-13 08:47:41 +02:00
- [x] Typed backbone/projector verification callbacks with full-pair setup
semantics preserved; 16 asset tests passed, including gated media download.
2026-10-13 09:27:40 +02:00
- [x] Integrate explicit text/media coverage, retained query readiness and
media retry status across engine/desktop; genuine CPU promotion reuses
identical document vectors with zero document re-encodes.
See [early text setup ](EARLY-TEXT-SETUP.md ).
2026-10-13 10:12:25 +02:00
- [x] Native Linux CPU first use during genuine delayed media download,
failure/retry and promotion; actual first-input timing recorded separately.
Focused activation retest preserves selection and query process, records
fresh timing/current generation after idle. Windows/macOS checks stay open.
2026-10-12 20:59:10 +02:00
- [ ] Buffered bounded media IPC and linear UTF-8 source-window offset processing.
- [ ] Evaluate 256d MRL and content-specific vision budgets on independent data.
- [ ] Validate GPU precision, synchronization-aware timings, parity and fallback
on actual accelerator hardware; CPU VM checks do not satisfy this gate.
2026-10-12 15:51:39 +02:00
Completion requires recorded evidence; targets are not measured results.
2026-10-12 21:22:11 +02:00
## 8. Source-code retrieval
- [x] Shared code classification, strict lexical/ANN filtering before top-k,
expanded source/build formats, bounded overlapping passages and actual lines.
- [x] Desktop Code scope, CLI `--code-only` , read-only MCP `code_only` , immutable
scope snapshots and code-format-preserving previews.
- [x] Google's CodeRetrieval query instruction, explicit worker capabilities,
task-scoped vector cache and genuine CPU prompt/debug parity.
- [x] Exact original source windows when Google's tokenizer normalizes text;
verified UTF-8 offsets, full tail coverage and actual token-budget checks.
- [x] Schema-11 migration schedules unchanged source for re-extraction; tests
preserve ordinary documents and filename availability through the upgrade.
- [x] Frozen 9-query real Google CPU regression with lexical comparison,
ordinary-document exclusion and exact MCP source excerpt verification.
- [ ] Evaluate independent real repositories, more languages, overloaded
symbols, near-duplicate routines and queries with no relevant implementation.
2026-10-13 08:26:01 +02:00
- [x] Audit mixed-task routing and eligibility cache behavior; see
[routing audit ](RETRIEVAL-ROUTING.md ).
2026-10-13 08:36:31 +02:00
- [x] Complete the unchanged 32-file/91-query real Google CPU mixed baseline:
Code scope 14/14 top-1 versus All files 10/14; all 10 no-answer queries
returned candidates. See [mixed quality ](MIXED-QUALITY.md ).
- [ ] Measure task fusion and cache cost against independently reviewed
held-out data, preserving the frozen diagnostic for regression comparison.
2026-10-13 10:25:35 +02:00
- [x] Freeze disjoint bilingual text/code calibration and held-out labels
after independent AI review; verify exact Git-preserved source/label bytes.
- [x] Add a shared-engine calibration-only runner retaining raw cosine,
lexical provenance, hybrid ranks and atomic incomplete evidence.
2026-10-13 11:43:20 +02:00
- [x] Execute the frozen calibration-only baseline with genuine Google CPU
inference; preserve raw results and leave held-out requests unevaluated.
- [ ] Freeze the proposed relevance policy, then
2026-10-13 10:25:35 +02:00
evaluate held-out queries without tuning against their results.
2026-10-13 08:47:41 +02:00
- [x] Explicit English/German code intent vetoes exclusive audio routing;
actual Google CPU regression preserves code among qualifying audio candidates.
2026-10-12 21:22:11 +02:00
- [ ] Validate code search UI interaction on native Windows/macOS/Linux and
actual GPU hardware. Tiny CPU fixtures do not close those release gates.
2026-10-12 15:51:39 +02:00
## Completed foundation details
- SQLite FTS5 with exact filename priority, text snippets and strict filters.
- Default directory exclusions and protection against indexing the index itself.
- Metadata reuse, atomic root reconciliation and schema-v1 migration coverage.
- Actual CLI process integration tests and independent-reader concurrency test.
- TypeScript desktop frontend build and dependency audit passed; native validation underway.
- Initial README commit pushed to Gitea main via protected Infisical injection.
2026-10-12 16:07:53 +02:00
- Native desktop build and guarded opening tests passed on Windows.
- Genuine multilingual CPU model proof: 5/6 top-1, 6/6 recall@3 ; quality gate remains open.
- Native Rust CPU embedding adapter verifies pinned model/tokenizer assets;
integration with the search index and platform runtime packaging remain open.
- PDF page and DOCX paragraph matches persist; corrupt/encrypted/over-limit
documents remain filename-searchable and report extraction errors.
- CLI exposes folder/type/size/date filters and additional directory exclusions.
2026-10-12 15:51:39 +02:00
The partial features above do not close broader production milestones such as
crash-resumable jobs, fuzzy retrieval, watcher reliability or performance targets.
2026-10-13 15:17:42 +02:00
## Document worker follow-up
- [x] Shared PDF/DOCX process adapter, exact-byte post-parser validation, cancellation rollback and desktop resource discovery. See [evidence ](EXTRACTION-ISOLATION.md ).
2026-10-13 15:38:34 +02:00
- [x] Build and require the extraction helper in Windows release packages; extracted installer helper passed genuine PDF indexing.
2026-10-13 15:45:18 +02:00
- [x] Build, verify and publish updated Linux headless archive with its required helper; genuine relocated PDF/DOCX and Google CPU checks passed.
2026-10-13 16:07:12 +02:00
## Current shared-code check
At source `fd67906` , Windows CPU-only validation passed strict
`cargo clippy --workspace --all-targets --locked -- -D warnings` and workspace
formatting. The public v0.1.3 tag resolves to the recorded application source
`3bf95583533ce44021c8ecd9733f32062488976e` .
A bounded Infrastructure Ops System-catalog query for macOS test availability
returned no registered Mac workstation. No Mac network availability or native
build was inferred from that lookup; macOS execution gates remain unchecked.
2026-10-13 16:14:13 +02:00
- [x] Current native Linux desktop release compilation and local loader closure on Ubuntu WSL; [evidence ](LINUX-DESKTOP.md ).
2026-10-13 16:21:36 +02:00
- [x] Linux desktop relocated normal resource discovery and CPU query-worker startup under an owned software-display test; no native UI interaction claim.
2026-10-13 17:02:16 +02:00
- [x] Publish Ubuntu 24.04 x86-64 Debian desktop preview; verify public bytes and extracted package CPU startup. See [evidence ](LINUX-DESKTOP.md ).
- [ ] Linux desktop graphical interaction and broader distribution compatibility.
## Rust audit and goal review, 2026-10-13
2026-10-13 21:20:31 +02:00
- [x] Publish Windows 0.1.6 and optional CLI/MCP client; verify public full bytes and install/start the normal distribution under CPU-only VM constraints. Settings preserved and database integrity checked. See [delivery evidence ](WINDOWS-0.1.6.md ).
- [x] Make local speech/media search usable with visible experimental status, preserving Google EmbeddingGemma 2 retrieval and source locations.
- [ ] Resolve absent-content false positives and establish broader music/speech quality and precise timestamp evidence. Preserve the failed longer-source evaluation; do not promote its score.
2026-10-13 17:02:16 +02:00
- [x] Audit core/service and desktop/CLI Rust boundaries; avoid unnecessary naming/module churn.
- [x] Correct valid pre-1970 filesystem timestamps across indexing, source eligibility and previews. Changed crates passed 100 tests (six ignored), desktop 47; strict Clippy and formatting passed. See [audit ](RUST-AUDIT.md ).
2026-10-13 17:09:55 +02:00
- [x] Publish Windows v0.1.4 with the timestamp correction; verify public bytes, packaged helper provenance and isolated real Google CPU query-worker startup. See [evidence ](WINDOWS-0.1.4.md ).
2026-10-13 17:25:37 +02:00
- [x] Publish updated Linux headless and Debian packages with the timestamp correction; genuine archived-document CPU search, unchanged rescan and packaged query-worker startup passed. Public bytes verified; [evidence ](LINUX-0.1.4.md ).
2026-10-13 17:30:52 +02:00
- [x] Protect mixed quality evidence from accidental report overwrite and partial JSON publication; eight evaluator contract tests passed.
2026-10-13 17:49:10 +02:00
- [x] Finish the unchanged frozen mixed regression on the current 0.1.4 engine; independent score recomputation confirms one known routing improvement and no scored regression. All-files positive top-1 74/80, recall@5 80/80; no-answer failures remain 10/10. See [comparison ](MIXED-QUALITY-0.1.4.md ); independent quality gates stay open.
2026-10-13 17:02:16 +02:00
Initial Windows delivery is complete. Native interaction/upgrade, actual GPU,
macOS execution and independent media-quality work remain real follow-up; the
full production goal remains active. Preserve the detailed unchecked gates above.