Define concrete document parser isolation rollout and invariants

docs/EXTRACTION-ISOLATION.md

@@ -0,0 +1,58 @@
+# Document parser isolation
+
+Status: implementation started; current PDF/DOCX parsing still runs in the
+indexing process. Existing byte/output limits do not establish parser memory,
+CPU or crash isolation. This work does not delay the published Windows preview.
+
+## Required execution boundary
+
+Keep the shared Rust engine independent of the UI. The parent reads each
+supported source once and fingerprints those exact bytes. PDF and DOCX workers
+receive a format enum and raw bytes, never a filesystem path, index handle or
+mutation capability. Existing bounded plain-text/code extraction retains its
+exact source bytes and line positions. Filename indexing remains available when
+a document parser fails.
+
+The protocol has a bounded, versioned header followed by length-delimited binary
+input. Avoid base64 allocation overhead. Responses carry the input SHA-256 and
+typed passages or explicit parser rejection. The parent independently validates
+version, digest, count, aggregate text size and source locations. Serialized
+JSON has its own output limit because escaping can expand extracted text.
+
+Run one process per document initially. Bound admission, stdin writes, stdout,
+stderr and total elapsed time; a deadline must cover a blocked input writer as
+well as reading the response. Kill and reap owned children on timeout,
+cancellation, malformed output and failure. Missing workers or failed resource
+setup must never silently fall back to unrestricted in-process parsing.
+Parser errors retain a filename-only record with an extraction error;
+cancellation remains distinct and rolls back reconciliation.
+
+Use Windows Job Objects for owned process termination and commitment limits.
+Use supported Unix resource limits before parsing, but verify their actual
+semantics per OS. A configured RLIMIT_AS alone does not prove a universal macOS
+memory ceiling. Prefer existing bounded runtime patterns and established OS
+bindings over inventing a new process manager.
+
+## Publication and rollout
+
+Current reconciliation holds a SQLite transaction across discovery, reads,
+parsing and writes. The current source-stat check is immediately after reading,
+before parsing. Add a post-worker source/revision check before inserting the
+result; retain the original parser-input digest, not a substituted later hash.
+Source files and SQLite are not one atomic transaction. Preserve semantic and
+agent excerpt provenance checks and clear old passages on parser rejection.
+
+- [ ] Buffer API and typed protocol with independent limits and genuine fixtures.
+- [ ] Headless Rust helper with no GUI dependencies or arbitrary path access.
+- [ ] Parent process deadlines, cancellation, resource setup and kill/reap tests.
+- [ ] Shared-engine document adapter, explicit failures and post-parser guards.
+- [ ] Normal helper discovery and required release staging on all target systems.
+- [ ] Relocated Windows/WSL packages and actual parser fixtures through the helper.
+- [ ] Record startup/throughput/memory costs before considering persistent workers.
+- [ ] Validate remaining macOS and representative corrupt/large-source behavior.
+
+Meaningful regressions include wrong version/hash, oversized header/output,
+invalid locations, blocked stdin, worker crash/timeout, missing helper,
+cancelled reconciliation, corrupt/encrypted documents and a source changing
+during parsing. Shipping only protocol types or unit fixtures does not close
+the isolated-extraction release gate.

docs/ROADMAP.md

@@ -105,6 +105,7 @@ release. Keep their evidence and unchecked production gates intact.
   decode; broader codec/ABI coverage and desktop packaging remain open.
 - [ ] CPU OCR, image retrieval, audio transcription, video segment locations.
 - [ ] Isolated extraction workers with timeouts and resource ceilings.
+  See [implementation subgoals and provenance contract](EXTRACTION-ISOLATION.md).
 
 ## 5. Release gates
 - [x] Normal worker build/staging defaults to hardware-detecting auto with CPU