Add bounded exact-byte document extraction protocol and transport

This commit is contained in:
karl committed 2026-10-13 14:49:21 +02:00
1 parent 99076d4b13
commit f84fe0a3f2
6 files changed
+477 -2

No files matched your search

+22 -1
View File
@@ -42,7 +42,7 @@ result; retain the original parser-input digest, not a substituted later hash.
Source files and SQLite are not one atomic transaction. Preserve semantic and
agent excerpt provenance checks and clear old passages on parser rejection.
- [ ] Buffer API and typed protocol with independent limits and genuine fixtures.
- [x] Buffer API and typed protocol with independent limits and genuine fixtures.
- [ ] Headless Rust helper with no GUI dependencies or arbitrary path access.
- [ ] Parent process deadlines, cancellation, resource setup and kill/reap tests.
- [ ] Shared-engine document adapter, explicit failures and post-parser guards.
@@ -56,3 +56,24 @@ invalid locations, blocked stdin, worker crash/timeout, missing helper,
cancelled reconciliation, corrupt/encrypted documents and a source changing
during parsing. Shipping only protocol types or unit fixtures does not close
the isolated-extraction release gate.
## Implemented first slice
`searchi-core::extraction_protocol` exposes path-free PDF/DOCX requests and
validated responses. Version one admits at most 32 MiB input, a 1-KiB JSON
header, 13 MiB serialized response and 2 MiB aggregate extracted text. Exact
payload length/SHA, response version/digest and document locations are checked;
parser rejection is explicit. Unknown nested source fields are rejected.
`extraction_transport` frames bounded JSON lengths as four-byte little-endian
integers followed by raw input bytes, with no base64 or filesystem paths.
It rejects oversized lengths before allocation and mismatched input before
writing. It is blocking I/O: the parent still must supply deadlines, cancellation,
EOF/process-status validation and owned-child cleanup.
All 50 core tests passed on Windows (18 unit and 32 integration), including
actual DOCX buffer extraction, corrupt PDF/DOCX rejection, raw binary byte
preservation, version/hash/location/size failures and truncated frames. Strict
all-target Clippy, Rust formatting and diff checks passed. Product extraction
defaults and code/text parsing are unchanged. No separate worker was launched;
process isolation and resource ceilings remain open.