Add bounded exact-byte document extraction protocol and transport
This commit is contained in:
1 parent
99076d4b13
commit
f84fe0a3f2
6 files changed
+477
-2
No files matched your search
@@ -42,7 +42,7 @@ result; retain the original parser-input digest, not a substituted later hash.
|
||||
Source files and SQLite are not one atomic transaction. Preserve semantic and
|
||||
agent excerpt provenance checks and clear old passages on parser rejection.
|
||||
|
||||
- [ ] Buffer API and typed protocol with independent limits and genuine fixtures.
|
||||
- [x] Buffer API and typed protocol with independent limits and genuine fixtures.
|
||||
- [ ] Headless Rust helper with no GUI dependencies or arbitrary path access.
|
||||
- [ ] Parent process deadlines, cancellation, resource setup and kill/reap tests.
|
||||
- [ ] Shared-engine document adapter, explicit failures and post-parser guards.
|
||||
@@ -56,3 +56,24 @@ invalid locations, blocked stdin, worker crash/timeout, missing helper,
|
||||
cancelled reconciliation, corrupt/encrypted documents and a source changing
|
||||
during parsing. Shipping only protocol types or unit fixtures does not close
|
||||
the isolated-extraction release gate.
|
||||
|
||||
## Implemented first slice
|
||||
|
||||
`searchi-core::extraction_protocol` exposes path-free PDF/DOCX requests and
|
||||
validated responses. Version one admits at most 32 MiB input, a 1-KiB JSON
|
||||
header, 13 MiB serialized response and 2 MiB aggregate extracted text. Exact
|
||||
payload length/SHA, response version/digest and document locations are checked;
|
||||
parser rejection is explicit. Unknown nested source fields are rejected.
|
||||
|
||||
`extraction_transport` frames bounded JSON lengths as four-byte little-endian
|
||||
integers followed by raw input bytes, with no base64 or filesystem paths.
|
||||
It rejects oversized lengths before allocation and mismatched input before
|
||||
writing. It is blocking I/O: the parent still must supply deadlines, cancellation,
|
||||
EOF/process-status validation and owned-child cleanup.
|
||||
|
||||
All 50 core tests passed on Windows (18 unit and 32 integration), including
|
||||
actual DOCX buffer extraction, corrupt PDF/DOCX rejection, raw binary byte
|
||||
preservation, version/hash/location/size failures and truncated frames. Strict
|
||||
all-target Clippy, Rust formatting and diff checks passed. Product extraction
|
||||
defaults and code/text parsing are unchanged. No separate worker was launched;
|
||||
process isolation and resource ceilings remain open.
|
||||
Reference in new issue
Block a user