Inspect static PDF and OOXML content with bounded local coverage

This commit is contained in:
Karl committed 2026-10-14 10:12:34 +02:00
1 parent 69d820e315
commit 9e7cb57093
14 files changed
+758 -51

No files matched your search

+2 -1
View File
@@ -4,7 +4,8 @@ All inspection is local. No scan downloads models or calls cloud inference.
- YARA: genuine pinned yara-python 4.5.4 in private Windows CPython 3.13 runtime. Centrally supplied sources have bounded size and count, includes disabled, imported modules disallowed, compile warnings treated as errors, and five-second match timeout. Rules compile from in-memory strings and scan bytes. Errors, unavailable engine, missing rules and oversize payloads block downloads. Compilation currently runs inside the inspector process; a native compiler hang is bounded by the browser's inspector request deadline but does not restart the inspector automatically.
- Text: UTF-8 extraction and deterministic secret/literal/fingerprint classification.
- PDF/OOXML: bounded extraction of text and embedded images; full rendered visual coverage is incomplete and outbound DLP fails closed.
- PDF: bounded static subset now inspects decoded text, metadata and all indexed objects, independently OCRs embedded page images, and renders/OCRs every page with the bundled offline PDFium/OCR runtime. Active content, forms, annotations, attachments, layers, incremental updates, orphan/unsupported streams and embedded/custom font encodings remain blocked. See [PDF coverage](PDF-COVERAGE.md).
- OOXML: bounded extraction of text and embedded images; full rendered visual coverage is incomplete and outbound DLP fails closed.
- Images: bundled RapidOCR 3.8.1 with CPU ONNXRuntime 1.24.4 and packaged PP-OCRv4 detection/recognition plus v2 classification weights. Explicit verified local model paths bypass all model downloads. Real Windows test renders a secret into an image and blocks it with network model downloading forbidden. Models primarily support Chinese/English; other languages and recognition accuracy require validation. OCR completion indicates all frames processed, not guaranteed recognition accuracy. Pillow parsing has dimension/frame limits. Optional `SAFETYNET_OCR_ENGINE=tesseract` uses administrator-installed trusted `TESSERACT_PATH`, optionally `TESSDATA_PREFIX` for local traineddata. Tesseract weights are not bundled. Missing OCR blocks DLP.
- Audio/video: recognition of container types only. Transcription and full video inspection are unconfigured; DLP blocks rather than claiming safe coverage.
- Semantic adapter: real Google EmbeddingGemma 2, Q8_0 GGUF export revision `bfcd298762cc34d0357ece5ebdd31791a3a374d8`, 768 dimensions. A verified native worker runs explicitly on CPU, reads bounded local NDJSON, and returns actual normalized vectors. Both GGUF weights and native runtime modules must pass size/SHA-256 checks before loading. Configure `EMBEDDING_MODEL_DIR` and `EMBEDDING_WORKER_PATH` to staged, protected local resources. No cloud inference, model download or Python ML framework is used. Text uses the same pinned `document-v1` profile for training and inspection; native token windows must cover exact UTF-8 bytes through the final byte, otherwise scanning fails closed. At most 64 vectors are allowed; incomplete or oversized training is refused. Native image embeddings process all bounded frames resized to at most672px with48px alignment. Native audio supports only mono16kHz PCM16 WAV up to30seconds; other media remains uninspectable. Actual model fixtures must run separately from protocol unit tests to establish operational readiness.