first commit
This commit is contained in:
Karl committed
commit
ff482ebe23
251 files changed
+19126
No files matched your search
@@ -0,0 +1,31 @@
|
||||
# Local inspector capabilities
|
||||
|
||||
All inspection is local. No scan downloads models or calls cloud inference.
|
||||
|
||||
- YARA: genuine pinned yara-python 4.5.4 in private Windows CPython 3.13 runtime. Centrally supplied sources have bounded size and count, includes disabled, imported modules disallowed, compile warnings treated as errors, and five-second match timeout. Rules compile from in-memory strings and scan bytes. Errors, unavailable engine, missing rules and oversize payloads block downloads. Compilation currently runs inside the inspector process; a native compiler hang is bounded by the browser's inspector request deadline but does not restart the inspector automatically.
|
||||
- Text: UTF-8 extraction and deterministic secret/literal/fingerprint classification.
|
||||
- PDF/OOXML: bounded extraction of text and embedded images; full rendered visual coverage is incomplete and outbound DLP fails closed.
|
||||
- Images: bundled RapidOCR 3.8.1 with CPU ONNXRuntime 1.24.4 and packaged PP-OCRv4 detection/recognition plus v2 classification weights. Explicit verified local model paths bypass all model downloads. Real Windows test renders a secret into an image and blocks it with network model downloading forbidden. Models primarily support Chinese/English; other languages and recognition accuracy require validation. OCR completion indicates all frames processed, not guaranteed recognition accuracy. Pillow parsing has dimension/frame limits. Optional `SAFETYNET_OCR_ENGINE=tesseract` uses administrator-installed trusted `TESSERACT_PATH`, optionally `TESSDATA_PREFIX` for local traineddata. Tesseract weights are not bundled. Missing OCR blocks DLP.
|
||||
- Audio/video: recognition of container types only. Transcription and full video inspection are unconfigured; DLP blocks rather than claiming safe coverage.
|
||||
- Semantic adapter: real Google EmbeddingGemma 2, Q8_0 GGUF export revision `bfcd298762cc34d0357ece5ebdd31791a3a374d8`, 768 dimensions. A verified native worker runs explicitly on CPU, reads bounded local NDJSON, and returns actual normalized vectors. Both GGUF weights and native runtime modules must pass size/SHA-256 checks before loading. Configure `EMBEDDING_MODEL_DIR` and `EMBEDDING_WORKER_PATH` to staged, protected local resources. No cloud inference, model download or Python ML framework is used. Text uses the same pinned `document-v1` profile for training and inspection; native token windows must cover exact UTF-8 bytes through the final byte, otherwise scanning fails closed. At most 64 vectors are allowed; incomplete or oversized training is refused. Native image embeddings process all bounded frames resized to at most672px with48px alignment. Native audio supports only mono16kHz PCM16 WAV up to30seconds; other media remains uninspectable. Actual model fixtures must run separately from protocol unit tests to establish operational readiness.
|
||||
- Reference registration: SHA-256 plus normalized eight-word shingle and paragraph hashes, coverage and registrationReady. Every shingle offset is inspected; paragraph normalization uses NFKC, lowercase, sharp-s to ss, final sigma to sigma and collapsed whitespace, shared with JavaScript policy enforcement. Up to 1000 hashes return per registration. Published `fragment-sha256` rules use this same canonicalization and detect excerpts after reformatting. No extracted plaintext returned or logged. Hashes remain sensitive reference metadata and require access control. The whole-file fingerprint is useful even for incomplete extraction, but registrationReady requires complete coverage.
|
||||
|
||||
`POST /scan-download` accepts `{data:base64,policy}` and returns malware-only `{decision,ruleId,engine,sha256,bytes,matchedRules,coverage}`. A YARA allow is not a DLP allow. `/inspect` applies enabled YARA first and then DLP. `/reference-document` accepts `{data:base64}` and returns registration features.
|
||||
|
||||
Dependencies and packaged model distributions are pinned with official PyPI wheel SHA-256 hashes and installed with `--require-hashes`. OCR model license review and a complete third-party software notice are still required before distribution. The Windows broker uses a Job Object limiting its process tree to four processes and 4 GiB, denying breakaway and killing descendants when the owner exits. Document parsing, YARA, OCR and inference now execute in a separate AppContainer per request. The worker verifies its actual AppContainer token and zero capabilities before importing parsers. Documents cross bounded anonymous pipes; source/runtime/model files receive package-SID read/execute access, never writable grants. Network and private-file probes, native child inheritance, deadlines, 16 MiB pipe integrity and orphan termination are exercised in `test_windows_appcontainer.py` and `artifacts/windows-appcontainer-proof.json`.
|
||||
|
||||
The AppContainer can use its Windows-provided private package storage and system resources that Windows exposes to application packages. This is not a promise that no filesystem writes anywhere are possible. Runtime roots and their ancestors reject reparse points; external Python/parser/model overrides are rejected rather than granting access to other projects. An explicit installer/deployment-owner command provisions specific-SID ACLs; normal startup reads native DACLs without changing them. Missing or writable grants fail closed. Packaging provisions the completed copied resources. Protected Program Files operation under a separate standard user remains untested. No capability or loopback exemption is added. Broker startup allows 30 seconds for cold ACL verification; each parser request retains a 29-second worker deadline and 30-second browser request budget. Broker/worker/verifier bytecode writes are disabled so cold startup does not add unapproved files. Independent exploit/bypass assessment remains required.
|
||||
|
||||
The isolated worker uses the bundled RapidOCR path. External Tesseract executables and their deployment environment are not forwarded into this boundary; a future separately validated sandbox configuration is required to support them.
|
||||
|
||||
Training retains only returned fingerprints, fragment hashes and coverage metadata. The inspector does not retain source documents or extracted plaintext, and its HTTP handlers clear transient request references after processing. Python immutable byte/string allocations and native parser/OCR buffers cannot guarantee forensic memory erasure; OS paging and crash dumps need endpoint controls. Text and bundled RapidOCR image training use memory only. Optional Tesseract writes temporary decoded image frames and removes them on normal completion; secure deletion is not guaranteed, so use the default RapidOCR mode for memory-only training.
|
||||
|
||||
Registration reports `extractionComplete` separately from `featuresComplete`, total/retained fragment counts and `fragmentHashesTruncated`. More than 1000 fragment hashes marks coverage incomplete and registrationReady false; a reduced document or separately approved whole-file fingerprint can then be used. A truncated feature set is never presented as complete protection.
|
||||
|
||||
Sources: https://yara.readthedocs.io/en/stable/yarapython.html https://pypi.org/project/yara-python/4.5.4/ https://github.com/RapidAI/RapidOCR https://pypi.org/project/rapidocr/3.8.1/
|
||||
|
||||
Google model sources: https://ai.google.dev/gemma/docs/embeddinggemma/model_card_2 https://ai.google.dev/gemma/docs/embeddinggemma/multimodal-embeddinggemma-with-sentence-transformers and https://huggingface.co/ggml-org/embeddinggemma-2-GGUF . Multimodal vectors support classification signals; unsupported parsing or incomplete extraction never becomes a DLP allow solely because a vector exists. Model similarity is probabilistic and requires customer-specific thresholds and false-negative testing. Native worker errors/timeouts are sanitized and kill the worker; startup readiness must attest CPU and no Vulkan module. Source strings and native memory are transient but cannot be securely erased by this Python adapter.
|
||||
|
||||
Actual Windows CPU validation on 2026-10-12: four real-weight fixtures passed in 25.774 seconds, covering native image embeddings, native mono 16 kHz audio embeddings, multi-window document tail coverage, and a semantic-only policy that blocks a protected paraphrase while allowing unrelated public text. All emitted vectors have 768 finite normalized values. A separate three-document check measured protected-paraphrase cosine 0.861730 versus unrelated-text 0.606880; these are synthetic fixture measurements, not a general accuracy claim. The worker handshake pins the exact hardened build `b11461-no-logits-v1`; policy/runtime manifest field `b11461` identifies its upstream runtime release. Other suffixes are refused, CPU must be attested, and loaded Vulkan is refused.
|
||||
|
||||
Repeat actual fixtures with staged resources using `SAFETYNET_TEST_NATIVE_SEMANTIC=1`, `EMBEDDING_MODEL_DIR`, and `EMBEDDING_WORKER_PATH`, then run the bundled Python `-m unittest discover -s apps/inspector -p test_semantic_real.py -v`. Routine protocol tests do not substitute for these opt-in real-model checks.
|
||||
@@ -0,0 +1,33 @@
|
||||
"""Current-user Windows capability protection; no browser profile crypto state."""
|
||||
import base64
|
||||
import ctypes
|
||||
import sys
|
||||
|
||||
class Blob(ctypes.Structure):
|
||||
_fields_=[('size',ctypes.c_uint32),('data',ctypes.POINTER(ctypes.c_ubyte))]
|
||||
|
||||
def convert(data,protect):
|
||||
crypt=ctypes.WinDLL('crypt32',use_last_error=True)
|
||||
kernel=ctypes.WinDLL('kernel32',use_last_error=True)
|
||||
allocation=(ctypes.c_ubyte*len(data)).from_buffer_copy(data)
|
||||
source=Blob(len(data),allocation);result=Blob()
|
||||
function=crypt.CryptProtectData if protect else crypt.CryptUnprotectData
|
||||
function.argtypes=[ctypes.POINTER(Blob),ctypes.c_void_p,ctypes.c_void_p,ctypes.c_void_p,ctypes.c_void_p,ctypes.c_uint32,ctypes.POINTER(Blob)]
|
||||
function.restype=ctypes.c_int
|
||||
# UI_FORBIDDEN; omit LOCAL_MACHINE so only the signed-in Windows user can decrypt.
|
||||
if not function(ctypes.byref(source),None,None,None,None,1,ctypes.byref(result)):
|
||||
raise ValueError('DPAPI rejected capability')
|
||||
kernel.LocalFree.argtypes=[ctypes.c_void_p];kernel.LocalFree.restype=ctypes.c_void_p
|
||||
try:return ctypes.string_at(result.data,result.size)
|
||||
finally:kernel.LocalFree(result.data)
|
||||
|
||||
if __name__=='__main__':
|
||||
try:
|
||||
if sys.platform!='win32' or len(sys.argv)!=2 or sys.argv[1] not in ['protect','unprotect']: raise ValueError()
|
||||
data=sys.stdin.buffer.read(32769)
|
||||
if not data or len(data)>32768:raise ValueError()
|
||||
if sys.argv[1]=='protect':output=base64.b64encode(convert(data,True))
|
||||
else:output=convert(base64.b64decode(data,validate=True),False)
|
||||
sys.stdout.buffer.write(output)
|
||||
except Exception:
|
||||
sys.stderr.write('Windows capability protection failed.\n');sys.exit(1)
|
||||
@@ -0,0 +1,340 @@
|
||||
"""Private content inspection. No external inference, raw-content logs or remote URLs."""
|
||||
import base64
|
||||
import hashlib
|
||||
import io
|
||||
import math
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unicodedata
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
os.environ['HF_HUB_OFFLINE']='1'
|
||||
os.environ['TRANSFORMERS_OFFLINE']='1'
|
||||
os.environ['HF_HUB_DISABLE_TELEMETRY']='1'
|
||||
|
||||
MAX_BYTES = 20 * 1024 * 1024
|
||||
MAX_TEXT = 1_000_000
|
||||
_MODEL = None
|
||||
_OCR = None
|
||||
|
||||
|
||||
def local_ocr():
|
||||
global _OCR
|
||||
if _OCR is None:
|
||||
import onnxruntime
|
||||
onnxruntime.disable_telemetry_events()
|
||||
import rapidocr
|
||||
from rapidocr import RapidOCR
|
||||
directory=Path(rapidocr.__file__).parent/"models"
|
||||
names={"Det":"ch_PP-OCRv4_det_infer.onnx","Rec":"ch_PP-OCRv4_rec_infer.onnx","Cls":"ch_ppocr_mobile_v2.0_cls_infer.onnx"}
|
||||
checksums={"Det":"d2a7720d45a54257208b1e13e36a8479894cb74155a5efe29462512d42f49da9",
|
||||
"Rec":"48fc40f24f6d2a207a2b1091d3437eb3cc3eb6b676dc3ef9c37384005483683b",
|
||||
"Cls":"e47acedf663230f8863ff1ab0e64dd2d82b838fceb5957146dab185a89d6215c"}
|
||||
params={"Global.log_level":"critical","EngineConfig.onnxruntime.intra_op_num_threads":2,
|
||||
"EngineConfig.onnxruntime.inter_op_num_threads":1}
|
||||
for task,name in names.items():
|
||||
model=directory/name
|
||||
if not model.is_file(): raise ValueError("Bundled offline OCR model unavailable")
|
||||
if hashlib.sha256(model.read_bytes()).hexdigest()!=checksums[task]: raise ValueError("Offline OCR model checksum mismatch")
|
||||
params[task+".model_path"]=str(model)
|
||||
# Explicit paths bypass RapidOCR's download path entirely.
|
||||
_OCR=RapidOCR(params=params)
|
||||
return _OCR
|
||||
|
||||
|
||||
def scan_download(data, policy):
|
||||
"""Malware gate only: never implies document/DLP coverage."""
|
||||
result={"decision":"block","ruleId":"yara-unavailable","engine":"yara",
|
||||
"sha256":hashlib.sha256(data).hexdigest(),"bytes":len(data),
|
||||
"matchedRules":[],"coverage":{"complete":False}}
|
||||
try:
|
||||
config=policy.get("security",{}).get("yara",{})
|
||||
if config.get("enabled") is not True:
|
||||
result["ruleId"]="yara-required";return result
|
||||
maximum=config.get("maxScanBytes",MAX_BYTES)
|
||||
if not isinstance(maximum,int) or maximum<=0 or len(data)>min(maximum,MAX_BYTES):
|
||||
result["ruleId"]="yara-size-limit";return result
|
||||
entries=config.get("rules",[])
|
||||
if not isinstance(entries,list) or not 1<=len(entries)<=128: raise ValueError()
|
||||
sources={};ids={};total=0
|
||||
for index,entry in enumerate(entries):
|
||||
source=entry["source"];identifier=entry["id"]
|
||||
if not isinstance(source,str) or not isinstance(identifier,str) or not identifier: raise ValueError()
|
||||
total+=len(source.encode("utf-8"))
|
||||
if total>262144: raise ValueError()
|
||||
# No imported modules: console can leak scanned content; others expand attack surface.
|
||||
if re.search(r'\bimport\s+"',source): raise ValueError()
|
||||
namespace="policy_"+str(index);sources[namespace]=source;ids[namespace]=identifier
|
||||
import yara
|
||||
rules=yara.compile(sources=sources,includes=False,error_on_warning=True)
|
||||
matches=rules.match(data=data,timeout=5)
|
||||
result["coverage"]["complete"]=True
|
||||
result["matchedRules"]=[{"id":ids[item.namespace],"rule":item.rule} for item in matches]
|
||||
if matches: result["ruleId"]=ids[matches[0].namespace]
|
||||
else: result["decision"]="allow";result["ruleId"]=None
|
||||
except ImportError: pass
|
||||
except Exception: result["ruleId"]="yara-inspection-failed"
|
||||
return result
|
||||
|
||||
|
||||
def reference_document(data):
|
||||
"""Return privacy-preserving registration features, without extracted plaintext."""
|
||||
result={"sha256":hashlib.sha256(data).hexdigest(),"bytes":len(data),
|
||||
"contentType":content_kind(data),"fragmentHashes":[],"coverage":{"complete":False}}
|
||||
text="";media=[];hashes=set()
|
||||
try:
|
||||
if len(data)>MAX_BYTES: raise ValueError()
|
||||
text,complete,media=extract(data,result["contentType"])
|
||||
if len(text)>MAX_TEXT: raise ValueError()
|
||||
hashes=fragment_hashes(text)
|
||||
result["fragmentHashes"]=sorted(hashes)[:1000]
|
||||
truncated=len(hashes)>1000
|
||||
result["coverage"]={"complete":complete and not truncated,"extractionComplete":complete,
|
||||
"featuresComplete":not truncated,"fragmentHashesTruncated":truncated,
|
||||
"fragmentHashesTotal":len(hashes),"fragmentHashesRetained":len(result["fragmentHashes"]),
|
||||
"textCharacters":len(text),"mediaItems":len(media)}
|
||||
result["registrationReady"]=bool(complete and not truncated and (hashes or text.strip()))
|
||||
except Exception: result["registrationReady"]=False
|
||||
finally:
|
||||
# Release transient content references; immutable Python strings cannot be securely wiped.
|
||||
text="";media.clear();hashes.clear();data=None
|
||||
return result
|
||||
|
||||
|
||||
def fragment_hashes(text):
|
||||
normalized=unicodedata.normalize("NFKC",text).lower().replace("ß","ss").replace("ς","σ")
|
||||
words=re.findall(r"\w+",normalized)
|
||||
# Every shingle matches excerpts even when their starting offset changes.
|
||||
hashes={hashlib.sha256(" ".join(words[index:index+8]).encode()).hexdigest()
|
||||
for index in range(max(0,len(words)-7))}
|
||||
for paragraph in re.split(r"\n\s*\n",normalized):
|
||||
paragraph=" ".join(paragraph.split())
|
||||
if len(paragraph)>=32: hashes.add(hashlib.sha256(paragraph.encode()).hexdigest())
|
||||
return hashes
|
||||
|
||||
|
||||
def cosine(left, right):
|
||||
if len(left) != len(right) or not left:
|
||||
raise ValueError("Embedding dimensions do not match")
|
||||
if not all(math.isfinite(float(v)) for v in [*left, *right]):
|
||||
raise ValueError("Invalid embedding")
|
||||
denominator = math.sqrt(sum(v*v for v in left)) * math.sqrt(sum(v*v for v in right))
|
||||
if not denominator:
|
||||
raise ValueError("Empty embedding")
|
||||
return sum(a*b for a, b in zip(left, right)) / denominator
|
||||
|
||||
|
||||
def content_kind(data):
|
||||
if data.startswith(b"%PDF-"): return "pdf"
|
||||
if data.startswith(b"PK\x03\x04"): return "office"
|
||||
if data.startswith(b"\x89PNG\r\n\x1a\n") or data.startswith(b"\xff\xd8\xff") or data.startswith((b"GIF87a", b"GIF89a")): return "image"
|
||||
if data.startswith(b"RIFF"):
|
||||
return {b"WAVE": "audio", b"WEBP": "image", b"AVI ": "video"}.get(data[8:12], "unknown")
|
||||
if data.startswith((b"ID3", b"fLaC", b"OggS")): return "audio"
|
||||
if len(data)>12 and data[4:8] == b"ftyp": return "video"
|
||||
try:
|
||||
text=data.decode("utf-8", errors="strict")
|
||||
if "\x00" in text: return "unknown"
|
||||
return "text"
|
||||
except UnicodeError: return "unknown"
|
||||
|
||||
|
||||
def ocr_image(data):
|
||||
from PIL import Image, ImageSequence
|
||||
image=Image.open(io.BytesIO(data))
|
||||
frames=getattr(image,"n_frames",1)
|
||||
if image.width*image.height>20_000_000 or frames>32 or image.width*image.height*frames>50_000_000:
|
||||
raise ValueError("Image processing budget exceeded")
|
||||
if os.environ.get("SAFETYNET_OCR_ENGINE","rapidocr")=="rapidocr":
|
||||
import numpy as np
|
||||
engine=local_ocr();parts=[]
|
||||
for frame in ImageSequence.Iterator(image):
|
||||
output=engine(np.asarray(frame.convert("RGB")))
|
||||
parts.extend(output.txts or [])
|
||||
return "\n".join(parts),True
|
||||
configured=os.environ.get("TESSERACT_PATH", "")
|
||||
executable=configured if configured and Path(configured).is_file() else shutil.which("tesseract")
|
||||
if not executable: return "", False
|
||||
parts=[]
|
||||
with tempfile.TemporaryDirectory(prefix="safetynet-ocr-") as temp:
|
||||
for index, frame in enumerate(ImageSequence.Iterator(image)):
|
||||
file=Path(temp)/f"frame-{index}.png"
|
||||
frame.convert("RGB").save(file)
|
||||
result=subprocess.run([executable,str(file),"stdout","--psm","3"],capture_output=True,timeout=20,check=True)
|
||||
parts.append(result.stdout.decode("utf-8"))
|
||||
return "\n".join(parts), True
|
||||
|
||||
|
||||
def extract(data, kind):
|
||||
if kind=="text": return data.decode("utf-8"), True, []
|
||||
if kind=="image":
|
||||
text,complete=ocr_image(data)
|
||||
return text,complete,[("image",data)]
|
||||
if kind=="pdf":
|
||||
from pypdf import PdfReader
|
||||
reader=PdfReader(io.BytesIO(data),strict=True)
|
||||
if reader.is_encrypted: raise ValueError("Encrypted PDF cannot be inspected")
|
||||
if len(reader.pages)>100: raise ValueError("PDF page budget exceeded")
|
||||
text=[];complete=True;media=[]
|
||||
for page in reader.pages:
|
||||
value=page.extract_text() or ""
|
||||
text.append(value)
|
||||
images=list(page.images)
|
||||
if not value.strip() and not images: complete=False
|
||||
for image in images:
|
||||
if len(image.data)>MAX_BYTES: raise ValueError("PDF image budget exceeded")
|
||||
scanned,ok=ocr_image(image.data);text.append(scanned);complete=complete and ok
|
||||
media.append(("image",image.data))
|
||||
if len(media)>100: raise ValueError("PDF media budget exceeded")
|
||||
# Attachments, JavaScript, forms, actions and vector drawings need deeper inspection.
|
||||
if page.get("/Annots"): complete=False
|
||||
root=reader.trailer["/Root"]
|
||||
if root.get("/AcroForm") or root.get("/OpenAction") or root.get("/Names"): complete=False
|
||||
# Text extraction is useful, but does not prove all visible/vector PDF content was covered.
|
||||
return "\n".join(text),False,media
|
||||
if kind=="office":
|
||||
from defusedxml.ElementTree import fromstring
|
||||
text=[];media=[];complete=True
|
||||
with zipfile.ZipFile(io.BytesIO(data)) as archive:
|
||||
members=archive.infolist()
|
||||
if len(members)>2000 or sum(x.file_size for x in members)>MAX_BYTES:
|
||||
raise ValueError("Archive expansion budget exceeded")
|
||||
if not any(x.filename in ["word/document.xml","ppt/presentation.xml","xl/workbook.xml"] for x in members):
|
||||
raise ValueError("Only OOXML document archives are supported")
|
||||
for entry in members:
|
||||
if entry.flag_bits&1 or entry.file_size>MAX_BYTES or entry.file_size>max(entry.compress_size,1)*200:
|
||||
raise ValueError("Unsafe document archive")
|
||||
name=entry.filename
|
||||
if any(s in name.lower() for s in ["vbaproject","embeddings/","activex/","customxml/"]): complete=False
|
||||
if name.endswith(".xml"):
|
||||
root=fromstring(archive.read(entry))
|
||||
text.extend(element.text for element in root.iter() if element.tag.rsplit("}",1)[-1] in ["t","v"] and element.text)
|
||||
elif "/media/" in name:
|
||||
payload=archive.read(entry)
|
||||
if content_kind(payload)!="image": complete=False; continue
|
||||
scanned,ok=ocr_image(payload);text.append(scanned);complete=complete and ok;media.append(("image",payload))
|
||||
if len(media)>100: raise ValueError("Office media budget exceeded")
|
||||
# Charts, relationships, equations and rendered layout are not yet fully inspected.
|
||||
return "\n".join(text),False,media
|
||||
if kind in ["audio","video"]: return "",False,[(kind,data)]
|
||||
raise ValueError("Unsupported content type")
|
||||
|
||||
|
||||
def get_model():
|
||||
global _MODEL
|
||||
directory=os.environ.get("EMBEDDING_MODEL_DIR", "")
|
||||
worker=os.environ.get("EMBEDDING_WORKER_PATH", "")
|
||||
if not directory or not Path(directory).is_dir() or not worker: raise ValueError("Verified local EmbeddingGemma 2 runtime is not configured")
|
||||
if _MODEL is None:
|
||||
from semantic import NativeSemanticModel
|
||||
_MODEL=NativeSemanticModel(directory,worker)
|
||||
return _MODEL
|
||||
|
||||
|
||||
def embed(data, kind):
|
||||
model=get_model()
|
||||
if kind=="text": vectors=model.text_vectors(data.decode("utf-8"))
|
||||
elif kind=="image": vectors=model.image_vectors(data)
|
||||
elif kind=="audio": vectors=model.audio_vectors(data)
|
||||
else: raise ValueError("Native semantic modality is not configured")
|
||||
return aggregate_vectors(vectors)
|
||||
|
||||
|
||||
def aggregate_vectors(vectors):
|
||||
from semantic import validate_vectors,DIMENSIONS
|
||||
validate_vectors(vectors)
|
||||
vector=[sum(item[index] for item in vectors)/len(vectors) for index in range(DIMENSIONS)]
|
||||
norm=math.sqrt(sum(value*value for value in vector))
|
||||
if norm<1e-12: raise ValueError("Degenerate semantic aggregate")
|
||||
return [value/norm for value in vector]
|
||||
|
||||
|
||||
def semantic_vector_features(text,media):
|
||||
model=get_model();vectors=model.text_vectors(text) if text.strip() else []
|
||||
text_chunks=len(vectors)
|
||||
for kind,payload in media:
|
||||
if kind=="image": vectors.extend(model.image_vectors(payload))
|
||||
elif kind=="audio": vectors.extend(model.audio_vectors(payload))
|
||||
else: raise ValueError("Native semantic modality is not configured")
|
||||
if len(vectors)>64: raise ValueError("Semantic vector budget exceeded")
|
||||
if not vectors: raise ValueError("No semantic inputs")
|
||||
return vectors,text_chunks
|
||||
|
||||
|
||||
def semantic_vectors(text,media):
|
||||
return semantic_vector_features(text,media)[0]
|
||||
|
||||
|
||||
def embedding_features(data):
|
||||
kind=content_kind(data)
|
||||
if len(data)>MAX_BYTES: raise ValueError("Embedding input budget exceeded")
|
||||
text,complete,media=extract(data,kind)
|
||||
if len(text)>MAX_TEXT: raise ValueError("Extracted text budget exceeded")
|
||||
vectors,text_chunks=semantic_vector_features(text,media)
|
||||
return {**get_model().info(),"vector":aggregate_vectors(vectors),"vectors":vectors,
|
||||
"coverage":{"complete":complete,"extractionComplete":complete,"textChunks":text_chunks,"mediaItems":len(media)}}
|
||||
|
||||
|
||||
def deterministic(text, data, policy):
|
||||
digest=hashlib.sha256(data).hexdigest()
|
||||
normalized=unicodedata.normalize("NFKC",text).casefold()
|
||||
patterns=[("private-key",r"-----BEGIN (?:RSA |EC |OPENSSH |DSA )?PRIVATE KEY-----"),
|
||||
("aws-access-key",r"\b(?:AKIA|ASIA)[A-Z0-9]{16}\b"),
|
||||
("api-secret",r"\b(?:sk-[A-Za-z0-9_-]{20,}|gh[pousr]_[A-Za-z0-9]{20,})\b")]
|
||||
for rule,pattern in patterns:
|
||||
if re.search(pattern,text): return rule
|
||||
fragments=None
|
||||
for rule in policy["dlp"]["rules"]:
|
||||
if rule["kind"]=="sha256" and rule["value"].lower()==digest: return rule["id"]
|
||||
if rule["kind"]=="literal" and unicodedata.normalize("NFKC",rule["value"]).casefold() in normalized: return rule["id"]
|
||||
if rule["kind"]=="fragment-sha256" and text:
|
||||
if fragments is None: fragments=fragment_hashes(text)
|
||||
if rule["value"].lower() in fragments: return rule["id"]
|
||||
return None
|
||||
|
||||
|
||||
def inspect_content(data, policy, encoder=embed):
|
||||
kind=content_kind(data)
|
||||
result={"decision":"block","ruleId":"uninspectable","contentType":kind,"coverage":{"complete":False},"model":None}
|
||||
if len(data)>min(MAX_BYTES,policy["dlp"]["maxBytes"]): result["ruleId"]="size-limit";return result
|
||||
if policy.get("security",{}).get("yara",{}).get("enabled"):
|
||||
malware=scan_download(data,policy)
|
||||
if malware["decision"]!="allow": return {**result,"ruleId":malware["ruleId"],"malware":malware}
|
||||
# Fingerprints can block any bytes before invoking parsers.
|
||||
rule=deterministic("",data,policy)
|
||||
if rule: result["ruleId"]=rule;return result
|
||||
try:
|
||||
text,complete,media=extract(data,kind)
|
||||
if len(text)>MAX_TEXT: raise ValueError("Extracted text budget exceeded")
|
||||
result["coverage"]={"complete":complete,"textCharacters":len(text),"mediaItems":len(media)}
|
||||
rule=deterministic(text,data,policy)
|
||||
if rule: result["ruleId"]=rule;return result
|
||||
semantic=policy["dlp"].get("semantic",{})
|
||||
if semantic.get("enabled"):
|
||||
if encoder is embed:
|
||||
info=get_model().info()
|
||||
if any(semantic.get(key)!=info[key] for key in ["model","revision","profile","dimensions","artifactFormat","quantization","runtimeRevision"]): raise ValueError("Pinned model profile unavailable")
|
||||
elif semantic.get("model")!="google/embeddinggemma-2" or semantic.get("revision")!=os.environ.get("EMBEDDING_MODEL_REVISION"): raise ValueError("Pinned test model revision unavailable")
|
||||
if not semantic.get("categories"): raise ValueError("Semantic policy has no reference categories")
|
||||
result["model"]=semantic["model"];result["similarity"]=[]
|
||||
if encoder is embed: vectors=semantic_vectors(text,media)
|
||||
else:
|
||||
inputs=[("text",text[i:i+4000].encode()) for i in range(0,len(text),3500)]+media
|
||||
if not inputs or len(inputs)>64: raise ValueError("Inference budget exceeded")
|
||||
vectors=[encoder(payload,modality) for modality,payload in inputs]
|
||||
if sum(len(category.get("prototypes") or [category["prototype"]]) for category in semantic["categories"])>256: raise ValueError("Semantic reference budget exceeded")
|
||||
for category in semantic["categories"]:
|
||||
prototypes=category.get("prototypes") or [category["prototype"]]
|
||||
if not isinstance(category.get("threshold"),(int,float)) or not math.isfinite(category["threshold"]) or not 0<category["threshold"]<=1: raise ValueError("Invalid semantic threshold")
|
||||
scores=[cosine(vector,prototype) for vector in vectors for prototype in prototypes]
|
||||
score=max(scores);result["similarity"].append({"categoryId":category["id"],"score":round(score,6)})
|
||||
if score>=category["threshold"]: result["ruleId"]=category["id"];return result
|
||||
if complete: result["decision"]="allow";result["ruleId"]=None
|
||||
return result
|
||||
except Exception:
|
||||
# Parser/model errors never release content and never serialize raw exception text.
|
||||
result["ruleId"]="inspection-incomplete";return result
|
||||
@@ -0,0 +1,37 @@
|
||||
"""Installer-only ACL provisioning for this exact SafetyNet installation.
|
||||
|
||||
Run with the installed private Python. No caller-controlled grant roots exist.
|
||||
Normal service startup verifies ACLs without changing them.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
sys.dont_write_bytecode=True
|
||||
from windows_appcontainer import AppContainer, SandboxError
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--provision', action='store_true', help='Installer/owner only: establish read/execute ACLs')
|
||||
args = parser.parse_args()
|
||||
source = Path(__file__).absolute().parent
|
||||
development = source.parent.name == 'apps'
|
||||
installation = source.parent.parent if development else source.parent
|
||||
runtime = installation / ('.runtime/windows/python' if development else 'python')
|
||||
semantic = installation / ('.runtime/windows/semantic' if development else 'semantic')
|
||||
if Path(sys.executable).resolve().parent != runtime.resolve():
|
||||
raise SandboxError('Use the private Python belonging to this installation')
|
||||
roots = [source, runtime]
|
||||
if semantic.is_dir(): roots.append(semantic)
|
||||
sandbox = AppContainer(roots, provision=args.provision)
|
||||
print(json.dumps({'verified': True, 'provisioned': args.provision, 'rootCount': len(roots), 'profile': sandbox.name}))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
try: main()
|
||||
except (SandboxError, OSError):
|
||||
print('Approved installation ACL verification/provisioning failed.', file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
@@ -0,0 +1,3 @@
|
||||
# Native offline EmbeddingGemma 2 inference uses the verified GGUF worker.
|
||||
# No sentence-transformers, PyTorch or model-download client is needed at runtime.
|
||||
-r requirements.txt
|
||||
@@ -0,0 +1,26 @@
|
||||
certifi==2026.7.22 --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775
|
||||
charset-normalizer==3.5.2 --hash=sha256:195c26fb65950f8fce54e26349852b7bdd7c5f120aeefbcc440b8a20faaed4a3 --hash=sha256:9373ad13ef0d2c0fb761e04e55bfdee5a08b52cef2c882c8fbe9935b1517152e --hash=sha256:ddf19c062bea7a0cc80f519243d2c01dd091be0cf952a0750d4ad576709559f5 --hash=sha256:3d14b50de6bf4d0edf857a9386836846f982b8f524e188e2e68b96d702bcf4aa --hash=sha256:28a15fdad492a99b6eccfaaed66ef3f74050680545ea61ec8b2f4c538f1f1320 --hash=sha256:8a893cc101149f80a653f82062ebc95b34525a2614382e1da5458fe7c6997249 --hash=sha256:619799369eeef6366ed3e8755a5670f4f2f0fb6b30a0fd7264dc0fdc2357058e --hash=sha256:447441e76ec720b15e64418d32e092297340387053047c7c694f579efb0ee1d9 --hash=sha256:62588a277bfb59def052abd940703fa35107152bf479781a878617d60faf8fb5 --hash=sha256:44bd4fbb29dfbeba60e7d2bd000c59e4b21ddb3cc53912b14048d37092706d7c --hash=sha256:30fcd120b732aa79317f08dee04d7de0847822e4cf7ee0e9f445bb958832252c --hash=sha256:50e3adfb96fc189eb27b1cf62d3b598b89b4bb0420d93a3d3e42e137409011be --hash=sha256:b736353c0a625bbd5fcec108576e2385db3496f4f771f785ff32e108d3c3bc45 --hash=sha256:f5833ad231be5eb6553de524a70f48d71b2c8563101750531e0b80184e175cd4 --hash=sha256:1461ac396c4fdb983a675f20aa555624f0ee18ac83d832b9244ffff3d8055275 --hash=sha256:c6708715abcf3c73b99508253e961a9967f02fe536532834149574eda6de0d1c --hash=sha256:3d21b8b13c7592db2ac5e544a6d83187b995257472b0c9e8351b6d507ae37ed6 --hash=sha256:d760fe2a4d7c3b226cb9026d6a842868d52a7901bd98420e1baf14e80da85cf5 --hash=sha256:c9790464842f85f437dbbb54417eda1e0e6bfc52dd8d22d6fd1c994b73b2dc74 --hash=sha256:4685902cf26edf013ed7a3da0f426ebba7a00ebb9541386d835afbf002c11cab --hash=sha256:4495c5002a7b28557e7e222e77e0b661183e432b7d6d2e788101e3f240e05b8c --hash=sha256:211d5a3eb6af8f513b8d4ca19a8c1b7accab1b5f0d3175f9826b03c1a920dc1f --hash=sha256:ef4fcbf3327382cd4c9f540babd61248208af7b93eec4de397b4d5f58a09e288 --hash=sha256:bd16aabe4a02a297c23417aa17ac6299dbd8c49f673bcd645b4929b11f5a4400 --hash=sha256:fb9e68df06293761f9fe66ade60a9bc6d0f5e42b8acf2939a9158af86ab0e5bd --hash=sha256:59f63901b0031c3136cf64704dcb21de0bbae62ce2c9529bc39d27665463de37 --hash=sha256:304d5463e65a35d7bb0850550e0780395395f6fcf452f04db7d5ca7cecc425ac --hash=sha256:9cf9b1a857e25c4baceeb3624e92a56df3668f398c4acba74e174d81fb4d1d3a --hash=sha256:114e4d0c92d618409ed82a99e22b5c5e768fe995f2973f78265f4524f49d4640 --hash=sha256:2625388c6c754520c37abaf3b41eb34d1cc4a373f457898f08606c8e362b891d --hash=sha256:87e50a3e7cb90af586b6c5faf23e302a970415ac73bd7bd90a515a04b427ef96 --hash=sha256:254eb48b9fa5ee9898a3c445825a1f340fe53712a098904b39b0bddba8ea3cb1 --hash=sha256:ed2a239c0ea213acc1908150a3037257083c7c083128f1a4cec2ec4b97dca491 --hash=sha256:b91363207bd9dc966a691e959bb47f64b30f7ac4b072be9968b366982f7db77c --hash=sha256:38a873987f3be698494da8b2e3085e29da02da7b633dce73e79c699a113d7bf0 --hash=sha256:355ad8011081dec5412240c087a9a0c9d4d5039f3ed11a3f13e18c2b29b56c51 --hash=sha256:ee21e28f0430bd6dc9086c6e525d5e818a44a5ad19720c8a0ef766792f3eb5e5 --hash=sha256:3d31298449090ab8d47b7b1b2a555ff73cac7ed438a08b7ac160980c7ebed649 --hash=sha256:5cde776b7cc66e4f6c99612cea4aa7269aa65863f7a15841b2c264f103822f4e --hash=sha256:ae4f5fea5b8b8ccff88238cc8569303e5ee95efae67fa62922a311397a71f346 --hash=sha256:f7d486c83842422badd511868fd8a9a20e9407ace71564b6af47ce7e60a336c1 --hash=sha256:11a4d68a6ecda3292cb1e50239e111543ba5d709bb62a6b4ea1afcfa729d8875 --hash=sha256:d6734d2ef8a50fbf8445c139477da401f50d62a0606bf00e20ec6d87773fefb1 --hash=sha256:a815775b6c38d4e0ff7bcffbeba67feded90202bb6a226b8dd35f1c855217413 --hash=sha256:23851fb4e1b85ed3f6c2a27b777cdfe2e19fb5b38429a8faf38c7542b7665869 --hash=sha256:db19d07e2e0129e974a0e65d0064fc222a446cd5122c2fd4184d2af9fc734a9e --hash=sha256:780fbe7cab297b81dad9fb8dc5eb003c0468ffb0d9e5f65068c53a34661a96bc --hash=sha256:e2af3aad578aa6bd1384bcf4750fc285e5a9de53f40b7d41e5a0bf748edeb2b3 --hash=sha256:ed905975ab14056a2e5eb1c376cb2e1ebc5396baf84163939c518556fccde9f5 --hash=sha256:a66c3bc5ab1f0ff2164fc9965ddd611ff0802173f4b9d24554c563f6ab7e1d6e --hash=sha256:d2374b62878abb00cd8309b32af6c0b715cd02dec0ca74ef12e5069bdc64144a --hash=sha256:d376bbd28b3a8999db1a103b3b388aee6f1ddeb3e51bc2172993efdcd86e064d --hash=sha256:6045373d5a89a5ec71afde535db987ca28e76dfa276c2d4c818265b375d4b055 --hash=sha256:849df64e889b2e17230d58410a03dba311a65b163508fd33679b2b737d4b7858 --hash=sha256:15c44f7edfd477b06f517a5cc317fc1707edb9de2c865f43d4b6513907473234 --hash=sha256:a89012d6d5476ee112d20d998570ed58df2260a852afb1758809cd6900411d21 --hash=sha256:0c951d5e6dd9c2ff60609476752bee49da4206adde960ebc247766937f72e718 --hash=sha256:7218e8f32b0956cfcd048fd42d9d5779809745ca1d86113ca56f66e7ae1549c4 --hash=sha256:a19a731138fc27d5682277d3b9df22855cea1239bce7fcec5f78f42ef2d1f3c3 --hash=sha256:62603db9a7caa0802eaa28c1c46fecd7b3a263a774069c24c3c28c302448721c --hash=sha256:b6856554c4f44d79fc2307d5768854310a8f0096e501c75637542c82292b0429 --hash=sha256:1bc0baf5ef96b6ede57d47f4b8fe4d9d84019c3bfcbeb20a41edc6a6ee341f1f --hash=sha256:56bc200a365efb37383b7852e4cc5898d3b2da5987289b543956cf8cad710Line truncated
|
||||
colorama==0.4.6 --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6
|
||||
colorlog==6.12.0 --hash=sha256:30d392604e9110045a2c2aeefc27d7a017abbab63f3a8aee594eac0801df784e
|
||||
defusedxml==0.7.1 --hash=sha256:a352e7e428770286cc899e2542b6cdaedb2b4953ff269a210103ec58f6198a61
|
||||
flatbuffers==25.12.19 --hash=sha256:7634f50c427838bb021c2d66a3d1168e9d199b0607e6329399f04846d42e20b4
|
||||
idna==3.20 --hash=sha256:ab7ae7122974553370f0bdb919e1a960b2cd1bc1ef0276416d896db81c14582c
|
||||
mpmath==1.3.0 --hash=sha256:a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c
|
||||
numpy==2.5.4 --hash=sha256:c6342f54c67093cae5c0227eb0eb772fdb79f2a2c37a6eb278b9909ee06aa356 --hash=sha256:b11e8fda06a7d69f15ebf542660b74466c2e51094800c1fb794f47ad4faeef17 --hash=sha256:9cb18a327b49c5c337f972b03682f6a49855525faaf3c0d3e9c96cd0fd8880a8 --hash=sha256:aec3fc4b32ff82421274f5d205c559c51c840c8df66a78efd7f3612dd005a26a --hash=sha256:fe4d21ab149f15e4e6043dfb0de87e6e5f34ac176cde83060e9802981fca2ac2 --hash=sha256:fbde6962867ee75b48b0ee29b2b9372ec5d617799dbaf38e82dc0596f2f7738a --hash=sha256:381a7a3d2e65e64c0ec302795ab9dc12bb1e73f150904699c153716177eebdaf --hash=sha256:b89d0aaae2fe498c648f4c4795c084db535af5bd98ef942b2a3681fb74ce8645 --hash=sha256:9968ab7e49b93ac6e1c3b2239732183152c9150f16308d30b66a372cffe3483c --hash=sha256:a7b1b6353e36a7e50de2973a38d705c88ee93adcf120673cee7f45a4a3fa223a --hash=sha256:aa1cce2ff3f8d953de38b76bf44602caeb69f101430208f64a10067f7cb4b1d3 --hash=sha256:2377da2dd3ba2c1200956acbab2a358c83b8e1f8531191672d1cd6ad83250d53 --hash=sha256:7415db95818b39ec475a5eea54d9e3b6bc83e3912158e46da3438cdce399804d --hash=sha256:6d6a71b9d9a97c03633aa12565ef2825ffa036cc1d99cfd50dacf0f128af4fe2 --hash=sha256:d8200f16437b289a5bb927c6e184eccc3e8389bc0070fea4cd5b9e13c1757959 --hash=sha256:1c2e71b04c6cad90026e544501bbe0ab9290fa8a4d845e7e8c0d124fb429c988 --hash=sha256:6ffa07666f8da0eef81d149934a626d0d95fbd6838432a33e66245423a9062c0 --hash=sha256:2fa3328f784fc8277fc48026f6cad516f5c561c5d8e2e39b3c9e0c8f23223b34 --hash=sha256:b86966fbe4ad7de710422175572bcdc75fdedadfb54bc6fab7deabccddd7780b --hash=sha256:5258bc06526964be5face2fc6f756857a3f24f21ec3e72ca131337a75b165d6c --hash=sha256:8b4d2fd2d34e5f8c9235ee787de5631a37a28402b15cb80814df973d2be54129 --hash=sha256:bc39ac66a7a9a3fbd6134fda43136b60ffde99c8f4501e64e0d2b24da137babf --hash=sha256:c668b2f0d651605b58892644b0e302c7157f7159544227758c896982ef384b18 --hash=sha256:ffa6ce09a1c6a08e9667dd9c97aa0b14184e8d18f2a14b78b2a2328c9147f076 --hash=sha256:956555e0603a4d38019ae6925711cb9dc43195c076a928accf7ea5d50bddfe53 --hash=sha256:2c2c4afffdeb7920e445028dd71eb932cac3e704792e964bc2a232426d4f1255 --hash=sha256:4054173604cd8658796053f1f3bc0befb68ec1c0762c57fdad61e199256a8617 --hash=sha256:d549420b8858885cea8838a727842249218b9c1da24dd517e25c9c7a948310a3 --hash=sha256:823874a507a84af050493b622affde94b6f7c3a0dc22cb2801381bc03b871c00 --hash=sha256:4e263278bfb5ee6409db8aedbc4cc32973b1b82bc1e8d3c668551d04d83a7e37 --hash=sha256:1ef3aa6d7e29bb13677323114280b05acc57607fa2300e66432d665d5418a162 --hash=sha256:98b053943e5a0474ec0da309d2cb9d3f18ea57f8a2067c2ab7b5f763d1068380 --hash=sha256:b64a85f40e154983960a4167d4c1d57a50c7f109b3d3264a3a984154e90a8454 --hash=sha256:a813ed7719bf45463c51779e6a98d0385fe905e48447526938a4b8337333d551 --hash=sha256:c9b80cdf5cedba0e90d93fa5f9a333c4d65bd545cd669b71bb97ce2b703c9d73 --hash=sha256:2199ed071f460487c8db2c0e5c0b564494190edb4772fe80f9aad88b2604def5 --hash=sha256:64f9c9878c1938476365e11ccfb6b770f3b9e5f045ccddc514235041e6959365 --hash=sha256:64d1c8ac28a4077cf987e0a71a7a0ef7e2df70722f07f0baa42dbb7eb6938647 --hash=sha256:067374eb538c34c745436365cf7b0112595c1d326f21ce4ff340f61230239fbb --hash=sha256:e94aef2c639da4a960ad0db8e06471208d8589974953d78b61d345b4eb99e394 --hash=sha256:cfd73180400042a7c532d30c5e287bdd03c59ff9ee1b4c0316af0539e29dfe23 --hash=sha256:2ca144f15135b6212a5c47b1e2aeca6e412f102f95a2d5d88d8aec77eb255de3 --hash=sha256:468397ba3c64427474706e5c9123fe266395496714dc684294eac75cd4930d1e --hash=sha256:8dddfbee2e68d26d0d7d7d9cb247b1fd4409241cce32d815a11d97ec2cfde179 --hash=sha256:81e3420b27048b65eb14c3acf0c174a8cb0e023277716110347d2dcb26026dad --hash=sha256:0b4724a19de67bea8cfc4970798efa78bcbbe2ac2613cfac16721a42d44de2a5 --hash=sha256:2132418bf8dd124a427ca9e6a1daf9ee1a87185344c95119ceae868b99466da1 --hash=sha256:325518d4245b9e331387702aa58c2ce1dc4cdcbb41dfb4ccd5dcbc7e08db1266 --hash=sha256:56733449d2544178beaa4545cee357370440cf056c197f9c7bfb19dbfdd0e86d --hash=sha256:5ec3753760c1a6d8bb91200666e545c3a9728e6269dfb5d6ce02340996698aa3 --hash=sha256:b1185012870173de7ae33d370bd45b1cf5baee747ea4b97036b65f4e93016877 --hash=sha256:950ea81d57ef070665581b6e1b5f6a029306423cd1739c5b95fe78aa30db6b9d --hash=sha256:c05ede731b03fb1b7591faca9389ade3267d2bddf1ad8882bb3f2cc5e101694f --hash=sha256:5fbf7141bbfd63aea22f435c9062a032b9ea0082fe9845dad7f021d3f1234e71 --hash=sha256:3573cd22564692a5b899ec344e5d5b9cc4576f2985b96f22af3564ed54f2710f --hash=sha256:6c109eac9cd439193678f69d70733c1108487546ca8eafc107b510ae10c1aecd --hash=sha256:80d6ef6e8620eb2c2b4c4caad50b5935d6db3cde2d51581b55dcc79e14016d1d --hash=sha256:77045a4b175bbf5316ec08003880804336c78f92281a1b72222b274ea85ec5ac --hash=sha256:0f02a46e49cfb6c73bdb7aea1c0d3461dbae9aba613542b65f657cd3d17b9fab --hash=sha256:ad62a416ddcf863bf44bba76fbf6b53366ab0692e294f51cae4b5fbe0d246788 --hash=sha256:38f47be9f74ab870d2633b5456ae519c43758a8d1fd05342f0ce4ecc034396ee --hash=sha256:7a14a461d9340f1b46b8648578aed9cdb8b3b018a8fac6c1dde2c9192a01a87f --hash=sha256:298eca75243f2cbbfdb460560b9fb2a1792a33cf2ab4286efd43d92e8d3df508 --hash=shLine truncated
|
||||
omegaconf==2.4.0 --hash=sha256:f141a664c777582405e3cc6e0545ecc79429e79f844995b1a6e71c35946a025c
|
||||
onnxruntime==1.24.4 --hash=sha256:0bdfce8e9a6497cec584aab407b71bf697dac5e1b7b7974adc50bf7533bdb3a2 --hash=sha256:046ff290045a387676941a02a8ae5c3ebec6b4f551ae228711968c4a69d8f6b7 --hash=sha256:e54ad52e61d2d4618dcff8fa1480ac66b24ee2eab73331322db1049f11ccf330 --hash=sha256:b43b63eb24a2bc8fc77a09be67587a570967a412cccb837b6245ccb546691153 --hash=sha256:e26478356dba25631fb3f20112e345f8e8bf62c499bb497e8a559f7d69cf7e7b --hash=sha256:cad1c2b3f455c55678ab2a8caa51fb420c25e6e3cf10f4c23653cdabedc8de78 --hash=sha256:1a5c5a544b22f90859c88617ecb30e161ee3349fcc73878854f43d77f00558b5 --hash=sha256:0d640eb9f3782689b55cfa715094474cd5662f2f137be6a6f847a594b6e9705c --hash=sha256:535b29475ca42b593c45fbb2152fbf1cdf3f287315bf650e6a724a0a1d065cdb --hash=sha256:e6214096e14b7b52e3bee1903dc12dc7ca09cb65e26664668a4620cc5e6f9a90 --hash=sha256:e99a48078baaefa2b50fe5836c319499f71f13f76ed32d0211f39109147a49e0 --hash=sha256:dc4aaed1e5e1aaacf2343c838a30a7c3ade78f13eeb16817411f929d04040a13 --hash=sha256:e30c972bc02e072911aabb6891453ec73795386c0af2b761b65444b8a4c4745f --hash=sha256:1700f559c8086d06b2a4d5de51e62cb4ff5e2631822f71a36db8c72383db71ee --hash=sha256:4c74e268dc808e61e63784d43f9ddcdaf50a776c2819e8bd1d1b11ef64bf7e36 --hash=sha256:3b6ba8b0181a3aa88edab00eb01424ffc06f42e71095a91186c2249415fcff93 --hash=sha256:71d6a5c1821d6e8586a024000ece458db8f2fc0ecd050435d45794827ce81e19 --hash=sha256:fbff2a248940e3398ae78374c5a839e49a2f39079b488bc64439fa0ec327a3e4 --hash=sha256:e2b7969e72d8cb53ffc88ab6d49dd5e75c1c663bda7be7eb0ece192f127343d1 --hash=sha256:14ed1f197fab812b695a5eaddb536c635e58a2fbbe50a517c78f082cc6ce9177 --hash=sha256:aa12ddc54c9c4594073abcaa265cd9681e95fb89dae982a6f508a794ca42e661 --hash=sha256:e1cc6a518255f012134bc791975a6294806be9a3b20c4a54cca25194c90cf731 --hash=sha256:311e309f573bf3c12aa5723e23823077f83d5e412a18499d4485c7eb41040858 --hash=sha256:3f0b910e86b759a4732663ec61fd57ac42ee1b0066f68299de164220b660546d
|
||||
opencv-python==5.0.0.93 --hash=sha256:198a75138241810206a17c829dbcc40a7cb1841cda538ca86cbbfc6c7d95f898 --hash=sha256:6bbc32f59e1b1a7db7b39c81f63d00625f041d333037fd8702f6da52cc39108b --hash=sha256:e2b4272e736836f66c2d176e43ab8101f3a00d45654916399f52e150c58981ac --hash=sha256:f8b6d0a212253dd26ad338c812f1f23ca118fdf05a9c8c6b9444f161aa8c5881 --hash=sha256:08d5d91d967b58d6db86073b2ad3eaef88ca4ebdfd45c9059bf59f5ded0c7ad2 --hash=sha256:c8de2dec111122a02e8beb28e16c31904992dfd6186560b142a92c71403c1039 --hash=sha256:4b4b1a34c79bf8d3738e3cfe9a9e67b51a79663f6b692cbdad8c31f570da4157 --hash=sha256:f90ba04b8f73bc5c3814037699739f0156f597338a98f05956c684e7c3ca10d2
|
||||
packaging==26.3 --hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c
|
||||
pillow==12.3.0 --hash=sha256:6c0016e7b354317c4e9e525b937ac8596c38d2d232b419529b9cd7a1cd46e39a --hash=sha256:bcc33feacfaefce60c12fd500a277533bdc02b10a19f7f6d348763d8140bbba7 --hash=sha256:5594fc43d548a7ed94949d139aa1341b270f1863f11cfd37f5a6c8b778a6b67f --hash=sha256:f0606c8bf2cdefea14a43530f7657cbbb7ecf1c4222512492ef4a4434a9501ec --hash=sha256:85f998ea1848bc6757289e739cfbdda3a04adfd58b02fc018ce54d754a5ce468 --hash=sha256:25b9b82bb22e6e2b3cd07b39c68b7b862001226cb3dff7130d1cb914121b39ed --hash=sha256:37dc8f7bbb66efe481bb60defacef820c950c24713fb44962ed6aa2a50966de1 --hash=sha256:300557495eb45ebb8aec96c2da9c4be642fbf7cd937278b4013ba894ea8eb0eb --hash=sha256:514435a37670e3e5e08f3945b68718b6ed329bb84367777e16f9f4dfe1e61a0f --hash=sha256:00808c5e14ef63ac5161091d242999076604ff74b883423a11e5d7bbb38bf756 --hash=sha256:37d6d0a00072fd2948eb22bce7e1475f34569d90c87c59f7a2ec59541b77f7a6 --hash=sha256:bcb46e2f9feff8d06323983bd83ed00c201fdcab3d74973e7072a889b3979fcd --hash=sha256:23d27a3e0307ec2244cc51e7287b919aa68d097504ebe19df4e76a98a3eea5bd --hash=sha256:4f883547d4b7f0495ebe7056b0cc2aea76094e7a4abc8e933540f3271df27d9c --hash=sha256:236ff70b9312fb68943c703aa842ca6a758abfa45ac187a5e7c1452e96ef72b5 --hash=sha256:10e41f0fbf1eec8cfd234b8fe17a4caac7c9d0db4c204d3c173a8f9f6ef3232b --hash=sha256:8e95e1385e4998ae9694eeaa4730ba5457ff61185b3a55e2e7bea0880aef452a --hash=sha256:ebaea975e03d3141d9d3a507df75c9b3ec90fa9d2ffd07567b3a978d9d790b26 --hash=sha256:ba09209fbe443b4acccebe845d8a138b89a8f4fbaeedd44953490b5315d5e965 --hash=sha256:ffd0c5368496f41b0944be820fcb7a838aa6e623d250b01acf2643939c3f99d7 --hash=sha256:d9c7f76c0673154f044e9d78c8655fb4213f6ca31a836df48b40fe5d187717b9 --hash=sha256:78cb2c6865a35ab8ff8b75fd122f6033b92a62c82801110e48ddd6c936a45d91 --hash=sha256:e491916b378fba47242221bb9ead245211b70d504f495d105d17b14a24b4907c --hash=sha256:0dd2064cbc55aaec028ef5fbb60fa47bb6c3e7918e07ff17935284b227a9d2df --hash=sha256:dbce0b29841537a2fa4a214c2bbf14de3587c9680caa9b4e217568472490b28f --hash=sha256:a2b55dd6b2a4c4b7d87ffa56bdb33fdc5fdb9a462173861a7bc097f17d91cb09 --hash=sha256:331b624368d4f1d069149002f25f44bc61c8919ce8ddb3c45bdad8f6e2d89510 --hash=sha256:21900ce7ba264168cd50defae43cd75d25c833ad4ad6e73ffc5596d12e25ac89 --hash=sha256:4e8c2a84d977f50b9daed6eeaf3baef67d00d5d74d932288f02cb94518ee3ace --hash=sha256:ae26d61dfa7a47befdc7572b521024e8745f3d809bd95ca9505a7bba9ef849ec --hash=sha256:7a743ff716f746fc19a9557f60dab1600d4613255f8a7aeb3cdde4db7eb15a66 --hash=sha256:d69141514cc30b774ceea5e3ed3a6635c8d8a96edf664689b890f4089111fb35 --hash=sha256:f7401aebd7f581d7f83a439d87d474999317ee099218e5ad25d125290990ba65 --hash=sha256:0847a763afefb695bc912d7c131e7e0632d4edc1d8698f58ddabec8e46b8b6d3 --hash=sha256:571b9fcb07b97ef3a492028fb3d2dc0993ca23a06138b0315286566d29ef718a --hash=sha256:756c768d0c9c2955feb7a56c37ea24aea2e369f8d36a88da270b6a9f19e62b5e --hash=sha256:a876864214e136f0eb367788dbd7df045f4806801518e2cfe9e13229cfe06d8f --hash=sha256:1cca606cd25738df4ed873d5ad46bbdb3d83b5cbca291f6b4ff13a4df6b0bbe8 --hash=sha256:b629de27fda84b42cde7edef0d85f13b958b47f6e9bbcbba9b673c562a89bd8b --hash=sha256:9cf95fe4d0f84c82d282745d9bb08ad9f926efa00be4697e767b814ce40d4330 --hash=sha256:8728f216dcdb6e6d555cf971cb34076139ad74b31fc2c14da4fafc741c5f6217 --hash=sha256:a45650e8ce7fafffd731db8550230db6b0d306d181a90b67d3e6bca2f1990930 --hash=sha256:ba54cfebe86920a559a7c4d6b9050791c20513650a1952ebe3368c7dc70306f8 --hash=sha256:e158cb00350dc278f3b91551101aa7d12415a66ebf2c91d8d5ac14e56ddd3ad0 --hash=sha256:e9aeb04d6aef139de265b29683e119b638208f88cf73cdd1658aa07221165321 --hash=sha256:251bf95b67017e27b13d82f5b326234ca62d70f9cf4c2b9032de2358a3b12c7b --hash=sha256:fe3cca2e4e8a592be0f269a1ca4835c25199d9f3ce815c8491048f785b0a0198 --hash=sha256:23aceaa007d6172b02c277f0cd359c79492bbb14f7072b4ede9fbcaf20648130 --hash=sha256:164b31cd1a0490ab6efae01aa5df49da7061be0af1b30e035b6e9a1bfe34ee6e --hash=sha256:5afb51d599ea772b8365ae807ae557f18bccfe46ab261fd1c2a9ed700fc6eb17 --hash=sha256:3edce1d53195db527e0191f84b71d02022de0540bf43a16ed734ed7537b07385 --hash=sha256:bf16ba1b4d0b6b7c8e534936632270cf70eb00dbe09005bc345b2677b726855c --hash=sha256:24870b09b224f7ae3c39ed07d10e819d06f8720bc551847b1d623832b5b0e28d --hash=sha256:30f2aa603c41533cc25c05acd0da21636e84a315768feb631c937177db558931 --hash=sha256:4b0a7fe987b14c31ebda6083f74f22b561fd3739bc0ac51e019622e3d72668c7 --hash=sha256:962864dc93511324d51ddbb5b9f8731bf71675b93ca612a07441896f4688fb8c --hash=sha256:0740a512dc522224c77d9aa5a8d70d8b7d73fb91f2c21125d8d025d3b8990e45 --hash=sha256:af8d94b0db561cf68b88a267c5c44b49e134f525d0dc2cb7ed413a66bc23559a --hash=sha256:fdafc9cce40277e0f7a0feabce0ee50dd2fa1800f3b38015e51296b5e814048d --hash=sha256:e91206ee562682b51b98ef4b26a6ef48fd84e15fd4c4bc5ec768eb641d206838 --hash=sha256:0feb2e9d6ad6c9e3c06effe9d00f3f1e618a6643273576b016f591e9315a7139 --hash=sha256:9e881fca225083806662a5c43d627d215f258ff43c890f831966c7d7ba9c7402 --hash=sha256:4998562bf62a445225f22e07c896bb04b35b1b1f2eb6d760584c9c51d7a5f78c --hash=Line truncated
|
||||
protobuf==7.36.2 --hash=sha256:cbc70b17ee27e28894c7fee8bb04be1abead49e936bc70eb60052531eee2079e --hash=sha256:e11e1f0180583a2af89db6a2ecd9e8dc40aa6d2988ca175bfd0e6d12ea72d74e --hash=sha256:f4fee11ec330d238b34a05c9b675f693c20415d1c5bd7d5320cc2f8a798eb9cf --hash=sha256:89f23aa53c24553a2416fd4fd1ec06f74fa42b14b546d8883128813f775bbfd2 --hash=sha256:912c1221170e16c08d1f086762f563dd61ff83c18b5fa6652952dfaded66f728 --hash=sha256:a300819d441e078a5608c0d3c709796bb548136058fda017ae51d425b44fd353 --hash=sha256:bdb3a345d48db958e6ce1f18e508beb0cc981d64f24088427549c866cd039f1e
|
||||
pyclipper==1.4.0 --hash=sha256:bafad70d2679c187120e8c44e1f9a8b06150bad8c0aecf612ad7dfbfa9510f73 --hash=sha256:0b74a9dd44b22a7fd35d65fb1ceeba57f3817f34a97a28c3255556362e491447 --hash=sha256:0a4d2736fb3c42e8eb1d38bf27a720d1015526c11e476bded55138a977c17d9d --hash=sha256:b3b3630051b53ad2564cb079e088b112dd576e3d91038338ad1cc7915e0f14dc --hash=sha256:8d42b07a2f6cfe2d9b87daf345443583f00a14e856927782fde52f3a255e305a --hash=sha256:6a97b961f182b92d899ca88c1bb3632faea2e00ce18d07c5f789666ebb021ca4 --hash=sha256:adcb7ca33c5bdc33cd775e8b3eadad54873c802a6d909067a57348bcb96e7a2d --hash=sha256:fd24849d2b94ec749ceac7c34c9f01010d23b6e9d9216cf2238b8481160e703d --hash=sha256:1b6c8d75ba20c6433c9ea8f1a0feb7e4d3ac06a09ad1fd6d571afc1ddf89b869 --hash=sha256:58e29d7443d7cc0e83ee9daf43927730386629786d00c63b04fe3b53ac01462c --hash=sha256:a8d2b5fb75ebe57e21ce61e79a9131edec2622ff23cc665e4d1d1f201bc1a801 --hash=sha256:e9b973467d9c5fa9bc30bb6ac95f9f4d7c3d9fc25f6cf2d1cc972088e5955c01 --hash=sha256:222ac96c8b8281b53d695b9c4fedc674f56d6d4320ad23f1bdbd168f4e316140 --hash=sha256:f3672dbafbb458f1b96e1ee3e610d174acb5ace5bd2ed5d1252603bb797f2fc6 --hash=sha256:d1f807e2b4760a8e5c6d6b4e8c1d71ef52b7fe1946ff088f4fa41e16a881a5ca --hash=sha256:ce1f83c9a4e10ea3de1959f0ae79e9a5bd41346dff648fee6228ba9eaf8b3872 --hash=sha256:3ef44b64666ebf1cb521a08a60c3e639d21b8c50bfbe846ba7c52a0415e936f4 --hash=sha256:d1e5498d883b706a4ce636247f0d830c6eb34a25b843a1b78e2c969754ca9037 --hash=sha256:d49df13cbb2627ccb13a1046f3ea6ebf7177b5504ec61bdef87d6a704046fd6e --hash=sha256:37bfec361e174110cdddffd5ecd070a8064015c99383d95eb692c253951eee8a --hash=sha256:14c8bdb5a72004b721c4e6f448d2c2262d74a7f0c9e3076aeff41e564a92389f --hash=sha256:f2a50c22c3a78cb4e48347ecf06930f61ce98cf9252f2e292aa025471e9d75b1 --hash=sha256:c9a3faa416ff536cee93417a72bfb690d9dea136dc39a39dbbe1e5dadf108c9c --hash=sha256:d4b2d7c41086f1927d14947c563dfc7beed2f6c0d9af13c42fe3dcdc20d35832 --hash=sha256:7c87480fc91a5af4c1ba310bdb7de2f089a3eeef5fe351a3cedc37da1fcced1c --hash=sha256:81d8bb2d1fb9d66dc7ea4373b176bb4b02443a7e328b3b603a73faec088b952e --hash=sha256:773c0e06b683214dcfc6711be230c83b03cddebe8a57eae053d4603dd63582f9 --hash=sha256:9bc45f2463d997848450dbed91c950ca37c6cf27f84a49a5cad4affc0b469e39 --hash=sha256:f160a2c6ba036f7eaf09f1f10f4fbfa734234af9112fb5187877efed78df9303 --hash=sha256:a9f11ad133257c52c40d50de7a0ca3370a0cdd8e3d11eec0604ad3c34ba549e9 --hash=sha256:bbc827b77442c99deaeee26e0e7f172355ddb097a5e126aea206d447d3b26286 --hash=sha256:29dae3e0296dff8502eeb7639fcfee794b0eec8590ba3563aee28db269da6b04 --hash=sha256:0b8c2105b3b3c44dbe1a266f64309407fe30bf372cf39a94dc8aaa97df00da5b --hash=sha256:6c317e182590c88ec0194149995e3d71a979cfef3b246383f4e035f9d4a11826 --hash=sha256:98b2a40f98e1fc1b29e8a6094072e7e0c7dfe901e573bf6cfc6eb7ce84a7ae87
|
||||
pypdf==6.20.1 --hash=sha256:aa5a55ddcffdc5e5ab291d5decb23f6383f4e56f8e3263dc39af41fff03885ad
|
||||
PyYAML==6.0.3 --hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b --hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 --hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 --hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 --hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b --hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 --hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 --hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e --hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c --hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e --hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 --hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c --hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 --hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d --hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a --hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 --hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b --hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf --hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 --hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 --hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c --hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc --hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e --hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea --hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 --hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b --hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd --hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 --hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 --hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c --hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 --hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 --hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 --hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be --hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 --hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c --hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb --hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac --hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 --hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 --hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 --hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 --hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 --hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 --hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 --hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba --hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c --hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c --hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 --hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 --hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 --hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b --hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac --hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 --hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f --hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 --hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 --hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 --hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 --hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 --hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b --hash=sLine truncated
|
||||
rapidocr==3.8.1 --hash=sha256:650044b1fbce9e6bae5cae462dcf8be754cde11e2f23fc51f65dcc08deae2c46
|
||||
requests==2.34.2 --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0
|
||||
shapely==2.2.0 --hash=sha256:596b7994ceafa526b6e0522ca29fbc41d19f86459161d6efe1f251d0acd49f3f --hash=sha256:7c0b262116bb75b86751440b42e19673911bc0a8f0d5ce723ce294c3d6e4d5c0 --hash=sha256:7765e0e5d51d63eae0a911861cbda87165a01677bc9bce6ed20d06858ccde99f --hash=sha256:5d61088e2ef71dafad0dd4fae8a521cc1f20da4a89d3096bab5b3260b39b3052 --hash=sha256:0edec813c81effaf4e20c18b1aa86827925ce27c0315621f2a1a080e22e0de5e --hash=sha256:8d6ffe94710f37535a47161120cd5f7f0f0d9bb800c2fddebbd089cb7f1b3453 --hash=sha256:ce858295be3947143a3f44f145fa6dbacd5dcc5c4103801d42cd3be4a2034614 --hash=sha256:806d399418b23eee7241736d572ad1e0b784782f9241d7c8e2cfceb00787831d --hash=sha256:5b740c9a197e5feb30bdc6e64a5eb3ca2a7324d11498844136dfc317daac6a99 --hash=sha256:626fe4c0d32860a98e75ecffabf5a62254c6168eac96b633ad313cd62a38bb2b --hash=sha256:c36ccbff5c3374c349c370bfdac22c7676b268b4a707c98e9031f498965aa02d --hash=sha256:a9a380624cdd7a7e661bf15a4d1625082766f07ccd2540cb0a9e0df1ad4f6c11 --hash=sha256:650a5f4d8a8e3c96982079d8c99b6ddbe6602bbd1e34c75c2b95dbc0d28ac997 --hash=sha256:a851e077f0f02a3383923e02eca5447a29ddbf234e39593b91c8b7ac75218133 --hash=sha256:dc5faa593948aa64d9afae48331b80f43f7aacc68425d99064a4d6772f53f1ad --hash=sha256:da47a0cc9e630b4dff0db46e8972b29d2d27f337425ce9d4c77fd046ce48eabd --hash=sha256:90895df6542ae039fc6557dec6194e3509e883fbd6f5788e3c3e7a38fe46b257 --hash=sha256:7cf5b3a801b9b4febf774efde2e31280e647388deae8452693d8e6420b3a1ff2 --hash=sha256:c037369c35510f51100dd6d386ee3203bac32f164d53e27ca12c3cea5bb643b1 --hash=sha256:d75957716368f919c63016dae1977a0d007e15f06861cd178701edb91b08d2b0 --hash=sha256:4ed79beb8d4b6cc7c67780fd381feed25848a5f9b8a2385ac5711eccd115647a --hash=sha256:f340e7f99aaee3df5acd6b247cddf723051a7c93d1e1ef09025b80d84e4c0ded --hash=sha256:17434cb9819c9974c3331333a3b878fa5bf8f85dd69cc3fb7ff5d260f6fbc102 --hash=sha256:b2338ac40e6652c8bfb857936ea9be9a16f43a362c6f67eb3bad741b05fd5683 --hash=sha256:40871d7135cd723f965d200181aa28418e9ec029fd85bdd010488259d1c01906 --hash=sha256:1eaa2cb64cdedaf65d6bc86f2819c9cd7d6d68f969aa3ebfdc93743ab581f437 --hash=sha256:f79b3b34ad2d067207f21f821489c720b14ce40f3bfda931987a193165f80133 --hash=sha256:000c0ce2a3ba49427e6288b7add9de5d8525d4e65d6ebc8840103040d4d57b86 --hash=sha256:0a63e6b68ec785ef3aae3935c4aa9fb8edccced94e23c79d5d85276442c60859 --hash=sha256:770d4db5cf0bfeed931a1c4aaf4f4eadad0f43f5fc72c27c88fe1f07904ae767 --hash=sha256:74f4313af38d6e49ea83532d6cedfb4fe5e6c5485d7c40202bd61b19d6ff09bf --hash=sha256:9ee11aeba1759d15a525ded58e17916d3edfa60d52110fd8df6a7609a871f066 --hash=sha256:24b175c570efc91d1180ac6cd527dc80e863bb7de37f8b2771703d822c65e023 --hash=sha256:c2915ae1b858e73d5832be7fb5e89497cc5140fa505da40a45223029dc6deace --hash=sha256:74028f468e05e461b30a479b08c1fb5094fa45062abeeec8e7905a6711761436 --hash=sha256:6ec5178a39803fa8626322f69d298037f182461dd28e3ae96c2c7a4309a6bf30 --hash=sha256:593e51cd04fe1122f1ab3fae87b306c36b2be0184a5e0d9c26849c55ff4580dc --hash=sha256:3575a323b7665d7a2e391b16a626caa6b6f6348f399183aca3fc656febd7cf04 --hash=sha256:776cc8571d53e42be8fa6d42ad52a599b8e2186dd0c752922831508099af71e2 --hash=sha256:f8cd733a66a2a10f461a70dde9fad7b2b62c6a48c7a66cea57ee6f1cd9f2bd2f --hash=sha256:7f68c1fbacab81c0c066d1c3051eeb0f680b7a7a2c511e741f77741640187896 --hash=sha256:9147ebc3b116a0511dca043937f85caf1a41690815643d5b89c8bc472f51c850 --hash=sha256:4e5830637c080bdc646c5982ad6f7cc296b93038879649f7a6acd8e0f1c4db04 --hash=sha256:48dd1d961391f314ab7fa8812c86ca2a727bee2bdca1478730eacaea007da18e --hash=sha256:c4127c064bc71f8b7f9b3f341d6627ed39977fd0b61a17c68d09179f5e0089ae --hash=sha256:715561ceda03b09ca1c6baf9922179392d8c2bc53a1b877965225f0dfb487a58 --hash=sha256:556f20346a7d96fefbb71b74640d84ca14041703d60f0d2ff47b29d9b3e0093d --hash=sha256:ff9e87b534edf35af65758fafb31ad3b797354cba9323899e263f450c69a2ff2 --hash=sha256:fdb599ec540cea5b635ac47bf24fca4cdfd1c39730ffc0b6cf0d2666b0dd9a33 --hash=sha256:b8cb04906b74db26f848f76744fa995cd6abeae9145d27cc405277de1f949660 --hash=sha256:d9b11d712ac72f1d869f2b6964dea5bd9f20b89901adcd796d6712496144ab22 --hash=sha256:c88b21a0e9599ebb741e08f71a95c8f07a434af909efb088828a9874d234d06d --hash=sha256:cbe184e1946cfe115a9dfeadd2effd88ab4a237ab1a4335d106defa80fbc2d82 --hash=sha256:8bc985ad731da2f2cedde9c3cfb3c3d946fe6fc63d2ca557673dc33dd1e389b9 --hash=sha256:c3caa4c6308e7eaf18f4661134a1575eb290a56df78d0ae1b02f919a4cc7bd9d --hash=sha256:2fd87e55d7a7d310553b527378545cdc6ef8702473ed9294926b892c3cfb2ba0 --hash=sha256:7416db8ff3a1003687d4118e741343b3cf9ac2a4a925a59d44d98a865ac4e9e7 --hash=sha256:778421a19085bef1fb38bc0699db1ee9b08fdd0e30a8768788d601a4371f2de0 --hash=sha256:287ec7602f7a114b862ae0123880e57160cebe059843a4c7028aaee9e74287f6 --hash=sha256:e414c78bc81aadd76a429111a350f4ef3d05fc13019805617b524951258468e5 --hash=sha256:1af6935acde1db0b6a1bcbea30cbad5ae900723dfd398367ae1488470dc53667 --hash=sha256:96e5101ad2d73df869255bae4c55537f372d32066e2328c376e09841f0f66800 --hash=sha256:446b2d5a323bddd1c2a27f41325fdb3a3e8e33c1f8f0f840bdb63e8c1515b29e
|
||||
six==1.17.0 --hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274
|
||||
sympy==1.14.0 --hash=sha256:e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5
|
||||
tqdm==4.70.1 --hash=sha256:c293e525e6fef9c20e8728fd4612df02a0aa31bb5fe91ecd93e123b1b7bffa73
|
||||
urllib3==2.8.0 --hash=sha256:0cf3cae568d36aa9576b28dfb35f11328f1cb974ca7647d9475ebb86c75ac6e3
|
||||
yara-python==4.5.4 --hash=sha256:721d341bd2013fbada4df5aba0eb79a9e4e21c4b86441f7f111ab8c31671f125 --hash=sha256:c785fd16475f2a3191cd4fae0cd608b1ba6271f495ec86fa26a3c5ac53f5f2e1 --hash=sha256:5eefa3b157cd5f4454a317907bab334036f61385324b70cb61dbc656c44168d5 --hash=sha256:bbc0c5a2ee67e6043e4c2622093ebfc7d2c173dc643dd089742aedbdea48d6a4 --hash=sha256:1b091f1bd6c5d2a9b5c0c682ba67ba31b87bb71b27876430775998b71c8e3f97 --hash=sha256:91276c90bb5e148e10050015fec8a1d4009a95eee9eb832d154f80355d0b4080 --hash=sha256:7e20e1f69b6239fe4f4da97e9ff361d9be25d6f1d747589ea44b8a9ec412a12d --hash=sha256:a83773b561727fc360f7a6874f7fac1409bc9c391134dc3e070f1c2515c0db98 --hash=sha256:14f203bd9fb33ad591e046429560133127fa4a6201dac28c525fa7c6c7ca36a7 --hash=sha256:e632daf9f38f8d4b433f08f798b07a45756e6c396e9e0ec54aac10045f6d241d --hash=sha256:a3866830f7f2d071f94cbce7c41d91444ac29e2cbbe279914abf518d57a2d41f --hash=sha256:135c1097ea0445a323038acd509162675ce6d5e21f848aa856779632d48dec42 --hash=sha256:e3e2a5575d61adc2b4ff2007737590783a43d16386b061ac12e6e70a82e5d1de --hash=sha256:f79a27dbdafb79fc2dc03c7c3ba66751551e3e0b350ab69cc499870b78a6cb95 --hash=sha256:9d9acf6f8135bcee03f47b1096ad69f4a2788abe37dd070aab6e9dd816742ecc --hash=sha256:0e762e6c5b47ddf30b0128ba723da46fcc2aa7959a252748497492cb452d1c84 --hash=sha256:adcfac4b225e76ab6dcbeaf10101f0de2731fdbee51610dbc77b96e667e85a3a --hash=sha256:a82c87038f0da2d90051bfd6449cf9a4b977a15ee8372f3512ce0a413ef822fd --hash=sha256:1a1721b61ee4e625a143e8e5bf32fa6774797c06724c45067f3e8919a8e5f8f3 --hash=sha256:57d80c7591bbc6d9e73934e0fa4cbbb35e3e733b2706c5fd6756edf495f42678 --hash=sha256:3872b5f5575d6f5077f86e2b8bcdfe8688f859a50854334a4085399331167abc --hash=sha256:fd84af5b6da3429236b61f3ad8760fdc739d0e1d6a08b8f3d90cd375e71594df --hash=sha256:491c9de854e4a47dfbef7b3a38686c574459779915be19dcf4421b65847a57ce --hash=sha256:2a1bf52cb7b9178cc1ee2acd1697a0c8468af0c76aa1beffe22534bd4f62698b --hash=sha256:ade234700c492bce0efda96c1cdcd763425016e40df4a8d30c4c4e6897be5ace --hash=sha256:e1dedd149be61992781f085b592d169d1d813f9b5ffc7c8c2b74e429b443414c --hash=sha256:92b233aae320ee9e59728ee23f9faf4a423ae407d4768b47c8f0e472a34dbae2 --hash=sha256:1f238f10d26e4701559f73a69b22e1e192a6fa20abdd76f57a7054566780aa89 --hash=sha256:d29d0e137e0d77dd110186369276e88381f784bdc45b5932a2fb3463e2a1b1c7 --hash=sha256:e7d0039d734705b123494acad7a00b67df171dd5b1c16ff7b18ff07578efd4cd --hash=sha256:e5eae935b05a9f8dc71df55a79c38f52abd93f8840310fe4e0d75fbd78284f24 --hash=sha256:30fc7959394532c6e3f48faf59337f5da124f1630668258276b6cfa54e555a6e --hash=sha256:e6d8c2acaf33931338fdb78aba8a68462b0151d833b2eeda712db87713ac2abf --hash=sha256:a3c7bc8cd0db5fb87ab579c755de83723030522f3c0cd5b3374044055a8ce6c6 --hash=sha256:d12e57101683e9270738a1bccf676747f93e86b5bc529e7a7fb7adf94f20bd77 --hash=sha256:bf14a8af06b2b980a889bdc3f9e8ccd6e703d2b3fa1c98da5fd3a1c3b551eb47 --hash=sha256:fe8ad189843c729eae74be3b8447a4753fac2cebe705e5e2a7280badfcc7e3b4 --hash=sha256:94e290d5035be23059d0475bff3eac8228acd51145bf0cabe355b1ddabab742b --hash=sha256:4537f8499d166d22a54739f440fb306f65b0438be2c6c4ecb2352ecb5adb5f1c --hash=sha256:ab5133a16e466db6fe9c1a08d1b171013507896175010fb85fc1b92da32e558c --hash=sha256:93f5f5aba88e2ed2aaebfbb697433a0c8020c6a6c6a711e900a29e9b512d5c3a --hash=sha256:473c52b53c39d5daedc1912bd8a82a1c88702a3e393688879d77f9ff5f396543 --hash=sha256:9d9a58b7dc87411a2443d2e0382a111bd892aef9f6db2a1ebb4a9215eef0db71 --hash=sha256:0b9de86fbe8a646c0644df9e1396d6941dc6ed0f89be2807e6c52ab39161fd9f --hash=sha256:973f0bc24470ac86b6009baf2800ad3eadfa4ab653b6546ba5c65e9239850f47 --hash=sha256:cb0f0e7183165426b09e2b1235e70909e540ac18e2c6be96070dfe17d7db4d78 --hash=sha256:7707b144c8fcdb30c069ea57b94799cd7601f694ba01b696bbd1832721f37fd0 --hash=sha256:5f1288448991d63c1f6351c9f6d112916b0177ceefaa27d1419427a6ff09f829 --hash=sha256:7205c36a6798251925e4c1763eba0737fec7a95360df8aaa4e74e2f2f6b5cdcf --hash=sha256:9c77711a4e469ea97bdd16e85b5ab0720d6a481862a540ea7216897a10e93d19 --hash=sha256:9e6f0ca64cf9b0125be8fe8b821ba7e0f80427bcee136f83914dff0d81b4f27a --hash=sha256:8caad9de64dc4fc9614f331a04ea220d57aea3fbf997f3e23a298ee67cf4a69c --hash=sha256:9d5fcf99187cb6ec0a27c755aec22774a1ea578fdc2a5734f484c601de4ad6c0 --hash=sha256:bdbb62003cced7739d5c98c5af7a08c819baedf12e09276b2bbf0f3cb47828af --hash=sha256:5f901ec61db76a326f77882c8000139b29f218a7c6b8dd997195bab265602345 --hash=sha256:99f8d1145c11f61340fcd19fd6f3f3ef6306910829f4218daece45e831ceb54e --hash=sha256:da826f12c6fa459c6b2f9e7dff3a57416ac3a6536264f7a10d1ff839d4bc943f --hash=sha256:ebacf0b3325b3fb6f15bc3526f39f39c3576bafa4857493cc90afd2651ec8d36 --hash=sha256:e6eba7d4387f8123dd69852ba6776799150c4019fcb3e1212a3b053d42e151ab --hash=sha256:9addd1d6fe9d3b1efe40c7a37fa5d88fe6cd7e22c7e454617beb382c9c74b11b
|
||||
@@ -0,0 +1,6 @@
|
||||
pypdf>=6,<8
|
||||
Pillow>=12,<14
|
||||
defusedxml>=0.7,<1
|
||||
yara-python==4.5.4
|
||||
rapidocr==3.8.1
|
||||
onnxruntime==1.24.4
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Single-request parser entrypoint, launched only inside the AppContainer."""
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
|
||||
def attest_appcontainer():
|
||||
import ctypes
|
||||
from ctypes import wintypes as W
|
||||
if sys.platform != 'win32': raise RuntimeError('Windows AppContainer required')
|
||||
kernel=ctypes.WinDLL('kernel32',use_last_error=True)
|
||||
advapi=ctypes.WinDLL('advapi32',use_last_error=True)
|
||||
kernel.GetCurrentProcess.restype=W.HANDLE
|
||||
kernel.CloseHandle.argtypes=[W.HANDLE]
|
||||
advapi.OpenProcessToken.argtypes=[W.HANDLE,W.DWORD,ctypes.POINTER(W.HANDLE)]
|
||||
advapi.GetTokenInformation.argtypes=[W.HANDLE,ctypes.c_int,ctypes.c_void_p,W.DWORD,ctypes.POINTER(W.DWORD)]
|
||||
token=W.HANDLE();value=W.DWORD();length=W.DWORD()
|
||||
if not advapi.OpenProcessToken(kernel.GetCurrentProcess(),8,ctypes.byref(token)):
|
||||
raise RuntimeError('Process token unavailable')
|
||||
try:
|
||||
if not advapi.GetTokenInformation(token,29,ctypes.byref(value),4,ctypes.byref(length)) or value.value!=1:
|
||||
raise RuntimeError('Uncontained parser process rejected')
|
||||
# TokenCapabilities is TOKEN_GROUPS; no capability SID may be enabled.
|
||||
advapi.GetTokenInformation(token,30,None,0,ctypes.byref(length))
|
||||
if not 4<=length.value<=65536: raise RuntimeError('Capability token unavailable')
|
||||
capabilities=ctypes.create_string_buffer(length.value)
|
||||
if not advapi.GetTokenInformation(token,30,capabilities,length.value,ctypes.byref(length)) or ctypes.cast(capabilities,ctypes.POINTER(W.DWORD)).contents.value!=0:
|
||||
raise RuntimeError('Unexpected parser process capability')
|
||||
finally:kernel.CloseHandle(token)
|
||||
|
||||
|
||||
def main():
|
||||
attest_appcontainer()
|
||||
# Preserve only the broker response pipe; parser/native diagnostics cannot
|
||||
# contaminate the protocol or leak document content through stderr.
|
||||
response = os.fdopen(os.dup(sys.stdout.fileno()), 'w', encoding='utf-8', buffering=1)
|
||||
sink = os.open(os.devnull, os.O_WRONLY)
|
||||
os.dup2(sink,sys.stdout.fileno());os.dup2(sink,sys.stderr.fileno());os.close(sink)
|
||||
# No socket listener, deployment credential or filesystem document input.
|
||||
from engine import MAX_BYTES, inspect_content, scan_download, reference_document, embedding_features
|
||||
request = json.loads(sys.stdin.buffer.read(MAX_BYTES*2+1))
|
||||
data = base64.b64decode(request['data'], validate=True)
|
||||
if len(data)>MAX_BYTES: raise ValueError('Document exceeds inspection budget')
|
||||
route = request['route']
|
||||
if route == '/inspect': result = inspect_content(data, request['policy'])
|
||||
elif route == '/scan-download': result = scan_download(data, request['policy'])
|
||||
elif route == '/reference-document': result = reference_document(data)
|
||||
elif route == '/embed': result = embedding_features(data)
|
||||
else: raise ValueError('Unknown inspection operation')
|
||||
response.write(json.dumps(result, allow_nan=False))
|
||||
response.flush()
|
||||
response.close()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
try: main()
|
||||
except Exception: sys.exit(2)
|
||||
@@ -0,0 +1,177 @@
|
||||
"""Pinned offline EmbeddingGemma 2 GGUF worker. No downloads or HTTP inference."""
|
||||
import atexit
|
||||
import base64
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
import queue
|
||||
import re
|
||||
import subprocess
|
||||
import threading
|
||||
import time
|
||||
import wave
|
||||
|
||||
MODEL_ID="google/embeddinggemma-2"
|
||||
REVISION="bfcd298762cc34d0357ece5ebdd31791a3a374d8"
|
||||
DIMENSIONS=768
|
||||
PROFILE="document-v1"
|
||||
RUNTIME_REVISION="b11461"
|
||||
RUNTIME_BUILD="b11461-no-logits-v1"
|
||||
MAX_CHUNKS=64
|
||||
MAX_LINE=4*1024*1024
|
||||
MAX_REQUEST=16*1024*1024
|
||||
MODEL_FILES={
|
||||
"embeddinggemma-2-Q8_0.gguf":(309855456,"2188ac1deca4b77dffefd603c2776a9d76d9d74ec01841392982ebb840b09135"),
|
||||
"mmproj-embeddinggemma-2-Q8_0.gguf":(554821024,"c4a8a52691ecef40618438928bdf9e68379b854e24166f292592353db0aab64f")}
|
||||
|
||||
|
||||
def verify_file(directory,name,size,digest):
|
||||
if not isinstance(name,str) or not name or Path(name).is_absolute() or ".." in Path(name).parts: raise ValueError("Invalid local model path")
|
||||
file=(directory/name).resolve()
|
||||
if not file.is_relative_to(directory) or not file.is_file() or file.stat().st_size!=size: raise ValueError("Local semantic artifact unavailable")
|
||||
if not re.fullmatch(r"[0-9a-f]{64}",digest): raise ValueError("Invalid artifact digest")
|
||||
calculated=hashlib.sha256()
|
||||
with file.open("rb") as source:
|
||||
while part:=source.read(1024*1024): calculated.update(part)
|
||||
if calculated.hexdigest()!=digest: raise ValueError("Local semantic artifact verification failed")
|
||||
return file
|
||||
|
||||
|
||||
def validate_vectors(vectors,expected=None):
|
||||
if not isinstance(vectors,list) or not 1<=len(vectors)<=MAX_CHUNKS or expected is not None and len(vectors)!=expected: raise ValueError("Invalid semantic vector batch")
|
||||
for vector in vectors:
|
||||
if not isinstance(vector,list) or len(vector)!=DIMENSIONS or any(not isinstance(value,(int,float)) or isinstance(value,bool) or not math.isfinite(value) for value in vector): raise ValueError("Invalid semantic output")
|
||||
norm=math.sqrt(sum(value*value for value in vector))
|
||||
if not 0.99<=norm<=1.01: raise ValueError("Semantic output is not normalized")
|
||||
return vectors
|
||||
|
||||
|
||||
def validate_readiness(ready):
|
||||
if not isinstance(ready,dict) or ready.get("ready") is not True or ready.get("model")!=MODEL_ID or ready.get("dimensions")!=DIMENSIONS or ready.get("runtime")!=RUNTIME_BUILD or ready.get("device")!="cpu" or ready.get("vulkan_module_loaded") is not False:
|
||||
raise ValueError("Local semantic worker identity mismatch")
|
||||
|
||||
|
||||
class NativeSemanticModel:
|
||||
def __init__(self,directory,worker_path):
|
||||
self.directory=Path(directory).resolve(); self.worker_path=Path(worker_path).resolve()
|
||||
manifest_path=self.directory/"model-manifest.json"
|
||||
if not manifest_path.is_file() or manifest_path.stat().st_size>65536: raise ValueError("Local model manifest unavailable")
|
||||
manifest=json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||
if manifest.get("model_id")!=MODEL_ID or manifest.get("revision")!=REVISION or manifest.get("dimensions")!=DIMENSIONS or manifest.get("quantization")!="Q8_0" or manifest.get("runtime_tag")!=RUNTIME_REVISION: raise ValueError("Pinned Google model identity mismatch")
|
||||
self.model=verify_file(self.directory,*next((name,*value) for name,value in MODEL_FILES.items() if not name.startswith("mmproj")))
|
||||
self.projector=verify_file(self.directory,*next((name,*value) for name,value in MODEL_FILES.items() if name.startswith("mmproj")))
|
||||
runtime_path=self.worker_path.parent/"runtime-manifest.json"
|
||||
if not runtime_path.is_file() or runtime_path.stat().st_size>65536: raise ValueError("Verified native runtime manifest unavailable")
|
||||
runtime=json.loads(runtime_path.read_text(encoding="utf-8")); files=runtime.get("files")
|
||||
if not isinstance(files,list) or not 1<=len(files)<=64 or not any(item.get("path")==self.worker_path.name for item in files): raise ValueError("Invalid native runtime manifest")
|
||||
for item in files: verify_file(self.worker_path.parent,item["path"],item["bytes"],item["sha256"])
|
||||
# All DLLs load from the private, verified runtime directory.
|
||||
verified={item["path"].casefold() for item in files}
|
||||
if any(file.name.casefold() not in verified for file in self.worker_path.parent.glob("*.dll")): raise ValueError("Unverified runtime module")
|
||||
self.lock=threading.Lock();self.process=None;self.responses=None
|
||||
atexit.register(self.close)
|
||||
|
||||
def info(self):
|
||||
return {"model":MODEL_ID,"revision":REVISION,"dimensions":DIMENSIONS,"profile":PROFILE,"artifactFormat":"gguf","quantization":"Q8_0","runtimeRevision":RUNTIME_REVISION,"local":True,"backend":"native-gguf","device":"cpu","modalities":["text","image","audio"]}
|
||||
|
||||
def close(self):
|
||||
process=self.process;self.process=None
|
||||
if process is not None:
|
||||
try: process.kill();process.wait(timeout=3)
|
||||
except Exception: pass
|
||||
for stream in (process.stdin,process.stdout):
|
||||
try: stream.close()
|
||||
except Exception: pass
|
||||
|
||||
def _read(self,process,responses):
|
||||
try:
|
||||
while True:
|
||||
line=process.stdout.readline(MAX_LINE+1)
|
||||
if not line or len(line)>MAX_LINE or not line.endswith(b"\n"): raise ValueError()
|
||||
response=json.loads(line)
|
||||
responses.put(response,timeout=1)
|
||||
except Exception:
|
||||
try: responses.put({"protocolFailure":True},timeout=1)
|
||||
except Exception: pass
|
||||
|
||||
def _receive(self,timeout):
|
||||
try:
|
||||
response=self.responses.get(timeout=timeout)
|
||||
if not isinstance(response,dict) or response.get("protocolFailure"): raise ValueError()
|
||||
return response
|
||||
except Exception:
|
||||
self.close();raise ValueError("Local semantic worker failed or timed out") from None
|
||||
|
||||
def _start(self):
|
||||
if self.process is not None and self.process.poll() is None: return
|
||||
self.close();self.responses=queue.Queue(maxsize=2)
|
||||
environment={key:value for key,value in os.environ.items() if key.upper() in {"SYSTEMROOT","WINDIR","TEMP","TMP"}}
|
||||
environment.update({"SEARCHI_FORCE_CPU":"1","GGML_CPU_ONLY":"1","GGML_DISABLE_VULKAN":"1"})
|
||||
self.process=subprocess.Popen([str(self.worker_path),"--model",str(self.model),"--threads","2","--device","cpu"],
|
||||
cwd=str(self.worker_path.parent),stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.DEVNULL,
|
||||
env=environment,creationflags=getattr(subprocess,"CREATE_NO_WINDOW",0))
|
||||
threading.Thread(target=self._read,args=(self.process,self.responses),daemon=True).start()
|
||||
ready=self._receive(20)
|
||||
try: validate_readiness(ready)
|
||||
except Exception: self.close();raise
|
||||
|
||||
def _request(self,request,timeout=30):
|
||||
payload=json.dumps(request,ensure_ascii=False,allow_nan=False).encode("utf-8")+b"\n"
|
||||
if len(payload)>=MAX_REQUEST: raise ValueError("Semantic request budget exceeded")
|
||||
with self.lock:
|
||||
self._start()
|
||||
process=self.process
|
||||
# A blocked pipe write is also bounded and tears down the process.
|
||||
timer=threading.Timer(timeout,lambda:process.kill() if process.poll() is None else None);timer.daemon=True;timer.start()
|
||||
try:
|
||||
process.stdin.write(payload);process.stdin.flush();response=self._receive(timeout)
|
||||
if response.get("ok") is not True or response.get("device")!="cpu": raise ValueError("Local semantic inference failed")
|
||||
return response
|
||||
except Exception:
|
||||
self.close();raise ValueError("Local semantic inference failed") from None
|
||||
finally: timer.cancel();payload=None
|
||||
|
||||
def text_vectors(self,text):
|
||||
if not isinstance(text,str) or not text.strip() or len(text.encode("utf-8"))>1024*1024: raise ValueError("Semantic text budget exceeded")
|
||||
encoded=text.encode("utf-8");windows=self._request({"op":"windows","text":text,"title":"none"}).get("windows")
|
||||
if not isinstance(windows,list) or not 1<=len(windows)<=MAX_CHUNKS: raise ValueError("Semantic window budget exceeded")
|
||||
previous_end=0;parts=[]
|
||||
for item in windows:
|
||||
start=item.get("start_byte");end=item.get("end_byte");part=item.get("text")
|
||||
if not isinstance(start,int) or not isinstance(end,int) or not isinstance(part,str) or start<0 or start>previous_end or end<=previous_end or end>len(encoded) or part.encode("utf-8")!=encoded[start:end]: raise ValueError("Semantic source coverage invalid")
|
||||
previous_end=end;parts.append(part)
|
||||
if previous_end!=len(encoded): raise ValueError("Semantic source tail missing")
|
||||
vectors=[];deadline=time.monotonic()+60
|
||||
for index in range(0,len(parts),8):
|
||||
remaining=deadline-time.monotonic()
|
||||
if remaining<=0: raise ValueError("Semantic inference budget exceeded")
|
||||
batch=parts[index:index+8];response=self._request({"op":"documents","texts":batch,"title":"none"},min(30,remaining))
|
||||
vectors.extend(validate_vectors(response.get("vectors"),len(batch)))
|
||||
return vectors
|
||||
|
||||
def image_vectors(self,data):
|
||||
from PIL import Image,ImageSequence,ImageOps
|
||||
image=Image.open(io.BytesIO(data));frames=getattr(image,"n_frames",1)
|
||||
if frames>32 or image.width*image.height*frames>50_000_000: raise ValueError("Semantic image budget exceeded")
|
||||
vectors=[]
|
||||
for frame in ImageSequence.Iterator(image):
|
||||
frame=ImageOps.exif_transpose(frame).convert("RGBA")
|
||||
background=Image.new("RGBA",frame.size,"white");background.alpha_composite(frame);frame=background.convert("RGB")
|
||||
scale=min(1,672/max(frame.size));width=max(48,round(frame.width*scale/48)*48);height=max(48,round(frame.height*scale/48)*48)
|
||||
frame=frame.resize((width,height),Image.Resampling.LANCZOS)
|
||||
request={"op":"image","projector":str(self.projector),"width":width,"height":height,"data":base64.b64encode(frame.tobytes()).decode("ascii")}
|
||||
vectors.extend(validate_vectors(self._request(request).get("vectors"),1))
|
||||
return vectors
|
||||
|
||||
def audio_vectors(self,data):
|
||||
import numpy as np
|
||||
with wave.open(io.BytesIO(data),"rb") as source:
|
||||
if source.getnchannels()!=1 or source.getframerate()!=16000 or source.getsampwidth()!=2 or not 0<source.getnframes()<=480000: raise ValueError("Only bounded mono16kHz PCM16 WAV is supported")
|
||||
payload=source.readframes(source.getnframes())
|
||||
if len(payload)!=source.getnframes()*2: raise ValueError("Truncated audio data")
|
||||
pcm=np.frombuffer(payload,dtype="<i2").astype("<f4")/32768
|
||||
request={"op":"audio","projector":str(self.projector),"data":base64.b64encode(pcm.tobytes()).decode("ascii")}
|
||||
return validate_vectors(self._request(request).get("vectors"),1)
|
||||
@@ -0,0 +1,109 @@
|
||||
import base64
|
||||
import hmac
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
# Embedded Python uses an isolated ._pth and does not add the script directory.
|
||||
# Resolve sibling modules from this service's source, not an old runtime copy.
|
||||
sys.path.insert(0,os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.dont_write_bytecode=True
|
||||
from windows_containment import contain_inspector
|
||||
# Apply limits before loading parsers, OCR libraries or native model workers.
|
||||
# An unavailable Windows boundary fails startup rather than silently weakening it.
|
||||
CONTAINMENT=contain_inspector() if __name__=="__main__" else {"active":False,"kind":"not-service-entrypoint"}
|
||||
# Broker imports no parser, OCR or model library. All untrusted document work
|
||||
# runs under a capability-free Windows AppContainer, with no network permission.
|
||||
MAX_BYTES = 20 * 1024 * 1024
|
||||
SANDBOX = None
|
||||
SANDBOX_ENV = {}
|
||||
if __name__ == "__main__":
|
||||
if sys.platform != 'win32': raise SystemExit('Production parser isolation requires Windows')
|
||||
from pathlib import Path
|
||||
from windows_appcontainer import AppContainer
|
||||
source = Path(__file__).resolve().parent
|
||||
python_root = Path(sys.executable).resolve().parent
|
||||
# Development and packaged roots are derived from our own installation;
|
||||
# externally configured model paths must already be copied into that tree.
|
||||
installation = source.parent.parent if source.name == 'inspector' and source.parent.name == 'apps' else source.parent
|
||||
semantic_root = installation / ('.runtime/windows/semantic' if source.parent.name == 'apps' else 'semantic')
|
||||
expected_python = installation / ('.runtime/windows/python' if source.parent.name == 'apps' else 'python')
|
||||
if python_root != expected_python.resolve():
|
||||
raise SystemExit('Parser runtime must reside in the private installed runtime')
|
||||
roots = [source, python_root]
|
||||
if semantic_root.is_dir(): roots.append(semantic_root)
|
||||
for key in ['EMBEDDING_MODEL_DIR', 'EMBEDDING_WORKER_PATH']:
|
||||
value = os.environ.get(key)
|
||||
if value:
|
||||
resolved = Path(value).resolve()
|
||||
if not resolved.is_relative_to(semantic_root.resolve()):
|
||||
raise SystemExit('Model artifacts must reside in the private installed runtime')
|
||||
SANDBOX_ENV[key] = str(resolved)
|
||||
SANDBOX_ENV.update({'SEARCHI_FORCE_CPU':'1', 'GGML_CPU_ONLY':'1', 'GGML_DISABLE_VULKAN':'1'})
|
||||
SANDBOX = AppContainer(roots)
|
||||
CONTAINMENT['parserIsolation'] = {'active':True, 'kind':'windows-appcontainer-v1', 'networkCapabilities':0, 'privatePipes':True, 'readOnlyRuntime':True}
|
||||
|
||||
|
||||
def dispatch(route, request):
|
||||
if SANDBOX is None: raise RuntimeError('Isolated parser unavailable')
|
||||
payload = json.dumps({'route':route, 'data':request['data'], 'policy':request.get('policy',{})}, allow_nan=False).encode('utf-8')
|
||||
code, output = SANDBOX.run(sys.executable, [str(Path(__file__).with_name('sandbox_worker.py'))],
|
||||
payload, environment=SANDBOX_ENV, timeout=29, max_output=8*1024*1024)
|
||||
if code: raise RuntimeError('Isolated parser failed')
|
||||
result = json.loads(output)
|
||||
if not isinstance(result, dict): raise RuntimeError('Isolated parser response invalid')
|
||||
if route in ['/inspect','/scan-download']:
|
||||
if result.get('decision') not in ['allow','block'] or not isinstance(result.get('coverage',{}).get('complete'),bool):
|
||||
raise RuntimeError('Isolated parser decision invalid')
|
||||
if result['decision']=='allow' and not result['coverage']['complete']:
|
||||
raise RuntimeError('Incomplete parser coverage cannot release content')
|
||||
return result
|
||||
|
||||
TOKEN=os.environ.get("INSPECTOR_TOKEN", "")
|
||||
if len(TOKEN)<32: raise SystemExit("INSPECTOR_TOKEN must contain at least 32 characters")
|
||||
LOCK=threading.Lock()
|
||||
|
||||
def watch_parent():
|
||||
parent=os.environ.get('INSPECTOR_PARENT_PID')
|
||||
if not parent or sys.platform!='win32': return
|
||||
import ctypes
|
||||
kernel=ctypes.WinDLL('kernel32',use_last_error=True)
|
||||
kernel.OpenProcess.argtypes=[ctypes.c_uint32,ctypes.c_int,ctypes.c_uint32]
|
||||
kernel.OpenProcess.restype=ctypes.c_void_p
|
||||
kernel.WaitForSingleObject.argtypes=[ctypes.c_void_p,ctypes.c_uint32]
|
||||
handle=kernel.OpenProcess(0x00100000,False,int(parent))
|
||||
if not handle: os._exit(0)
|
||||
while kernel.WaitForSingleObject(handle,1000)==258: pass
|
||||
os._exit(0)
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self, *_args): pass
|
||||
def do_POST(self):
|
||||
if not hmac.compare_digest(self.headers.get("Authorization", ""), "Bearer "+TOKEN): return self.reply(401,{"error":"Inspector authentication required"})
|
||||
request=None;data=None
|
||||
try:
|
||||
length=int(self.headers.get("Content-Length", "0"))
|
||||
if length<=0 or length>MAX_BYTES*2: return self.reply(413,{"error":"Inspection request too large"})
|
||||
self.connection.settimeout(30)
|
||||
request=json.loads(self.rfile.read(length))
|
||||
data=base64.b64decode(request["data"],validate=True)
|
||||
if len(data)>MAX_BYTES: return self.reply(413,{"error":"Content too large"})
|
||||
with LOCK:
|
||||
if self.path not in ['/inspect','/scan-download','/reference-document','/embed']:
|
||||
return self.reply(404,{"error":"Not found"})
|
||||
result=dispatch(self.path,request)
|
||||
return self.reply(200,result)
|
||||
except Exception: return self.reply(422,{"error":"Content inspection failed"})
|
||||
finally:
|
||||
if isinstance(request,dict): request.clear()
|
||||
data=None
|
||||
def reply(self,status,data):
|
||||
body=json.dumps(data,allow_nan=False).encode();self.send_response(status);self.send_header("Content-Type","application/json");self.send_header("Content-Length",str(len(body)));self.send_header("Cache-Control","no-store");self.end_headers();self.wfile.write(body)
|
||||
|
||||
if __name__=="__main__":
|
||||
threading.Thread(target=watch_parent,daemon=True).start()
|
||||
server=HTTPServer(("127.0.0.1",int(os.environ.get("INSPECTOR_PORT","4311"))),Handler)
|
||||
server.timeout=30
|
||||
print(json.dumps({"port":server.server_address[1],"containment":CONTAINMENT}),flush=True)
|
||||
server.serve_forever()
|
||||
@@ -0,0 +1,41 @@
|
||||
import io
|
||||
import os
|
||||
import unittest
|
||||
import zipfile
|
||||
from engine import cosine, inspect_content, content_kind
|
||||
|
||||
def policy(): return {"dlp":{"maxBytes":1048576,"rules":[{"id":"secret","kind":"literal","value":"Secret Roadmap"}],"semantic":{"enabled":False}}}
|
||||
|
||||
class InspectionTests(unittest.TestCase):
|
||||
def test_text_and_binary(self):
|
||||
self.assertEqual(inspect_content(b"Public information",policy())["decision"],"allow")
|
||||
self.assertEqual(inspect_content(b"Secret Roadmap",policy())["ruleId"],"secret")
|
||||
self.assertEqual(inspect_content(b"\xff\x00",policy())["decision"],"block")
|
||||
def test_office_extraction(self):
|
||||
output=io.BytesIO()
|
||||
with zipfile.ZipFile(output,"w") as archive:
|
||||
archive.writestr("word/document.xml",'<document xmlns="urn:test"><t>Secret Roadmap</t></document>')
|
||||
result=inspect_content(output.getvalue(),policy())
|
||||
self.assertEqual(result["contentType"],"office");self.assertEqual(result["ruleId"],"secret")
|
||||
def test_pdf_extraction(self):
|
||||
from pypdf import PdfWriter
|
||||
from pypdf.generic import DecodedStreamObject,NameObject,DictionaryObject
|
||||
writer=PdfWriter();page=writer.add_blank_page(300,300)
|
||||
font=DictionaryObject({NameObject('/Type'):NameObject('/Font'),NameObject('/Subtype'):NameObject('/Type1'),NameObject('/BaseFont'):NameObject('/Helvetica')})
|
||||
page[NameObject('/Resources')]=DictionaryObject({NameObject('/Font'):DictionaryObject({NameObject('/F1'):writer._add_object(font)})})
|
||||
content=DecodedStreamObject();content.set_data(b'BT /F1 12 Tf 10 100 Td (Secret Roadmap) Tj ET');page[NameObject('/Contents')]=writer._add_object(content)
|
||||
output=io.BytesIO();writer.write(output)
|
||||
result=inspect_content(output.getvalue(),policy());self.assertEqual(result['contentType'],'pdf');self.assertEqual(result['ruleId'],'secret')
|
||||
def test_semantic_threshold_and_missing_model(self):
|
||||
p=policy();p['dlp']['semantic']={'enabled':True,'model':'google/embeddinggemma-2','revision':'test-revision','categories':[{'id':'semantic-secret','threshold':.8,'prototype':[1.,0.]}]}
|
||||
os.environ['EMBEDDING_MODEL_REVISION']='test-revision'
|
||||
result=inspect_content(b'Confidential paraphrase',p,encoder=lambda *_:[1.,0.]);self.assertEqual(result['ruleId'],'semantic-secret')
|
||||
result=inspect_content(b'Confidential paraphrase',p,encoder=lambda *_: (_ for _ in ()).throw(ValueError()));self.assertEqual(result['decision'],'block')
|
||||
def test_cosine_validates(self):
|
||||
self.assertAlmostEqual(cosine([1,0],[1,0]),1)
|
||||
with self.assertRaises(ValueError):cosine([0,0],[0,0])
|
||||
def test_video_has_no_false_safe_coverage(self):
|
||||
video=b'\x00\x00\x00\x18ftypisom'+b'\x00'*20
|
||||
self.assertEqual(content_kind(video),'video');result=inspect_content(video,policy());self.assertEqual(result['decision'],'block');self.assertFalse(result['coverage']['complete'])
|
||||
|
||||
if __name__=='__main__':unittest.main()
|
||||
@@ -0,0 +1,68 @@
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
import io
|
||||
from engine import scan_download, reference_document, inspect_content
|
||||
|
||||
def policy(source='rule malicious { strings: $a = "MALWARE_TEST_PAYLOAD" condition: $a }'):
|
||||
return {'security':{'yara':{'enabled':True,'rules':[{'id':'central-malware','source':source}], 'maxScanBytes':1048576}},
|
||||
'dlp':{'maxBytes':1048576,'rules':[],'semantic':{'enabled':False}}}
|
||||
|
||||
class SecurityTests(unittest.TestCase):
|
||||
def test_real_yara_match(self):
|
||||
import yara
|
||||
result=scan_download(b'MZ\x00MALWARE_TEST_PAYLOAD',policy())
|
||||
self.assertEqual(result['decision'],'block')
|
||||
self.assertEqual(result['ruleId'],'central-malware')
|
||||
self.assertTrue(result['coverage']['complete'])
|
||||
def test_binary_malware_coverage_separate_from_dlp(self):
|
||||
data=b'MZ\x00\xffbenign'
|
||||
self.assertEqual(scan_download(data,policy())['decision'],'allow')
|
||||
self.assertEqual(inspect_content(data,policy())['decision'],'block')
|
||||
def test_invalid_or_empty_rules_fail_closed(self):
|
||||
self.assertEqual(scan_download(b'safe',policy('invalid'))['decision'],'block')
|
||||
p=policy();p['security']['yara']['rules']=[]
|
||||
self.assertEqual(scan_download(b'safe',p)['decision'],'block')
|
||||
self.assertEqual(scan_download(b'safe',{})['decision'],'block')
|
||||
def test_include_and_console_import_rejected(self):
|
||||
self.assertEqual(scan_download(b'safe',policy('include "C:/secret"'))['decision'],'block')
|
||||
self.assertEqual(scan_download(b'safe',policy('import "console" rule leak { condition: console.log("secret") }'))['decision'],'block')
|
||||
def test_size_failure(self):
|
||||
p=policy();p['security']['yara']['maxScanBytes']=2
|
||||
self.assertEqual(scan_download(b'large',p)['ruleId'],'yara-size-limit')
|
||||
def test_reference_registration_contains_no_plaintext(self):
|
||||
text=b'Internal confidential roadmap for new product launch during next quarter'
|
||||
result=reference_document(text)
|
||||
self.assertTrue(result['registrationReady']);self.assertTrue(result['fragmentHashes'])
|
||||
self.assertNotIn('confidential',str(result))
|
||||
self.assertFalse(reference_document(b'\xff\x00')['registrationReady'])
|
||||
def test_training_truncation_never_claims_complete(self):
|
||||
result=reference_document(' '.join('distinctword'+str(index) for index in range(1200)).encode())
|
||||
self.assertEqual(len(result['fragmentHashes']),1000)
|
||||
self.assertGreater(result['coverage']['fragmentHashesTotal'],1000)
|
||||
self.assertTrue(result['coverage']['fragmentHashesTruncated'])
|
||||
self.assertTrue(result['coverage']['extractionComplete'])
|
||||
self.assertFalse(result['coverage']['complete'])
|
||||
self.assertFalse(result['registrationReady'])
|
||||
def test_text_training_does_not_create_temporary_content_files(self):
|
||||
with patch('engine.tempfile.TemporaryDirectory',side_effect=AssertionError('Raw content persistence forbidden')):
|
||||
result=reference_document(b'Confidential strategic roadmap contains enough words for fingerprint training')
|
||||
self.assertTrue(result['registrationReady'])
|
||||
def test_trained_fragments_block_reformatted_excerpt(self):
|
||||
text=b'Internal confidential roadmap for new product launch during next quarter'
|
||||
reference=reference_document(text)
|
||||
p=policy();p['dlp']['rules']=[{'id':'trained','kind':'fragment-sha256','value':value} for value in reference['fragmentHashes']]
|
||||
result=inspect_content(b'Introduction. INTERNAL confidential roadmap for new product launch during next quarter. Ending.',p)
|
||||
self.assertEqual(result['ruleId'],'trained')
|
||||
def test_real_bundled_ocr_offline_detects_image_secret(self):
|
||||
from PIL import Image, ImageDraw, ImageFont
|
||||
image=Image.new('RGB',(1000,180),'white')
|
||||
ImageDraw.Draw(image).text((30,45),'Secret Roadmap',font=ImageFont.truetype('C:/Windows/Fonts/arial.ttf',60),fill='black')
|
||||
output=io.BytesIO();image.save(output,format='PNG')
|
||||
p=policy();p['dlp']['rules']=[{'id':'image-secret','kind':'literal','value':'Secret Roadmap'}]
|
||||
# Inference and lazy initialization must not enter a model-download path.
|
||||
with patch('rapidocr.utils.download_file.DownloadFile.run',side_effect=AssertionError('Network model download forbidden')):
|
||||
result=inspect_content(output.getvalue(),p)
|
||||
self.assertEqual(result['ruleId'],'image-secret')
|
||||
self.assertTrue(result['coverage']['complete'])
|
||||
|
||||
if __name__=='__main__':unittest.main()
|
||||
@@ -0,0 +1,87 @@
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import queue
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
from semantic import NativeSemanticModel,validate_vectors,validate_readiness,verify_file,DIMENSIONS,PROFILE,RUNTIME_BUILD
|
||||
from engine import aggregate_vectors,embedding_features,inspect_content
|
||||
|
||||
def vector(index=0):
|
||||
result=[0.0]*DIMENSIONS;result[index]=1.0;return result
|
||||
|
||||
class SemanticProtocolTests(unittest.TestCase):
|
||||
def model(self,windows):
|
||||
model=NativeSemanticModel.__new__(NativeSemanticModel)
|
||||
def request(payload,timeout=30):
|
||||
if payload['op']=='windows':return {'windows':windows}
|
||||
return {'vectors':[vector() for _ in payload['texts']]}
|
||||
model._request=request
|
||||
return model
|
||||
def test_unicode_windows_cover_exact_source_and_tail(self):
|
||||
text='Confidential café 中央 secret';encoded=text.encode()
|
||||
windows=[{'text':text,'start_byte':0,'end_byte':len(encoded)}]
|
||||
self.assertEqual(self.model(windows).text_vectors(text),[vector()])
|
||||
windows[0]['end_byte']-=1
|
||||
with self.assertRaises(ValueError):self.model(windows).text_vectors(text)
|
||||
def test_missing_tail_gaps_and_substitution_are_refused(self):
|
||||
for windows in [[{'text':'hello','start_byte':0,'end_byte':5}],
|
||||
[{'text':'hello','start_byte':0,'end_byte':5},{'text':'world','start_byte':6,'end_byte':11}],
|
||||
[{'text':'altered text','start_byte':0,'end_byte':11}]]:
|
||||
with self.assertRaises(ValueError):self.model(windows).text_vectors('hello world')
|
||||
def test_full_window_batch_is_not_silently_truncated(self):
|
||||
text='a'*65
|
||||
windows=[{'text':'a','start_byte':index,'end_byte':index+1} for index in range(65)]
|
||||
with self.assertRaises(ValueError):self.model(windows).text_vectors(text)
|
||||
def test_output_dimensions_finite_norm_and_aggregation(self):
|
||||
self.assertEqual(validate_vectors([vector()]),[vector()])
|
||||
for invalid in [[0.0]*768,[1.0]*767,[float('nan')]*768,[float('inf')]*768]:
|
||||
with self.assertRaises(ValueError):validate_vectors([invalid])
|
||||
aggregate=aggregate_vectors([vector(0),vector(1)])
|
||||
self.assertAlmostEqual(sum(value*value for value in aggregate),1)
|
||||
def test_only_exact_hardened_cpu_worker_build_is_accepted(self):
|
||||
ready={'ready':True,'model':'google/embeddinggemma-2','dimensions':768,'runtime':RUNTIME_BUILD,'device':'cpu','vulkan_module_loaded':False}
|
||||
validate_readiness(ready)
|
||||
for changed in [{'runtime':'b11461'},{'runtime':'b11461-no-logits-v2'},{'device':'gpu'},{'vulkan_module_loaded':True},{'dimensions':128}]:
|
||||
with self.assertRaises(ValueError):validate_readiness({**ready,**changed})
|
||||
def test_artifact_hash_size_and_escape_checks(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root=Path(directory).resolve();file=root/'artifact';file.write_bytes(b'public model fixture')
|
||||
digest=hashlib.sha256(file.read_bytes()).hexdigest()
|
||||
self.assertEqual(verify_file(root,'artifact',file.stat().st_size,digest),file)
|
||||
with self.assertRaises(ValueError):verify_file(root,'artifact',file.stat().st_size,'0'*64)
|
||||
with self.assertRaises(ValueError):verify_file(root,'artifact',1,digest)
|
||||
with self.assertRaises(ValueError):verify_file(root,'../escape',1,digest)
|
||||
def test_training_features_return_only_vectors_and_metadata(self):
|
||||
class FixtureModel:
|
||||
def text_vectors(self,text):return [vector(),vector(1)]
|
||||
def info(self):return {'model':'google/embeddinggemma-2','revision':'a'*40,'dimensions':768,'profile':PROFILE,'local':True}
|
||||
with patch('engine.get_model',return_value=FixtureModel()):result=embedding_features(b'Internal secret not to retain in reference data')
|
||||
self.assertTrue(result['coverage']['complete']);self.assertEqual(result['coverage']['textChunks'],2)
|
||||
self.assertEqual(len(result['vectors']),2);self.assertNotIn('Internal secret',json.dumps(result))
|
||||
def test_category_matching_compares_all_reference_prototypes(self):
|
||||
policy={'dlp':{'maxBytes':1024,'rules':[],'semantic':{'enabled':True,'model':'google/embeddinggemma-2','revision':'a'*40,'categories':[{'id':'reference','prototype':vector(0),'prototypes':[vector(0),vector(1)],'threshold':.9}]}}}
|
||||
with patch.dict('os.environ',{'EMBEDDING_MODEL_REVISION':'a'*40}):
|
||||
result=inspect_content(b'Paraphrased protected concept',policy,encoder=lambda *_:vector(1))
|
||||
self.assertEqual(result['ruleId'],'reference')
|
||||
def test_worker_timeout_kills_and_sanitizes_errors(self):
|
||||
model=NativeSemanticModel.__new__(NativeSemanticModel);model.responses=queue.Queue()
|
||||
with patch.object(model,'close') as close:
|
||||
with self.assertRaisesRegex(ValueError,'timed out'):model._receive(.001)
|
||||
close.assert_called_once()
|
||||
def test_worker_private_exception_text_is_never_returned(self):
|
||||
import threading
|
||||
class Process:
|
||||
stdin=io.BytesIO()
|
||||
def poll(self):return None
|
||||
def kill(self):pass
|
||||
model=NativeSemanticModel.__new__(NativeSemanticModel);model.process=Process();model.lock=threading.Lock()
|
||||
model._start=lambda:None
|
||||
model._receive=lambda timeout:{'ok':False,'device':'cpu','error':'PRIVATE DOC CONTENT'}
|
||||
with patch.object(model,'close') as close:
|
||||
with self.assertRaises(ValueError) as error:model._request({'op':'documents','texts':['public fixture']})
|
||||
self.assertNotIn('PRIVATE',str(error.exception));close.assert_called_once()
|
||||
|
||||
if __name__=='__main__':unittest.main()
|
||||
@@ -0,0 +1,47 @@
|
||||
"""Opt-in operational fixtures using staged real Google weights, CPU only."""
|
||||
import io
|
||||
import math
|
||||
import os
|
||||
import unittest
|
||||
import wave
|
||||
from engine import get_model,embedding_features,inspect_content,cosine
|
||||
from semantic import validate_vectors,DIMENSIONS
|
||||
|
||||
@unittest.skipUnless(os.environ.get('SAFETYNET_TEST_NATIVE_SEMANTIC')=='1','Opt in to verified native CPU fixtures')
|
||||
class NativeCPUFixtures(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):cls.model=get_model()
|
||||
@classmethod
|
||||
def tearDownClass(cls):cls.model.close()
|
||||
def test_real_paraphrase_blocks_and_unrelated_text_allows(self):
|
||||
reference=embedding_features(b'Our confidential launch plan schedules the new product announcement for next October.')
|
||||
protected=b'The secret product rollout will be announced in October next year.'
|
||||
public=b'The public cafeteria menu includes tomato soup and bread.'
|
||||
related=embedding_features(protected);unrelated=embedding_features(public)
|
||||
self.assertGreater(cosine(reference['vector'],related['vector']),.8)
|
||||
self.assertLess(cosine(reference['vector'],unrelated['vector']),.8)
|
||||
policy={'dlp':{'maxBytes':1048576,'rules':[],'semantic':{**self.model.info(),'enabled':True,'categories':[{'id':'native-reference','prototype':reference['vector'],'prototypes':reference['vectors'],'threshold':.8}]}}}
|
||||
self.assertEqual(inspect_content(protected,policy)['ruleId'],'native-reference')
|
||||
self.assertEqual(inspect_content(public,policy)['decision'],'allow')
|
||||
def test_native_token_windows_cover_long_document_tail(self):
|
||||
text='Public synthetic office lunch menu and team meeting schedule. '*600+'Final unique coverage marker.'
|
||||
result=embedding_features(text.encode())
|
||||
self.assertGreater(result['coverage']['textChunks'],1)
|
||||
self.assertTrue(result['coverage']['complete']);validate_vectors(result['vectors'])
|
||||
def test_native_image_encoder_returns_real_normalized_vector(self):
|
||||
from PIL import Image,ImageDraw
|
||||
image=Image.new('RGB',(192,192),'white');draw=ImageDraw.Draw(image)
|
||||
draw.rectangle((20,20,90,170),fill='blue');draw.ellipse((110,50,175,115),fill='red')
|
||||
output=io.BytesIO();image.save(output,format='PNG')
|
||||
vectors=self.model.image_vectors(output.getvalue())
|
||||
self.assertEqual(len(vectors),1);validate_vectors(vectors)
|
||||
def test_native_audio_encoder_returns_real_normalized_vector(self):
|
||||
import numpy as np
|
||||
samples=(np.sin(np.arange(16000)*2*math.pi*440/16000)*12000).astype('<i2')
|
||||
output=io.BytesIO()
|
||||
with wave.open(output,'wb') as stream:
|
||||
stream.setnchannels(1);stream.setsampwidth(2);stream.setframerate(16000);stream.writeframes(samples.tobytes())
|
||||
vectors=self.model.audio_vectors(output.getvalue())
|
||||
self.assertEqual(len(vectors),1);validate_vectors(vectors)
|
||||
|
||||
if __name__=='__main__':unittest.main()
|
||||
@@ -0,0 +1,48 @@
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import queue
|
||||
import secrets
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import unittest
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
|
||||
@unittest.skipUnless(sys.platform=='win32','Production inspector isolation requires Windows')
|
||||
class InspectorHTTPTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.token=secrets.token_urlsafe(32)
|
||||
env={key:value for key,value in os.environ.items() if key.upper() in {'SYSTEMROOT','WINDIR','USERPROFILE','LOCALAPPDATA','APPDATA','TEMP','TMP'}}
|
||||
env.update({'INSPECTOR_TOKEN':cls.token,'INSPECTOR_PORT':'0'})
|
||||
cls.server=subprocess.Popen([sys.executable,str(Path(__file__).with_name('server.py'))],stdout=subprocess.PIPE,stderr=subprocess.PIPE,env=env,creationflags=subprocess.CREATE_NO_WINDOW)
|
||||
lines=queue.Queue();threading.Thread(target=lambda:lines.put(cls.server.stdout.readline()),daemon=True).start()
|
||||
try:
|
||||
cls.ready=json.loads(lines.get(timeout=40));cls.port=cls.ready['port']
|
||||
if cls.ready['containment']['parserIsolation']['kind']!='windows-appcontainer-v1':raise RuntimeError('Production sandbox attestation missing')
|
||||
except Exception:
|
||||
cls.server.kill();cls.server.wait(timeout=5);cls.server.stdout.close();cls.server.stderr.close();raise
|
||||
@classmethod
|
||||
def tearDownClass(cls):
|
||||
cls.server.kill();cls.server.wait(timeout=5);cls.server.stdout.close();cls.server.stderr.close()
|
||||
def request(self,data,credential):
|
||||
policy={'dlp':{'maxBytes':1048576,'rules':[{'id':'secret','kind':'literal','value':'Secret Roadmap'}],'semantic':{'enabled':False}}}
|
||||
body=json.dumps({'data':base64.b64encode(data).decode(),'policy':policy}).encode()
|
||||
request=urllib.request.Request(f'http://127.0.0.1:{self.port}/inspect',data=body,headers={'Content-Type':'application/json','Authorization':'Bearer '+credential})
|
||||
return urllib.request.urlopen(request,timeout=30)
|
||||
def test_authenticated_content_inspection(self):
|
||||
with self.request(b'Secret Roadmap',self.token) as response:
|
||||
result=json.load(response);self.assertEqual(result['decision'],'block');self.assertEqual(result['ruleId'],'secret')
|
||||
with self.request(b'Public information',self.token) as response:self.assertEqual(json.load(response)['decision'],'allow')
|
||||
def test_rejects_unauthenticated_content(self):
|
||||
with self.assertRaises(urllib.error.HTTPError) as error:self.request(b'Public information','wrong')
|
||||
self.assertEqual(error.exception.code,401);error.exception.close()
|
||||
def test_actual_parser_isolation_attestation(self):
|
||||
isolation=self.ready['containment']['parserIsolation'];self.assertTrue(isolation['active']);self.assertEqual(isolation['networkCapabilities'],0);self.assertTrue(isolation['privatePipes']);self.assertTrue(isolation['readOnlyRuntime'])
|
||||
|
||||
|
||||
if __name__=='__main__':unittest.main()
|
||||
@@ -0,0 +1,175 @@
|
||||
"""Native OS boundary tests; assertions inspect actual Windows tokens/access."""
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
@unittest.skipUnless(sys.platform == 'win32', 'Windows AppContainer native integration')
|
||||
class WindowsAppContainerTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
from windows_appcontainer import AppContainer
|
||||
cls.source = Path(__file__).resolve().parent
|
||||
cls.runtime = Path(sys.executable).resolve().parent
|
||||
cls.sandbox = AppContainer([cls.source, cls.runtime], provision=True)
|
||||
|
||||
def test_read_only_startup_never_mutates_permissions(self):
|
||||
from unittest.mock import patch
|
||||
from windows_appcontainer import AppContainer
|
||||
with patch('windows_appcontainer.subprocess.run', side_effect=AssertionError('Startup must not invoke icacls')):
|
||||
sandbox = AppContainer([self.source, self.runtime])
|
||||
code, output = sandbox.run(sys.executable, ['-c', 'print("read-only-startup")'])
|
||||
self.assertEqual(code, 0)
|
||||
self.assertEqual(output.strip(), b'read-only-startup')
|
||||
|
||||
def test_unprovisioned_runtime_fails_closed(self):
|
||||
from unittest.mock import patch
|
||||
from windows_appcontainer import AppContainer, SandboxError
|
||||
with tempfile.TemporaryDirectory(prefix='safetynet-unprovisioned-') as folder:
|
||||
with patch('windows_appcontainer.subprocess.run', side_effect=AssertionError('No automatic grants')):
|
||||
with self.assertRaisesRegex(SandboxError, 'provisioning is required'):
|
||||
AppContainer([folder])
|
||||
|
||||
def test_real_token_filesystem_network_and_secret_boundary(self):
|
||||
source = '''import ctypes,json,os,socket,sys
|
||||
from ctypes import wintypes as W
|
||||
k=ctypes.WinDLL('kernel32');a=ctypes.WinDLL('advapi32')
|
||||
k.GetCurrentProcess.restype=W.HANDLE
|
||||
a.OpenProcessToken.argtypes=[W.HANDLE,W.DWORD,ctypes.POINTER(W.HANDLE)]
|
||||
a.GetTokenInformation.argtypes=[W.HANDLE,ctypes.c_int,ctypes.c_void_p,W.DWORD,ctypes.POINTER(W.DWORD)]
|
||||
t=W.HANDLE();a.OpenProcessToken(k.GetCurrentProcess(),8,ctypes.byref(t));v=W.DWORD();n=W.DWORD();a.GetTokenInformation(t,29,ctypes.byref(v),4,ctypes.byref(n))
|
||||
r={'appContainer':v.value==1,'secretAbsent':'INSPECTOR_TOKEN' not in os.environ,'echo':sys.stdin.buffer.read().decode()}
|
||||
for key,path,mode in [('privateDenied',sys.argv[1],'rb'),('runtimeWriteDenied',sys.argv[2],'wb')]:
|
||||
try:open(path,mode).close();r[key]=False
|
||||
except PermissionError:r[key]=True
|
||||
for key,kind in [('tcpDenied',socket.SOCK_STREAM),('udpDenied',socket.SOCK_DGRAM)]:
|
||||
try:
|
||||
s=socket.socket(type=kind);s.settimeout(2)
|
||||
if kind==socket.SOCK_STREAM:s.connect(('127.0.0.1',int(sys.argv[3])))
|
||||
else:s.sendto(b'probe',('127.0.0.1',int(sys.argv[4])))
|
||||
r[key]=False
|
||||
except OSError:r[key]=True
|
||||
finally:s.close()
|
||||
print(json.dumps(r))
|
||||
'''
|
||||
listener = socket.socket();listener.bind(('127.0.0.1',0));listener.listen()
|
||||
udp = socket.socket(type=socket.SOCK_DGRAM);udp.bind(('127.0.0.1',0));udp.settimeout(0.2)
|
||||
with tempfile.TemporaryDirectory(prefix='safetynet-private-boundary-') as folder:
|
||||
private = Path(folder)/'private.txt';private.write_text('synthetic private fixture')
|
||||
target = self.runtime/'appcontainer-unauthorized-write.txt'
|
||||
try:
|
||||
code, output = self.sandbox.run(sys.executable,['-c',source,str(private),str(target),str(listener.getsockname()[1]),str(udp.getsockname()[1])], b'pipe-only fixture')
|
||||
self.assertEqual(code,0,output.decode(errors='replace'))
|
||||
result=json.loads(output)
|
||||
for key in ['appContainer','secretAbsent','privateDenied','runtimeWriteDenied','tcpDenied']:self.assertTrue(result[key],key)
|
||||
self.assertEqual(result['echo'],'pipe-only fixture')
|
||||
self.assertFalse(target.exists())
|
||||
# Windows firewall can silently discard UDP, so verify the live
|
||||
# receiver instead of trusting sendto's immediate return value.
|
||||
with self.assertRaises(socket.timeout): udp.recvfrom(1024)
|
||||
finally:listener.close();udp.close()
|
||||
|
||||
def test_child_process_inherits_sandbox(self):
|
||||
child = "import ctypes,json;from ctypes import wintypes as W;k=ctypes.WinDLL('kernel32');a=ctypes.WinDLL('advapi32');k.GetCurrentProcess.restype=W.HANDLE;a.OpenProcessToken.argtypes=[W.HANDLE,W.DWORD,ctypes.POINTER(W.HANDLE)];a.GetTokenInformation.argtypes=[W.HANDLE,ctypes.c_int,ctypes.c_void_p,W.DWORD,ctypes.POINTER(W.DWORD)];t=W.HANDLE();a.OpenProcessToken(k.GetCurrentProcess(),8,ctypes.byref(t));v=W.DWORD();n=W.DWORD();a.GetTokenInformation(t,29,ctypes.byref(v),4,ctypes.byref(n));print(v.value)"
|
||||
parent='import subprocess,sys; result=subprocess.run([sys.executable,"-c",'+repr(child)+'],capture_output=True,text=True,timeout=5);print(result.stdout.strip());sys.exit(result.returncode)'
|
||||
code, output=self.sandbox.run(sys.executable,['-c',parent])
|
||||
self.assertEqual(code,0,output.decode());self.assertEqual(output.strip(),b'1')
|
||||
|
||||
def test_worker_cannot_open_broker_memory(self):
|
||||
# Test access rights without reading any actual parent memory or secrets.
|
||||
source='''import ctypes,json,sys
|
||||
from ctypes import wintypes as W
|
||||
k=ctypes.WinDLL('kernel32',use_last_error=True)
|
||||
k.OpenProcess.argtypes=[W.DWORD,W.BOOL,W.DWORD];k.OpenProcess.restype=W.HANDLE
|
||||
k.CloseHandle.argtypes=[W.HANDLE]
|
||||
handle=k.OpenProcess(0x0010,False,int(sys.argv[1]))
|
||||
result={'brokerMemoryReadDenied':not bool(handle),'error':ctypes.get_last_error()}
|
||||
if handle:k.CloseHandle(handle)
|
||||
print(json.dumps(result))
|
||||
'''
|
||||
code, output=self.sandbox.run(sys.executable,['-c',source,str(os.getpid())])
|
||||
self.assertEqual(code,0,output.decode())
|
||||
result=json.loads(output)
|
||||
self.assertTrue(result['brokerMemoryReadDenied'])
|
||||
self.assertEqual(result['error'],5)
|
||||
|
||||
def test_real_parser_crosses_only_pipe(self):
|
||||
request={'route':'/inspect','data':base64.b64encode(b'Public text document').decode(),'policy':{'security':{},'dlp':{'rules':[], 'maxBytes':20971520}}}
|
||||
code,output=self.sandbox.run(sys.executable,[str(self.source/'sandbox_worker.py')],json.dumps(request).encode())
|
||||
self.assertEqual(code,0,output.decode(errors='replace'))
|
||||
result=json.loads(output);self.assertEqual(result['decision'],'allow');self.assertTrue(result['coverage']['complete'])
|
||||
|
||||
def test_real_yara_upload_and_download_inside_sandbox(self):
|
||||
rule={'id':'test-marker','source':'rule Block_Test { strings: $a = "SAFETYNET_BLOCK_TEST" condition: $a }'}
|
||||
policy={'security':{'yara':{'enabled':True,'rules':[rule],'maxScanBytes':1048576}},'dlp':{'maxBytes':1048576,'rules':[]}}
|
||||
for route,data,decision in [('/inspect',b'Public information','allow'),('/inspect',b'SAFETYNET_BLOCK_TEST','block'),('/scan-download',bytes([0,1,2,3,255]),'allow'),('/scan-download',b'SAFETYNET_BLOCK_TEST','block')]:
|
||||
request={'route':route,'data':base64.b64encode(data).decode(),'policy':policy}
|
||||
code,output=self.sandbox.run(sys.executable,[str(self.source/'sandbox_worker.py')],json.dumps(request).encode())
|
||||
self.assertEqual(code,0,output.decode(errors='replace'));result=json.loads(output)
|
||||
self.assertEqual(result['decision'],decision,(route,result))
|
||||
if route=='/inspect' and decision=='block':self.assertTrue(result['malware']['coverage']['complete'])
|
||||
else:self.assertTrue(result['coverage']['complete'])
|
||||
if decision=='block':self.assertEqual(result['ruleId'],'test-marker')
|
||||
|
||||
def test_orphan_worker_is_killed_when_request_ends(self):
|
||||
import ctypes
|
||||
from ctypes import wintypes as W
|
||||
source='import subprocess,sys;child=subprocess.Popen([sys.executable,"-c","import time;time.sleep(60)"]);print(child.pid,flush=True)'
|
||||
code,output=self.sandbox.run(sys.executable,['-c',source])
|
||||
self.assertEqual(code,0,output.decode());pid=int(output.strip())
|
||||
kernel=ctypes.WinDLL('kernel32',use_last_error=True)
|
||||
kernel.OpenProcess.argtypes=[W.DWORD,W.BOOL,W.DWORD];kernel.OpenProcess.restype=W.HANDLE
|
||||
kernel.WaitForSingleObject.argtypes=[W.HANDLE,W.DWORD];kernel.CloseHandle.argtypes=[W.HANDLE]
|
||||
handle=kernel.OpenProcess(0x00100000,False,pid)
|
||||
if handle:
|
||||
try:self.assertEqual(kernel.WaitForSingleObject(handle,3000),0)
|
||||
finally:kernel.CloseHandle(handle)
|
||||
|
||||
def test_large_binary_pipe_complete(self):
|
||||
import hashlib
|
||||
data=bytes(range(256))*65536
|
||||
code,output=self.sandbox.run(sys.executable,['-c','import hashlib,sys;print(hashlib.sha256(sys.stdin.buffer.read()).hexdigest())'],data)
|
||||
self.assertEqual(code,0);self.assertEqual(output.strip().decode(),hashlib.sha256(data).hexdigest())
|
||||
|
||||
def test_worker_rejects_unsandboxed_execution(self):
|
||||
result=subprocess.run([sys.executable,str(self.source/'sandbox_worker.py')],input=b'{}',capture_output=True,timeout=5)
|
||||
self.assertNotEqual(result.returncode,0);self.assertEqual(result.stdout,b'');self.assertEqual(result.stderr,b'')
|
||||
|
||||
def test_broker_dispatch_remains_authenticated_and_isolated(self):
|
||||
import queue
|
||||
import secrets
|
||||
import threading
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
token=secrets.token_urlsafe(32)
|
||||
env={key:value for key,value in os.environ.items() if key.upper() in {'SYSTEMROOT','WINDIR','USERPROFILE','APPDATA','LOCALAPPDATA','TEMP','TMP'}}
|
||||
env.update({'INSPECTOR_TOKEN':token,'INSPECTOR_PORT':'0'})
|
||||
process=subprocess.Popen([sys.executable,str(self.source/'server.py')],stdout=subprocess.PIPE,stderr=subprocess.PIPE,env=env,creationflags=subprocess.CREATE_NO_WINDOW)
|
||||
lines=queue.Queue();threading.Thread(target=lambda:lines.put(process.stdout.readline()),daemon=True).start()
|
||||
try:
|
||||
ready=json.loads(lines.get(timeout=40))
|
||||
self.assertEqual(ready['containment']['parserIsolation']['kind'],'windows-appcontainer-v1')
|
||||
request={'data':base64.b64encode(b'Public allowed document').decode(),'policy':{'security':{},'dlp':{'maxBytes':20971520,'rules':[]}}}
|
||||
url='http://127.0.0.1:'+str(ready['port'])+'/inspect'
|
||||
body=json.dumps(request).encode()
|
||||
with self.assertRaises(urllib.error.HTTPError) as denied:urllib.request.urlopen(urllib.request.Request(url,data=body),timeout=5)
|
||||
self.assertEqual(denied.exception.code,401)
|
||||
with urllib.request.urlopen(urllib.request.Request(url,data=body,headers={'Authorization':'Bearer '+token}),timeout=30) as response:result=json.load(response)
|
||||
self.assertEqual(result['decision'],'allow');self.assertTrue(result['coverage']['complete'])
|
||||
finally:
|
||||
process.kill();process.wait(timeout=5)
|
||||
process.stdout.close();process.stderr.close()
|
||||
|
||||
def test_deadline_and_response_bound_fail_closed(self):
|
||||
from windows_appcontainer import SandboxError
|
||||
with self.assertRaises(SandboxError):self.sandbox.run(sys.executable,['-c','import time;time.sleep(30)'],timeout=0.2)
|
||||
with self.assertRaises(SandboxError):self.sandbox.run(sys.executable,['-c','print("x"*10000)'],max_output=100)
|
||||
|
||||
|
||||
if __name__=='__main__':unittest.main()
|
||||
@@ -0,0 +1,83 @@
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@unittest.skipUnless(sys.platform == "win32", "Windows Job Object integration")
|
||||
class WindowsContainmentTests(unittest.TestCase):
|
||||
def run_child(self, source):
|
||||
source = f"import sys; sys.path.insert(0, {str(Path(__file__).parent)!r});\n" + source
|
||||
return subprocess.run([sys.executable, "-c", source], cwd=Path(__file__).parent,
|
||||
capture_output=True, text=True, timeout=20,
|
||||
creationflags=subprocess.CREATE_NO_WINDOW)
|
||||
|
||||
def test_process_limit_and_no_breakaway(self):
|
||||
result = self.run_child("""
|
||||
import json, subprocess, sys
|
||||
from windows_containment import contain_inspector
|
||||
status=contain_inspector()
|
||||
children=[]
|
||||
try:
|
||||
for i in range(3):
|
||||
children.append(subprocess.Popen([sys.executable,'-c','import time; time.sleep(15)'],creationflags=subprocess.CREATE_NO_WINDOW))
|
||||
denied=False
|
||||
try:
|
||||
extra=subprocess.Popen([sys.executable,'-c','pass'],creationflags=subprocess.CREATE_NO_WINDOW)
|
||||
extra.wait()
|
||||
except OSError:
|
||||
denied=True
|
||||
breakaway=False
|
||||
try:
|
||||
escaped=subprocess.Popen([sys.executable,'-c','pass'],creationflags=subprocess.CREATE_NO_WINDOW | subprocess.CREATE_BREAKAWAY_FROM_JOB)
|
||||
escaped.wait()
|
||||
except OSError:
|
||||
breakaway=True
|
||||
print(json.dumps({'status':status,'denied':denied,'breakawayDenied':breakaway}),flush=True)
|
||||
finally:
|
||||
for child in children:
|
||||
child.terminate();child.wait()
|
||||
""")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
proof = json.loads(result.stdout)
|
||||
self.assertTrue(proof["status"]["active"])
|
||||
self.assertEqual(proof["status"]["memoryBytes"], 4 * 1024 ** 3)
|
||||
self.assertTrue(proof["denied"])
|
||||
self.assertTrue(proof["breakawayDenied"])
|
||||
|
||||
def test_descendant_is_killed_when_owner_is_terminated(self):
|
||||
source = """
|
||||
import subprocess,sys,time
|
||||
from windows_containment import contain_inspector
|
||||
contain_inspector()
|
||||
child=subprocess.Popen([sys.executable,'-c','import time; time.sleep(60)'],creationflags=subprocess.CREATE_NO_WINDOW)
|
||||
print(child.pid,flush=True)
|
||||
time.sleep(60)
|
||||
"""
|
||||
source = f"import sys; sys.path.insert(0, {str(Path(__file__).parent)!r});\n" + source
|
||||
owner = subprocess.Popen([sys.executable, "-c", source], cwd=Path(__file__).parent,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True,
|
||||
creationflags=subprocess.CREATE_NO_WINDOW)
|
||||
handle = None
|
||||
try:
|
||||
# Opening the descendant before termination permits a reliable wait,
|
||||
# avoiding PID reuse or process enumeration heuristics.
|
||||
child_pid = int(owner.stdout.readline())
|
||||
import ctypes
|
||||
kernel = ctypes.WinDLL("kernel32", use_last_error=True)
|
||||
kernel.OpenProcess.argtypes = [ctypes.c_uint32, ctypes.c_int, ctypes.c_uint32]
|
||||
kernel.OpenProcess.restype = ctypes.c_void_p
|
||||
kernel.WaitForSingleObject.argtypes = [ctypes.c_void_p, ctypes.c_uint32]
|
||||
kernel.CloseHandle.argtypes = [ctypes.c_void_p]
|
||||
handle = kernel.OpenProcess(0x00100000, False, child_pid)
|
||||
self.assertTrue(handle)
|
||||
owner.terminate();owner.wait(timeout=10)
|
||||
self.assertEqual(kernel.WaitForSingleObject(handle, 10000), 0)
|
||||
finally:
|
||||
if owner.poll() is None:
|
||||
owner.terminate();owner.wait(timeout=10)
|
||||
if handle:
|
||||
kernel.CloseHandle(handle)
|
||||
owner.stdout.close();owner.stderr.close()
|
||||
@@ -0,0 +1,363 @@
|
||||
{
|
||||
"purpose": "Unused abandoned ONNX-tokenizer experiment; do not delete locked dependencies",
|
||||
"distributions": [
|
||||
{
|
||||
"name": "anyio",
|
||||
"version": "4.15.1",
|
||||
"requires": [
|
||||
"exceptiongroup>=1.0.2; python_version < \"3.11\"",
|
||||
"idna>=2.8",
|
||||
"typing_extensions>=4.16.0; python_version < \"3.15\"",
|
||||
"trio>=0.32.0; extra == \"trio\""
|
||||
],
|
||||
"topLevelPaths": [
|
||||
"anyio",
|
||||
"anyio-4.15.1.dist-info"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "click",
|
||||
"version": "8.5.0",
|
||||
"requires": [],
|
||||
"topLevelPaths": [
|
||||
"click",
|
||||
"click-8.5.0.dist-info"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "filelock",
|
||||
"version": "4.1.1",
|
||||
"requires": [],
|
||||
"topLevelPaths": [
|
||||
"filelock",
|
||||
"filelock-4.1.1.dist-info"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "fsspec",
|
||||
"version": "2026.9.0",
|
||||
"requires": [
|
||||
"adlfs; extra == 'abfs'",
|
||||
"adlfs; extra == 'adl'",
|
||||
"pyarrow>=1; extra == 'arrow'",
|
||||
"dask; extra == 'dask'",
|
||||
"distributed; extra == 'dask'",
|
||||
"pre-commit; extra == 'dev'",
|
||||
"ruff>=0.5; extra == 'dev'",
|
||||
"numpydoc; extra == 'doc'",
|
||||
"sphinx; extra == 'doc'",
|
||||
"sphinx-design; extra == 'doc'",
|
||||
"sphinx-rtd-theme; extra == 'doc'",
|
||||
"yarl; extra == 'doc'",
|
||||
"dropbox; extra == 'dropbox'",
|
||||
"dropboxdrivefs; extra == 'dropbox'",
|
||||
"requests; extra == 'dropbox'",
|
||||
"adlfs; extra == 'full'",
|
||||
"aiohttp!=4.0.0a0,!=4.0.0a1; extra == 'full'",
|
||||
"dask; extra == 'full'",
|
||||
"distributed; extra == 'full'",
|
||||
"dropbox; extra == 'full'",
|
||||
"dropboxdrivefs; extra == 'full'",
|
||||
"fusepy; extra == 'full'",
|
||||
"gcsfs>=2026.4.0; extra == 'full'",
|
||||
"libarchive-c; extra == 'full'",
|
||||
"ocifs; extra == 'full'",
|
||||
"panel; extra == 'full'",
|
||||
"paramiko; extra == 'full'",
|
||||
"pyarrow>=1; extra == 'full'",
|
||||
"pygit2; extra == 'full'",
|
||||
"requests; extra == 'full'",
|
||||
"s3fs>=2026.6.0; extra == 'full'",
|
||||
"smbprotocol; extra == 'full'",
|
||||
"tqdm; extra == 'full'",
|
||||
"fusepy; extra == 'fuse'",
|
||||
"gcsfs>=2026.4.0; extra == 'gcs'",
|
||||
"pygit2; extra == 'git'",
|
||||
"requests; extra == 'github'",
|
||||
"gcsfs>=2026.4.0; extra == 'gs'",
|
||||
"panel; extra == 'gui'",
|
||||
"pyarrow>=1; extra == 'hdfs'",
|
||||
"aiohttp!=4.0.0a0,!=4.0.0a1; extra == 'http'",
|
||||
"libarchive-c; extra == 'libarchive'",
|
||||
"ocifs; extra == 'oci'",
|
||||
"s3fs>=2026.6.0; extra == 's3'",
|
||||
"paramiko; extra == 'sftp'",
|
||||
"smbprotocol; extra == 'smb'",
|
||||
"paramiko; extra == 'ssh'",
|
||||
"aiohttp!=4.0.0a0,!=4.0.0a1; extra == 'test'",
|
||||
"numpy; extra == 'test'",
|
||||
"pytest; extra == 'test'",
|
||||
"pytest-asyncio!=0.22.0; extra == 'test'",
|
||||
"pytest-benchmark; extra == 'test'",
|
||||
"pytest-cov; extra == 'test'",
|
||||
"pytest-mock; extra == 'test'",
|
||||
"pytest-recording; extra == 'test'",
|
||||
"pytest-rerunfailures; extra == 'test'",
|
||||
"requests; extra == 'test'",
|
||||
"aiobotocore<3.0.0,>=2.5.4; extra == 'test-downstream'",
|
||||
"dask[dataframe,test]; extra == 'test-downstream'",
|
||||
"moto[server]<5,>4; extra == 'test-downstream'",
|
||||
"pytest-timeout; extra == 'test-downstream'",
|
||||
"xarray; extra == 'test-downstream'",
|
||||
"zarr; extra == 'test-downstream'",
|
||||
"adlfs; extra == 'test-full'",
|
||||
"aiohttp!=4.0.0a0,!=4.0.0a1; extra == 'test-full'",
|
||||
"backports-zstd; (python_version < '3.14') and extra == 'test-full'",
|
||||
"cloudpickle; extra == 'test-full'",
|
||||
"dask; extra == 'test-full'",
|
||||
"distributed; extra == 'test-full'",
|
||||
"dropbox; extra == 'test-full'",
|
||||
"dropboxdrivefs; extra == 'test-full'",
|
||||
"fastparquet; extra == 'test-full'",
|
||||
"fusepy; extra == 'test-full'",
|
||||
"gcsfs>=2026.4.0; extra == 'test-full'",
|
||||
"jinja2; extra == 'test-full'",
|
||||
"kerchunk; extra == 'test-full'",
|
||||
"libarchive-c; extra == 'test-full'",
|
||||
"lz4; extra == 'test-full'",
|
||||
"notebook; extra == 'test-full'",
|
||||
"numpy; extra == 'test-full'",
|
||||
"ocifs; extra == 'test-full'",
|
||||
"pandas<3.0.0; extra == 'test-full'",
|
||||
"panel; extra == 'test-full'",
|
||||
"paramiko; extra == 'test-full'",
|
||||
"pyarrow>=1; extra == 'test-full'",
|
||||
"pyftpdlib; extra == 'test-full'",
|
||||
"pygit2; extra == 'test-full'",
|
||||
"pytest; extra == 'test-full'",
|
||||
"pytest-asyncio!=0.22.0; extra == 'test-full'",
|
||||
"pytest-benchmark; extra == 'test-full'",
|
||||
"pytest-cov; extra == 'test-full'",
|
||||
"pytest-mock; extra == 'test-full'",
|
||||
"pytest-recording; extra == 'test-full'",
|
||||
"pytest-rerunfailures; extra == 'test-full'",
|
||||
"python-snappy; extra == 'test-full'",
|
||||
"requests; extra == 'test-full'",
|
||||
"s3fs>=2026.6.0; extra == 'test-full'",
|
||||
"smbprotocol; extra == 'test-full'",
|
||||
"tqdm; extra == 'test-full'",
|
||||
"urllib3; extra == 'test-full'",
|
||||
"zarr<3.2.0; extra == 'test-full'",
|
||||
"zstandard; (python_version < '3.14') and extra == 'test-full'",
|
||||
"tqdm; extra == 'tqdm'"
|
||||
],
|
||||
"topLevelPaths": [
|
||||
"fsspec",
|
||||
"fsspec-2026.9.0.dist-info"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "h11",
|
||||
"version": "0.16.0",
|
||||
"requires": [],
|
||||
"topLevelPaths": [
|
||||
"h11",
|
||||
"h11-0.16.0.dist-info"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "hf-xet",
|
||||
"version": "1.7.0",
|
||||
"requires": [
|
||||
"pytest ; extra == 'tests'"
|
||||
],
|
||||
"topLevelPaths": [
|
||||
"hf_xet",
|
||||
"hf_xet-1.7.0.dist-info"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "httpcore",
|
||||
"version": "1.0.9",
|
||||
"requires": [
|
||||
"certifi",
|
||||
"h11>=0.16",
|
||||
"anyio<5.0,>=4.0; extra == 'asyncio'",
|
||||
"h2<5,>=3; extra == 'http2'",
|
||||
"socksio==1.*; extra == 'socks'",
|
||||
"trio<1.0,>=0.22.0; extra == 'trio'"
|
||||
],
|
||||
"topLevelPaths": [
|
||||
"httpcore",
|
||||
"httpcore-1.0.9.dist-info"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "httpx",
|
||||
"version": "0.28.1",
|
||||
"requires": [
|
||||
"anyio",
|
||||
"certifi",
|
||||
"httpcore==1.*",
|
||||
"idna",
|
||||
"brotli; (platform_python_implementation == 'CPython') and extra == 'brotli'",
|
||||
"brotlicffi; (platform_python_implementation != 'CPython') and extra == 'brotli'",
|
||||
"click==8.*; extra == 'cli'",
|
||||
"pygments==2.*; extra == 'cli'",
|
||||
"rich<14,>=10; extra == 'cli'",
|
||||
"h2<5,>=3; extra == 'http2'",
|
||||
"socksio==1.*; extra == 'socks'",
|
||||
"zstandard>=0.18.0; extra == 'zstd'"
|
||||
],
|
||||
"topLevelPaths": [
|
||||
"httpx",
|
||||
"httpx-0.28.1.dist-info"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "huggingface_hub",
|
||||
"version": "1.33.0",
|
||||
"requires": [
|
||||
"click<9.0.0,>=8.4.2",
|
||||
"filelock>=3.10.0",
|
||||
"fsspec>=2023.5.0",
|
||||
"hf-xet<2.0.0,>=1.6.0; platform_machine == \"x86_64\" or platform_machine == \"amd64\" or platform_machine == \"AMD64\" or platform_machine == \"arm64\" or platform_machine == \"aarch64\"",
|
||||
"httpx<1,>=0.23.0",
|
||||
"packaging>=20.9",
|
||||
"pyyaml>=5.1",
|
||||
"tomli>=1.1.0; python_version < \"3.11\"",
|
||||
"tqdm>=4.42.1",
|
||||
"typing-extensions>=4.1.0",
|
||||
"authlib>=1.3.2; extra == \"oauth\"",
|
||||
"fastapi; extra == \"oauth\"",
|
||||
"httpx; extra == \"oauth\"",
|
||||
"itsdangerous; extra == \"oauth\"",
|
||||
"torch; extra == \"torch\"",
|
||||
"safetensors[torch]; extra == \"torch\"",
|
||||
"toml; extra == \"fastai\"",
|
||||
"fastai>=2.4; extra == \"fastai\"",
|
||||
"fastcore>=1.3.27; extra == \"fastai\"",
|
||||
"hf-xet<2.0.0,>=1.6.0; extra == \"hf-xet\"",
|
||||
"mcp<2.0.0,>=1.9.4; extra == \"mcp\"",
|
||||
"authlib>=1.3.2; extra == \"testing\"",
|
||||
"fastapi; extra == \"testing\"",
|
||||
"httpx; extra == \"testing\"",
|
||||
"itsdangerous; extra == \"testing\"",
|
||||
"jedi; extra == \"testing\"",
|
||||
"Jinja2; extra == \"testing\"",
|
||||
"pytest>=8.4.2; extra == \"testing\"",
|
||||
"pytest-cov; extra == \"testing\"",
|
||||
"pytest-env; extra == \"testing\"",
|
||||
"pytest-xdist; extra == \"testing\"",
|
||||
"pytest-vcr; extra == \"testing\"",
|
||||
"pytest-asyncio; extra == \"testing\"",
|
||||
"pytest-rerunfailures>=16.2; extra == \"testing\"",
|
||||
"pytest-mock; extra == \"testing\"",
|
||||
"urllib3<2.0; extra == \"testing\"",
|
||||
"soundfile; extra == \"testing\"",
|
||||
"Pillow; extra == \"testing\"",
|
||||
"numpy; extra == \"testing\"",
|
||||
"duckdb; extra == \"testing\"",
|
||||
"fastapi; extra == \"testing\"",
|
||||
"gradio>=5.0.0; extra == \"gradio\"",
|
||||
"requests; extra == \"gradio\"",
|
||||
"typing-extensions>=4.8.0; extra == \"typing\"",
|
||||
"types-PyYAML; extra == \"typing\"",
|
||||
"types-simplejson; extra == \"typing\"",
|
||||
"types-toml; extra == \"typing\"",
|
||||
"types-tqdm; extra == \"typing\"",
|
||||
"types-urllib3; extra == \"typing\"",
|
||||
"ruff>=0.9.0; extra == \"quality\"",
|
||||
"mypy==1.15.0; extra == \"quality\"",
|
||||
"libcst>=1.4.0; extra == \"quality\"",
|
||||
"ty; extra == \"quality\"",
|
||||
"authlib>=1.3.2; extra == \"all\"",
|
||||
"fastapi; extra == \"all\"",
|
||||
"httpx; extra == \"all\"",
|
||||
"itsdangerous; extra == \"all\"",
|
||||
"jedi; extra == \"all\"",
|
||||
"Jinja2; extra == \"all\"",
|
||||
"pytest>=8.4.2; extra == \"all\"",
|
||||
"pytest-cov; extra == \"all\"",
|
||||
"pytest-env; extra == \"all\"",
|
||||
"pytest-xdist; extra == \"all\"",
|
||||
"pytest-vcr; extra == \"all\"",
|
||||
"pytest-asyncio; extra == \"all\"",
|
||||
"pytest-rerunfailures>=16.2; extra == \"all\"",
|
||||
"pytest-mock; extra == \"all\"",
|
||||
"urllib3<2.0; extra == \"all\"",
|
||||
"soundfile; extra == \"all\"",
|
||||
"Pillow; extra == \"all\"",
|
||||
"numpy; extra == \"all\"",
|
||||
"duckdb; extra == \"all\"",
|
||||
"fastapi; extra == \"all\"",
|
||||
"ruff>=0.9.0; extra == \"all\"",
|
||||
"mypy==1.15.0; extra == \"all\"",
|
||||
"libcst>=1.4.0; extra == \"all\"",
|
||||
"ty; extra == \"all\"",
|
||||
"typing-extensions>=4.8.0; extra == \"all\"",
|
||||
"types-PyYAML; extra == \"all\"",
|
||||
"types-simplejson; extra == \"all\"",
|
||||
"types-toml; extra == \"all\"",
|
||||
"types-tqdm; extra == \"all\"",
|
||||
"types-urllib3; extra == \"all\"",
|
||||
"authlib>=1.3.2; extra == \"dev\"",
|
||||
"fastapi; extra == \"dev\"",
|
||||
"httpx; extra == \"dev\"",
|
||||
"itsdangerous; extra == \"dev\"",
|
||||
"jedi; extra == \"dev\"",
|
||||
"Jinja2; extra == \"dev\"",
|
||||
"pytest>=8.4.2; extra == \"dev\"",
|
||||
"pytest-cov; extra == \"dev\"",
|
||||
"pytest-env; extra == \"dev\"",
|
||||
"pytest-xdist; extra == \"dev\"",
|
||||
"pytest-vcr; extra == \"dev\"",
|
||||
"pytest-asyncio; extra == \"dev\"",
|
||||
"pytest-rerunfailures>=16.2; extra == \"dev\"",
|
||||
"pytest-mock; extra == \"dev\"",
|
||||
"urllib3<2.0; extra == \"dev\"",
|
||||
"soundfile; extra == \"dev\"",
|
||||
"Pillow; extra == \"dev\"",
|
||||
"numpy; extra == \"dev\"",
|
||||
"duckdb; extra == \"dev\"",
|
||||
"fastapi; extra == \"dev\"",
|
||||
"ruff>=0.9.0; extra == \"dev\"",
|
||||
"mypy==1.15.0; extra == \"dev\"",
|
||||
"libcst>=1.4.0; extra == \"dev\"",
|
||||
"ty; extra == \"dev\"",
|
||||
"typing-extensions>=4.8.0; extra == \"dev\"",
|
||||
"types-PyYAML; extra == \"dev\"",
|
||||
"types-simplejson; extra == \"dev\"",
|
||||
"types-toml; extra == \"dev\"",
|
||||
"types-tqdm; extra == \"dev\"",
|
||||
"types-urllib3; extra == \"dev\""
|
||||
],
|
||||
"topLevelPaths": [
|
||||
"huggingface_hub",
|
||||
"huggingface_hub-1.33.0.dist-info"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "tokenizers",
|
||||
"version": "0.22.2",
|
||||
"requires": [
|
||||
"huggingface-hub>=0.16.4,<2.0",
|
||||
"pytest ; extra == 'testing'",
|
||||
"pytest-asyncio ; extra == 'testing'",
|
||||
"requests ; extra == 'testing'",
|
||||
"numpy ; extra == 'testing'",
|
||||
"datasets ; extra == 'testing'",
|
||||
"ruff ; extra == 'testing'",
|
||||
"ty ; extra == 'testing'",
|
||||
"sphinx ; extra == 'docs'",
|
||||
"sphinx-rtd-theme ; extra == 'docs'",
|
||||
"setuptools-rust ; extra == 'docs'",
|
||||
"tokenizers[testing] ; extra == 'dev'"
|
||||
],
|
||||
"topLevelPaths": [
|
||||
"tokenizers",
|
||||
"tokenizers-0.22.2.dist-info"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "typing_extensions",
|
||||
"version": "4.16.0",
|
||||
"requires": [],
|
||||
"topLevelPaths": [
|
||||
"__pycache__",
|
||||
"typing_extensions-4.16.0.dist-info",
|
||||
"typing_extensions.py"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,271 @@
|
||||
"""Capability-free Windows AppContainer process launcher.
|
||||
|
||||
The HTTP broker must stay outside this boundary. Only approved immutable runtime,
|
||||
parser source and model directories receive read/execute access. Documents travel
|
||||
through anonymous pipes, never a temporary plaintext file. No network capability
|
||||
or loopback exemption is assigned. Windows supplies the package-private storage.
|
||||
"""
|
||||
import ctypes
|
||||
import hashlib
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from ctypes import wintypes as W
|
||||
|
||||
|
||||
class SandboxError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
def _verify_read_acl(path, sid):
|
||||
"""Read the DACL directly; startup never needs WRITE_DAC or icacls."""
|
||||
a = ctypes.WinDLL('advapi32', use_last_error=True)
|
||||
k = ctypes.WinDLL('kernel32', use_last_error=True)
|
||||
a.GetNamedSecurityInfoW.argtypes = [W.LPWSTR, W.DWORD, W.DWORD, ctypes.c_void_p, ctypes.c_void_p, ctypes.POINTER(ctypes.c_void_p), ctypes.c_void_p, ctypes.POINTER(ctypes.c_void_p)]
|
||||
a.GetNamedSecurityInfoW.restype = W.DWORD
|
||||
a.GetAce.argtypes = [ctypes.c_void_p, W.DWORD, ctypes.POINTER(ctypes.c_void_p)]
|
||||
a.GetAce.restype = W.BOOL
|
||||
a.EqualSid.argtypes = [ctypes.c_void_p, ctypes.c_void_p]; a.EqualSid.restype = W.BOOL
|
||||
k.LocalFree.argtypes = [ctypes.c_void_p]; k.LocalFree.restype = ctypes.c_void_p
|
||||
dacl = ctypes.c_void_p(); descriptor = ctypes.c_void_p()
|
||||
if a.GetNamedSecurityInfoW(str(path), 1, 4, None, None, ctypes.byref(dacl), None, ctypes.byref(descriptor)):
|
||||
raise SandboxError('Approved runtime ACL could not be inspected')
|
||||
try:
|
||||
if not dacl: raise SandboxError('Approved runtime has unrestricted access')
|
||||
count = ctypes.c_ushort.from_address(dacl.value + 4).value
|
||||
allow = 0
|
||||
required = 0x1200a9 # FILE_GENERIC_READ | FILE_GENERIC_EXECUTE
|
||||
for index in range(count):
|
||||
ace = ctypes.c_void_p()
|
||||
if not a.GetAce(dacl, index, ctypes.byref(ace)): raise SandboxError('Runtime ACL is invalid')
|
||||
kind = ctypes.c_ubyte.from_address(ace.value).value
|
||||
flags = ctypes.c_ubyte.from_address(ace.value + 1).value
|
||||
if flags & 8: continue # INHERIT_ONLY does not apply to this object.
|
||||
mask = ctypes.c_uint32.from_address(ace.value + 4).value
|
||||
# Conservative: a relevant deny ACE requires administrator review.
|
||||
if kind == 1 and mask & (required | 0x90000000):
|
||||
raise SandboxError('Approved runtime has conflicting deny permissions')
|
||||
if kind == 0 and a.EqualSid(ctypes.c_void_p(ace.value + 8), sid):
|
||||
if mask & 0x10000000: mask |= 0x1f01ff
|
||||
if mask & 0x80000000: mask |= 0x120089
|
||||
if mask & 0x20000000: mask |= 0x1200a0
|
||||
allow |= mask
|
||||
if allow & required != required:
|
||||
raise SandboxError('Runtime ACL provisioning is required before inspector startup')
|
||||
if allow & 0x400d0116: # write/append/EA/attributes/delete/WRITE_DAC/WRITE_OWNER
|
||||
raise SandboxError('AppContainer runtime access must be read-only')
|
||||
finally:
|
||||
k.LocalFree(descriptor)
|
||||
|
||||
|
||||
def _api():
|
||||
if sys.platform != 'win32':
|
||||
raise SandboxError('AppContainer requires Windows')
|
||||
k = ctypes.WinDLL('kernel32', use_last_error=True)
|
||||
u = ctypes.WinDLL('userenv', use_last_error=True)
|
||||
a = ctypes.WinDLL('advapi32', use_last_error=True)
|
||||
for name, args, result in [
|
||||
('CloseHandle', [W.HANDLE], W.BOOL),
|
||||
('CreatePipe', [ctypes.POINTER(W.HANDLE), ctypes.POINTER(W.HANDLE), ctypes.c_void_p, W.DWORD], W.BOOL),
|
||||
('SetHandleInformation', [W.HANDLE, W.DWORD, W.DWORD], W.BOOL),
|
||||
('InitializeProcThreadAttributeList', [ctypes.c_void_p, W.DWORD, W.DWORD, ctypes.POINTER(ctypes.c_size_t)], W.BOOL),
|
||||
('UpdateProcThreadAttribute', [ctypes.c_void_p, W.DWORD, ctypes.c_size_t, ctypes.c_void_p, ctypes.c_size_t, ctypes.c_void_p, ctypes.c_void_p], W.BOOL),
|
||||
('DeleteProcThreadAttributeList', [ctypes.c_void_p], None),
|
||||
('CreateProcessW', [W.LPCWSTR, W.LPWSTR, ctypes.c_void_p, ctypes.c_void_p, W.BOOL, W.DWORD, ctypes.c_void_p, W.LPCWSTR, ctypes.c_void_p, ctypes.c_void_p], W.BOOL),
|
||||
('WaitForSingleObject', [W.HANDLE, W.DWORD], W.DWORD),
|
||||
('TerminateProcess', [W.HANDLE, W.UINT], W.BOOL),
|
||||
('GetExitCodeProcess', [W.HANDLE, ctypes.POINTER(W.DWORD)], W.BOOL),
|
||||
('LocalFree', [ctypes.c_void_p], ctypes.c_void_p),
|
||||
('CreateJobObjectW', [ctypes.c_void_p, W.LPCWSTR], W.HANDLE),
|
||||
('SetInformationJobObject', [W.HANDLE, ctypes.c_int, ctypes.c_void_p, W.DWORD], W.BOOL),
|
||||
('AssignProcessToJobObject', [W.HANDLE, W.HANDLE], W.BOOL),
|
||||
('ResumeThread', [W.HANDLE], W.DWORD),
|
||||
]:
|
||||
fn = getattr(k, name); fn.argtypes = args; fn.restype = result
|
||||
u.CreateAppContainerProfile.argtypes = [W.LPCWSTR, W.LPCWSTR, W.LPCWSTR, ctypes.c_void_p, W.DWORD, ctypes.POINTER(ctypes.c_void_p)]
|
||||
u.CreateAppContainerProfile.restype = ctypes.c_long
|
||||
u.DeriveAppContainerSidFromAppContainerName.argtypes = [W.LPCWSTR, ctypes.POINTER(ctypes.c_void_p)]
|
||||
u.DeriveAppContainerSidFromAppContainerName.restype = ctypes.c_long
|
||||
a.ConvertSidToStringSidW.argtypes = [ctypes.c_void_p, ctypes.POINTER(W.LPWSTR)]
|
||||
a.ConvertSidToStringSidW.restype = W.BOOL
|
||||
a.FreeSid.argtypes = [ctypes.c_void_p]; a.FreeSid.restype = ctypes.c_void_p
|
||||
return k, u, a
|
||||
|
||||
|
||||
class AppContainer:
|
||||
"""Broker-owned launcher. Caller supplies trusted deployment directories."""
|
||||
def __init__(self, read_directories, *, provision=False):
|
||||
self.k, u, self.a = _api()
|
||||
roots_input = [Path(p).absolute() for p in read_directories]
|
||||
for root in roots_input:
|
||||
for component in [root, *root.parents]:
|
||||
if component.lstat().st_file_attributes & 0x400:
|
||||
raise SandboxError('Reparse point in approved sandbox root')
|
||||
roots = tuple(sorted({str(p.resolve(strict=True)) for p in roots_input}))
|
||||
if not roots or any(not Path(p).is_dir() or Path(p).parent == Path(p) for p in roots):
|
||||
raise SandboxError('Invalid approved sandbox runtime directories')
|
||||
self.name = 'SafetyNet.Parser.' + hashlib.sha256('\0'.join(roots).encode()).hexdigest()[:24]
|
||||
self.sid = ctypes.c_void_p()
|
||||
result = u.CreateAppContainerProfile(self.name, 'SafetyNet parser', 'Offline content inspection', None, 0, ctypes.byref(self.sid))
|
||||
if result != 0 and u.DeriveAppContainerSidFromAppContainerName(self.name, ctypes.byref(self.sid)) != 0:
|
||||
raise SandboxError('AppContainer profile could not be established')
|
||||
sid_text = W.LPWSTR()
|
||||
if not self.a.ConvertSidToStringSidW(self.sid, ctypes.byref(sid_text)):
|
||||
raise SandboxError('AppContainer identity unavailable')
|
||||
try:
|
||||
self.sid_string = sid_text.value
|
||||
finally:
|
||||
self.k.LocalFree(sid_text)
|
||||
# Package SID grants are specific to this deployment, never Everyone or
|
||||
# ALL APPLICATION PACKAGES. No writable permission is added to source.
|
||||
for root in roots:
|
||||
# Reject junctions/symlinks before recursive grants; /L alone is not
|
||||
# sufficient evidence that every possible reparse target stays local.
|
||||
entries = [Path(root)]
|
||||
for directory, folders, files in os.walk(root, followlinks=False):
|
||||
for name in folders + files:
|
||||
entry = Path(directory) / name
|
||||
if entry.lstat().st_file_attributes & 0x400:
|
||||
raise SandboxError('Reparse point in approved sandbox runtime')
|
||||
entries.append(entry)
|
||||
if provision:
|
||||
completed = subprocess.run(['icacls.exe', root, '/grant:r', '*' + self.sid_string + ':(RX)', '/T', '/L', '/Q'],
|
||||
capture_output=True, timeout=60, creationflags=subprocess.CREATE_NO_WINDOW)
|
||||
if completed.returncode:
|
||||
raise SandboxError('Approved runtime access could not be established')
|
||||
for entry in entries:
|
||||
_verify_read_acl(entry, self.sid)
|
||||
self.roots = roots
|
||||
|
||||
def run(self, executable, arguments, data=b'', environment=None, timeout=30, max_output=8*1024*1024):
|
||||
"""One bounded pipe exchange. Child descendants remain in broker Job."""
|
||||
import msvcrt
|
||||
executable = str(Path(executable).resolve(strict=True))
|
||||
if not any(Path(executable).is_relative_to(Path(root)) for root in self.roots):
|
||||
raise SandboxError('Executable is outside approved runtime')
|
||||
class SA(ctypes.Structure):
|
||||
_fields_ = [('length', W.DWORD), ('descriptor', ctypes.c_void_p), ('inherit', W.BOOL)]
|
||||
class SI(ctypes.Structure):
|
||||
_fields_ = [('cb', W.DWORD), ('reserved', W.LPWSTR), ('desktop', W.LPWSTR), ('title', W.LPWSTR),
|
||||
('x', W.DWORD), ('y', W.DWORD), ('xs', W.DWORD), ('ys', W.DWORD), ('xc', W.DWORD), ('yc', W.DWORD),
|
||||
('fill', W.DWORD), ('flags', W.DWORD), ('show', W.WORD), ('reserved2size', W.WORD),
|
||||
('reserved2', ctypes.c_void_p), ('stdin', W.HANDLE), ('stdout', W.HANDLE), ('stderr', W.HANDLE)]
|
||||
class SIX(ctypes.Structure):
|
||||
_fields_ = [('startup', SI), ('attributes', ctypes.c_void_p)]
|
||||
class PI(ctypes.Structure):
|
||||
_fields_ = [('process', W.HANDLE), ('thread', W.HANDLE), ('pid', W.DWORD), ('tid', W.DWORD)]
|
||||
class SC(ctypes.Structure):
|
||||
_fields_ = [('sid', ctypes.c_void_p), ('capabilities', ctypes.c_void_p), ('count', W.DWORD), ('reserved', W.DWORD)]
|
||||
security = SA(ctypes.sizeof(SA), None, True)
|
||||
handles = []
|
||||
attribute = None
|
||||
process = PI()
|
||||
job = None
|
||||
owned_files = []
|
||||
try:
|
||||
ir, iw, or_, ow = W.HANDLE(), W.HANDLE(), W.HANDLE(), W.HANDLE()
|
||||
for read, write in [(ir, iw), (or_, ow)]:
|
||||
if not self.k.CreatePipe(ctypes.byref(read), ctypes.byref(write), ctypes.byref(security), 0):
|
||||
raise SandboxError('Sandbox pipes unavailable')
|
||||
handles.extend([read.value, write.value])
|
||||
for parent_handle in [iw, or_]:
|
||||
if not self.k.SetHandleInformation(parent_handle, 1, 0):
|
||||
raise SandboxError('Sandbox pipe inheritance unavailable')
|
||||
size = ctypes.c_size_t()
|
||||
self.k.InitializeProcThreadAttributeList(None, 2, 0, ctypes.byref(size))
|
||||
attribute = ctypes.create_string_buffer(size.value)
|
||||
if not self.k.InitializeProcThreadAttributeList(attribute, 2, 0, ctypes.byref(size)):
|
||||
raise SandboxError('Sandbox launch attributes unavailable')
|
||||
caps = SC(self.sid, None, 0, 0)
|
||||
inheritance = (W.HANDLE * 2)(ir.value, ow.value)
|
||||
for key, value in [(0x20009, caps), (0x20002, inheritance)]:
|
||||
if not self.k.UpdateProcThreadAttribute(attribute, 0, key, ctypes.byref(value), ctypes.sizeof(value), None, None):
|
||||
raise SandboxError('Sandbox launch boundary unavailable')
|
||||
startup = SIX(); startup.startup.cb = ctypes.sizeof(SIX)
|
||||
startup.startup.flags = 0x100; startup.startup.stdin = ir; startup.startup.stdout = ow; startup.startup.stderr = ow
|
||||
startup.attributes = ctypes.cast(attribute, ctypes.c_void_p)
|
||||
# Explicit environment: no broker credential or inherited deployment secrets.
|
||||
env = {'SystemRoot': os.environ.get('SystemRoot', 'C:\\Windows'), 'PYTHONDONTWRITEBYTECODE': '1', 'PYTHONIOENCODING':'utf-8'}
|
||||
for key in ['USERPROFILE','LOCALAPPDATA','APPDATA','TEMP','TMP','WINDIR']:
|
||||
if key in os.environ: env[key] = os.environ[key]
|
||||
allowed = {'EMBEDDING_MODEL_DIR','EMBEDDING_WORKER_PATH','SEARCHI_FORCE_CPU','GGML_CPU_ONLY','GGML_DISABLE_VULKAN'}
|
||||
if any(key not in allowed or not isinstance(value,str) or '\0' in value for key,value in (environment or {}).items()):
|
||||
raise SandboxError('Unapproved AppContainer environment field')
|
||||
env.update(environment or {})
|
||||
environment_block = ctypes.create_unicode_buffer('\0'.join(f'{key}={value}' for key, value in sorted(env.items())) + '\0\0')
|
||||
command = ctypes.create_unicode_buffer(subprocess.list2cmdline([executable, *arguments]))
|
||||
if not self.k.CreateProcessW(executable, command, None, None, True, 0x80000|0x400|0x8000000|0x4,
|
||||
environment_block, self.roots[0], ctypes.byref(startup), ctypes.byref(process)):
|
||||
raise SandboxError('AppContainer process launch failed: ' + str(ctypes.get_last_error()))
|
||||
# Suspended launch avoids an execution window before descendant cleanup.
|
||||
class BasicLimits(ctypes.Structure):
|
||||
_fields_ = [('processTime', ctypes.c_int64), ('jobTime', ctypes.c_int64), ('flags', W.DWORD),
|
||||
('minWorking', ctypes.c_size_t), ('maxWorking', ctypes.c_size_t), ('processes', W.DWORD),
|
||||
('affinity', ctypes.c_size_t), ('priority', W.DWORD), ('scheduling', W.DWORD)]
|
||||
class Extended(ctypes.Structure):
|
||||
_fields_ = [('basic', BasicLimits), ('io', ctypes.c_uint64 * 6), ('processMemory', ctypes.c_size_t), ('jobMemory', ctypes.c_size_t), ('peakProcess', ctypes.c_size_t), ('peakJob', ctypes.c_size_t)]
|
||||
limits = Extended(); limits.basic.flags = 0x2000 | 0x8; limits.basic.processes = 2
|
||||
job = self.k.CreateJobObjectW(None, None)
|
||||
if not job or not self.k.SetInformationJobObject(job, 9, ctypes.byref(limits), ctypes.sizeof(limits)) or not self.k.AssignProcessToJobObject(job, process.process):
|
||||
self.k.TerminateProcess(process.process, 1)
|
||||
raise SandboxError('AppContainer descendant containment unavailable')
|
||||
if self.k.ResumeThread(process.thread) == 0xffffffff:
|
||||
raise SandboxError('AppContainer launch resume failed')
|
||||
self.k.CloseHandle(process.thread); process.thread = None
|
||||
for child_handle in [ir.value, ow.value]:
|
||||
self.k.CloseHandle(child_handle); handles.remove(child_handle)
|
||||
writer = os.fdopen(msvcrt.open_osfhandle(iw.value, os.O_WRONLY|os.O_BINARY), 'wb', buffering=0)
|
||||
reader = os.fdopen(msvcrt.open_osfhandle(or_.value, os.O_RDONLY|os.O_BINARY), 'rb', buffering=0)
|
||||
handles.remove(iw.value); handles.remove(or_.value); owned_files = [writer, reader]
|
||||
chunks = []; exceeded = threading.Event(); errors = []
|
||||
def write():
|
||||
try:
|
||||
position = 0
|
||||
while position < len(data):
|
||||
count = writer.write(memoryview(data)[position:])
|
||||
if not count: raise OSError()
|
||||
position += count
|
||||
except OSError:
|
||||
pass
|
||||
finally:
|
||||
writer.close()
|
||||
def read():
|
||||
total = 0
|
||||
try:
|
||||
while True:
|
||||
chunk = reader.read(65536)
|
||||
if not chunk: break
|
||||
total += len(chunk)
|
||||
if total > max_output:
|
||||
exceeded.set(); self.k.TerminateProcess(process.process, 1); break
|
||||
chunks.append(chunk)
|
||||
except OSError:
|
||||
errors.append(True)
|
||||
threads = [threading.Thread(target=write, daemon=True), threading.Thread(target=read, daemon=True)]
|
||||
for thread in threads: thread.start()
|
||||
if self.k.WaitForSingleObject(process.process, int(timeout*1000)) != 0:
|
||||
self.k.TerminateProcess(process.process, 1)
|
||||
self.k.WaitForSingleObject(process.process, 5000)
|
||||
raise SandboxError('AppContainer inspection deadline exceeded')
|
||||
self.k.CloseHandle(job);job=None
|
||||
for thread in threads: thread.join(2)
|
||||
if exceeded.is_set() or errors or any(thread.is_alive() for thread in threads):
|
||||
raise SandboxError('AppContainer response failed bounded exchange')
|
||||
code = W.DWORD()
|
||||
if not self.k.GetExitCodeProcess(process.process, ctypes.byref(code)):
|
||||
raise SandboxError('AppContainer process status unavailable')
|
||||
return code.value, b''.join(chunks)
|
||||
finally:
|
||||
if job: self.k.CloseHandle(job);job=None
|
||||
for file in owned_files:
|
||||
if not file.closed: file.close()
|
||||
for handle in handles: self.k.CloseHandle(handle)
|
||||
if job: self.k.CloseHandle(job)
|
||||
if process.thread: self.k.CloseHandle(process.thread)
|
||||
if process.process: self.k.CloseHandle(process.process)
|
||||
if attribute is not None: self.k.DeleteProcThreadAttributeList(attribute)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Windows resource containment for the inspector and all descendant workers.
|
||||
|
||||
This is a Job Object, not a restricted-token or filesystem/network sandbox.
|
||||
The inspector owns the handle: terminating it also terminates its model workers.
|
||||
"""
|
||||
import ctypes
|
||||
import sys
|
||||
from ctypes import wintypes
|
||||
|
||||
_job_handle = None
|
||||
MEMORY_LIMIT = 4 * 1024 * 1024 * 1024
|
||||
PROCESS_LIMIT = 4
|
||||
|
||||
|
||||
def contain_inspector():
|
||||
global _job_handle
|
||||
if sys.platform != "win32":
|
||||
return {"active": False, "kind": "unsupported-platform"}
|
||||
if _job_handle:
|
||||
return {"active": True, "kind": "windows-job-v1"}
|
||||
|
||||
class BasicLimits(ctypes.Structure):
|
||||
_fields_ = [("PerProcessUserTimeLimit", ctypes.c_int64),
|
||||
("PerJobUserTimeLimit", ctypes.c_int64), ("LimitFlags", wintypes.DWORD),
|
||||
("MinimumWorkingSetSize", ctypes.c_size_t), ("MaximumWorkingSetSize", ctypes.c_size_t),
|
||||
("ActiveProcessLimit", wintypes.DWORD), ("Affinity", ctypes.c_size_t),
|
||||
("PriorityClass", wintypes.DWORD), ("SchedulingClass", wintypes.DWORD)]
|
||||
|
||||
class IOCounters(ctypes.Structure):
|
||||
_fields_ = [(name, ctypes.c_uint64) for name in
|
||||
("ReadOperationCount", "WriteOperationCount", "OtherOperationCount",
|
||||
"ReadTransferCount", "WriteTransferCount", "OtherTransferCount")]
|
||||
|
||||
class ExtendedLimits(ctypes.Structure):
|
||||
_fields_ = [("BasicLimitInformation", BasicLimits), ("IoInfo", IOCounters),
|
||||
("ProcessMemoryLimit", ctypes.c_size_t), ("JobMemoryLimit", ctypes.c_size_t),
|
||||
("PeakProcessMemoryUsed", ctypes.c_size_t), ("PeakJobMemoryUsed", ctypes.c_size_t)]
|
||||
|
||||
kernel = ctypes.WinDLL("kernel32", use_last_error=True)
|
||||
kernel.CreateJobObjectW.argtypes = [ctypes.c_void_p, wintypes.LPCWSTR]
|
||||
kernel.CreateJobObjectW.restype = wintypes.HANDLE
|
||||
kernel.SetInformationJobObject.argtypes = [wintypes.HANDLE, ctypes.c_int, ctypes.c_void_p, wintypes.DWORD]
|
||||
kernel.SetInformationJobObject.restype = wintypes.BOOL
|
||||
kernel.AssignProcessToJobObject.argtypes = [wintypes.HANDLE, wintypes.HANDLE]
|
||||
kernel.AssignProcessToJobObject.restype = wintypes.BOOL
|
||||
kernel.GetCurrentProcess.restype = wintypes.HANDLE
|
||||
kernel.CloseHandle.argtypes = [wintypes.HANDLE]
|
||||
|
||||
handle = kernel.CreateJobObjectW(None, None)
|
||||
if not handle:
|
||||
raise RuntimeError("Inspector resource containment could not be created")
|
||||
limits = ExtendedLimits()
|
||||
# KILL_ON_JOB_CLOSE | JOB_MEMORY | ACTIVE_PROCESS. No BREAKAWAY flags.
|
||||
limits.BasicLimitInformation.LimitFlags = 0x2000 | 0x200 | 0x8
|
||||
limits.BasicLimitInformation.ActiveProcessLimit = PROCESS_LIMIT
|
||||
limits.JobMemoryLimit = MEMORY_LIMIT
|
||||
if not kernel.SetInformationJobObject(handle, 9, ctypes.byref(limits), ctypes.sizeof(limits)):
|
||||
kernel.CloseHandle(handle)
|
||||
raise RuntimeError("Inspector resource containment limits could not be applied")
|
||||
if not kernel.AssignProcessToJobObject(handle, kernel.GetCurrentProcess()):
|
||||
kernel.CloseHandle(handle)
|
||||
raise RuntimeError("Inspector resource containment could not be attached")
|
||||
# Retain a non-inheritable owner handle until process exit. Do not close early:
|
||||
# KILL_ON_JOB_CLOSE also terminates the current inspector process.
|
||||
_job_handle = handle
|
||||
return {"active": True, "kind": "windows-job-v1", "memoryBytes": MEMORY_LIMIT,
|
||||
"maxProcesses": PROCESS_LIMIT, "killsDescendants": True}
|
||||
Reference in new issue
Block a user