134 lines
7.3 KiB
Plaintext
134 lines
7.3 KiB
Plaintext
VALIDATION SCOPE
|
|
|
|
CPython 3.14.8 on macOS/ARM64. Windows and Linux were not executed in this
|
|
validation environment. Synthetic path checks do not replace target-OS tests.
|
|
|
|
CODE AUDIT
|
|
|
|
134 tests passed with Python warnings treated as errors in 36.475 seconds.
|
|
Strict mypy passed for 35 source files. Ruff lint and format checks passed for
|
|
36 files. The enabled
|
|
lint families are E, F, I, UP, B, SIM, RUF and PERF; this is not a claim that every
|
|
optional rule from every analyzer is enabled.
|
|
|
|
Reproduce from the source directory:
|
|
uv sync --locked
|
|
uv run python -W error -m unittest discover -s tests
|
|
uv run mypy
|
|
uv run ruff check src tests typing_stubs
|
|
uv run ruff format --check src tests typing_stubs
|
|
|
|
Tests cover archive preparation, encrypted and nested archives, original file
|
|
preservation, multi-appliance parsing, time ambiguity, update evidence, bounded
|
|
decoding, defanging, rule packs, correlations, incomplete results, interruption
|
|
and release hygiene. Sigma tests exercise supported semantics, decoded views,
|
|
explicit unsupported-feature rejection and local matching limits.
|
|
|
|
A separate sequential CLI matrix passed all 21 invocations with warnings fatal:
|
|
common and advanced help, scan/extract/decode/indicators/inventory, JSON output,
|
|
implicit folder and archive inputs, current-folder discovery of multiple
|
|
archives, encrypted 7z input, URL decoding and Base64 decoding.
|
|
|
|
RELEASE INSTALLATION
|
|
|
|
A wheel and source distribution were built. Locked runtime requirements were
|
|
exported with hashes and installed into a fresh virtual environment outside the
|
|
source tree, followed by the wheel without dependency resolution. Installed-package
|
|
verification is repeated after changes to code or bundled intelligence. Source
|
|
metadata is included so the packaging allowlist test remains active.
|
|
CLI help, bundled Sigma matching and the 246-entry public IOC catalog passed.
|
|
|
|
The wheel includes 47 public Sigma rules and the attributed public reference
|
|
catalog. Private packs are opt-in, gitignored and excluded from both release
|
|
formats. Artifact inventories and text contents were checked for private
|
|
case indicators, identities, home paths, logs, evidence and runtime directories.
|
|
Tests and examples use synthetic documentation addresses.
|
|
|
|
Runtime versions checked: pySigma 1.5.1, regex 2026.9.29, PyYAML 6.0.3,
|
|
py7zr 1.1.3, pyzipper 0.4.0, rich 15.0.0, rich-argparse 1.8.0, tzdata 2026.4.
|
|
The lockfile records all dependency versions.
|
|
|
|
INTERPRETING RESULTS
|
|
|
|
No private incident data is required to run these software tests. Independent
|
|
real-dataset findings are retained in separate customer case folders in their
|
|
BERICHT.txt reports, outside this shared tool and its release.
|
|
Original inputs are hashed before and after analysis. Forensic coverage limits
|
|
are retained even when software tests and code checks pass: binary logs,
|
|
ambiguous timestamps, invalid UTF-8 and failed decoding are not silently fixed.
|
|
|
|
A match, decoded command or correlation is not proof of execution or compromise.
|
|
Passing software tests does not establish investigation completeness. The local
|
|
Sigma evaluator implements the subset documented in RULES.txt, not every Sigma
|
|
feature or a Splunk query exporter.
|
|
|
|
Scanning is sequential with buffered input/output, bounded decoding caches and
|
|
an approximately 8 MiB SQLite page cache. File inventory, archive metadata,
|
|
custom rule packs and result volume still affect total process memory. There
|
|
is no hard process-wide CPU or RAM quota and no automatic parallel worker pool.
|
|
|
|
INTELLIGENCE REFRESH CHECKS
|
|
|
|
All 37 requested public URLs were retrieved or attempted and recorded, including
|
|
access errors, application-only responses and verified alternate source URLs.
|
|
The inventory has 52 sources, 246 unique indicators and 41 behavior patterns,
|
|
expressed as 47 Sigma rules. All source/indicator/rule joins are checked.
|
|
|
|
Every catalog indicator has a literal positive test; 245 have raw and decoded
|
|
Sigma positives. The legitimate customsnmpd pathname is deliberately excluded
|
|
from standalone Sigma matching and retains its contextual catalog occurrence.
|
|
All IPv4 values are checked against prefix/suffix overmatching. HTTP tests cover
|
|
percent encoding, Base64, no padding, nesting, Referer and additional headers.
|
|
Original raw text and decoding spans remain verified.
|
|
|
|
The preceding frozen baseline completed two independent real datasets: 2,405
|
|
files and 4,043,139 records, with every original/reference hash preserved. That
|
|
full run predates this intelligence refresh and decoding-gate change. Its counts
|
|
and timings must not be presented as a full-run test of the updated rule pack.
|
|
Updated bounded real-data findings are retained in the corresponding customer
|
|
BERICHT.txt reports. Temporary verification outputs are not required runtime inputs.
|
|
|
|
A separate frozen-baseline, warm-cache repeated-sample experiment used the same
|
|
30,720 records per mode: one/two/four processes took 12.555/6.230/3.913 seconds.
|
|
All twelve per-job outputs were semantically equivalent. Four processes used a
|
|
conservative aggregate peak sum of 262.9 MiB. This is a throughput experiment,
|
|
not an implemented parallel scanner, hard memory cap or full-archive forecast.
|
|
|
|
The final gate also checks every public IOC encoded as padded and unpadded
|
|
Base64, including all-uppercase tokens and four-character encodings. Canonical
|
|
short/full-field decoding requires a printable UTF-8 roundtrip and is bounded
|
|
to 4-4096 characters. This does not attempt to decode every ordinary word.
|
|
|
|
PREVIOUS INSTALLATION AND RUNTIME SMOKE CHECK
|
|
|
|
The previous installed wheel passed all 106 tests with warnings treated as errors
|
|
in 28.839 seconds in a fresh environment outside the source tree. Locked
|
|
runtime installation used 33 dependencies. CLI help, default Sigma matching and
|
|
uppercase-only Base64 decoding passed. After relocating the interpreter into
|
|
the shared tool directory, the same 106 tests passed in 29.001 seconds and CLI
|
|
help passed again. Python imports and module paths no longer depend on an
|
|
obsolete analysis checkout. These checks validate software operation, not
|
|
customer-specific compromise conclusions.
|
|
|
|
EXTENSIBLE RULE PACK REGRESSION
|
|
|
|
The current source passes 134 tests, including recursive pack loading, offline
|
|
validation, typed indicator boundaries, overlapping references, contextual Sigma
|
|
conditions, numeric selectors, limits and encoded command probes. Catalog
|
|
metadata cannot bypass a Sigma condition. The release was rebuilt after these
|
|
changes. The older installed-wheel result above is not a test of this revision.
|
|
|
|
A bounded private regression verified 19,625 original referenced log lines from
|
|
two separate investigations, including all 94 recorded finding families in the
|
|
earlier investigation. Public-rule matching completed without matching errors.
|
|
The case-IOC run recorded one literal matching timeout; an independent retry of
|
|
that original record completed with a finding and no matching error. The initial
|
|
partial coverage remains recorded; decoding-candidate errors are reported
|
|
separately and are not silently treated as successful decoding.
|
|
|
|
The bounded regression is not a repeat of the earlier full 4,043,139-record run.
|
|
A subsequent Unicode literal-lookup guard has synthetic regression coverage,
|
|
but the entire private fixture set was not rerun after that guard. These checks
|
|
validate known examples and important negative cases, not zero false positives
|
|
or complete detection of all future attacks. Private evidence is not shipped.
|