Remove duplicate plain-text README and consolidate documentation
This commit is contained in:
StasonJatham committed
1 parent
8c3a90225f
commit
7ee77c3fc2
4 files changed
+17
-185
No files matched your search
@@ -129,7 +129,7 @@ repository. Examples under [examples/](examples/) use synthetic data.
|
||||
- Exit `2`: an error or incomplete processing; inspect `SUMMARY.txt` and
|
||||
`details/summary.json`. If the scan failed before presentation, inspect
|
||||
`summary.json` at the result root instead.
|
||||
- Missing years/timezones are not guessed. See [README.txt](README.txt) for time configuration.
|
||||
- Missing years/timezones are not guessed. See time configuration below.
|
||||
- Binary appliance logs may need an external decoder. Decode failures can also come from ordinary text.
|
||||
- Damaged inputs and regex timeouts are reported. Archive path traversal, unsafe links and exhausted quotas stop preparation.
|
||||
- Processing uses one worker, incremental reads and disk-backed SQLite. There is no hard process-wide RAM/CPU quota or automatic resume.
|
||||
@@ -138,6 +138,20 @@ repository. Examples under [examples/](examples/) use synthetic data.
|
||||
Tests were executed on macOS/ARM64. Windows and Linux instructions are provided,
|
||||
but those platforms have not yet been validated in the recorded test environment.
|
||||
|
||||
## Time configuration
|
||||
|
||||
Use known device settings when logs omit their year or timezone:
|
||||
|
||||
```sh
|
||||
netscaler-triage scan --input ADC-A=./logs --year 2026 --timezone ADC-A=UTC
|
||||
```
|
||||
|
||||
The appliance name, year and timezone above are examples. Explicit timestamps
|
||||
are retained; ambiguous daylight-saving times remain unresolved. Use
|
||||
`--time-config` for per-device settings and per-file year overrides; see
|
||||
[examples/time.json](examples/time.json). The effective assumptions are recorded
|
||||
in `details/configuration.json`.
|
||||
|
||||
## Help and development
|
||||
|
||||
```sh
|
||||
@@ -154,7 +168,7 @@ uv run ruff format --check src tests typing_stubs
|
||||
uv build
|
||||
```
|
||||
|
||||
See [README.txt](README.txt) for the detailed plain-text manual and
|
||||
See [RULES.txt](RULES.txt) for rule-pack semantics and
|
||||
[VALIDATION.txt](VALIDATION.txt) for test scope and known coverage limits.
|
||||
|
||||
## License
|
||||
|
||||
-181
@@ -1,181 +0,0 @@
|
||||
NETSCALER TRIAGE
|
||||
|
||||
Offline log triage for NetScaler ADC and Gateway.
|
||||
Give it a log folder or archive. It prepares the input, searches for suspicious
|
||||
patterns and public indicators, explains matching rules and decoded command text,
|
||||
finds update/version records, and links related events on the same appliance.
|
||||
Original files are not modified. Payloads are never executed or fetched.
|
||||
|
||||
INSTALL
|
||||
|
||||
Requires Python 3.14 or newer. From the unpacked source folder, on macOS/Linux:
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
|
||||
Or on Windows PowerShell:
|
||||
py -m venv .venv
|
||||
.venv\Scripts\Activate.ps1
|
||||
|
||||
Then install the local project:
|
||||
python -m pip install .
|
||||
|
||||
A built wheel can instead be installed with python -m pip install PATH_TO_WHEEL.
|
||||
For a locked developer installation, use uv sync --locked with a suitable
|
||||
installed Python interpreter. The lockfile records exact dependency versions.
|
||||
Installation may need internet access; analysis itself runs offline.
|
||||
|
||||
START HERE
|
||||
|
||||
netscaler-triage logs.7z
|
||||
|
||||
Or use an unpacked folder:
|
||||
netscaler-triage ./logs
|
||||
|
||||
Or run inside the folder containing your logs:
|
||||
netscaler-triage
|
||||
|
||||
ZIP, TAR, TAR.GZ/TGZ, GZIP and 7z are detected automatically, including nested
|
||||
archives. Unsupported formats are reported. Several inputs can be combined:
|
||||
netscaler-triage ./appliance-a ./appliance-b another-export.zip
|
||||
|
||||
The result folder is created automatically and printed when the run finishes.
|
||||
--output PATH chooses a new result folder. Existing results are never overwritten.
|
||||
Appliance groups are inferred from folder structure; review the preparation
|
||||
manifest when a combined export has an unusual layout.
|
||||
|
||||
For an encrypted archive, request a hidden password prompt:
|
||||
netscaler-triage logs.7z --password
|
||||
|
||||
--password VALUE also works, but may expose the password in shell history or
|
||||
process listings. The same password is used for nested archives. 7z, ZipCrypto
|
||||
and AES-ZIP are supported. Passwords are not saved in results.
|
||||
|
||||
WHAT TO READ FIRST
|
||||
|
||||
SUMMARY.txt is the readable overview: status, counts, limitations and next steps.
|
||||
FINDINGS.csv contains only rule/indicator matches, explanations, source references
|
||||
and indicator statuses. A match is not proof of execution or compromise.
|
||||
IOCS.csv contains raw search candidates and defanged display values, not a blocklist.
|
||||
|
||||
Technical exports are grouped under details/: findings.jsonl (all selected events,
|
||||
including context/errors), summary.json, updates.jsonl, correlations.jsonl,
|
||||
coverage.json, input_manifest.json, configuration.json, indicator_hits.csv,
|
||||
file_hash_hits.json, preparation_manifest.json and evidence.sqlite.
|
||||
Prepared input copies remain in inputs/. Preparation-manifest paths are relative
|
||||
to the result root. Decoding details retain originals, transformations and errors.
|
||||
|
||||
FINDINGS.csv prefixes spreadsheet-sensitive text with an apostrophe; exact values
|
||||
remain in JSON. Authorized-test/context/reference hits are explicitly labelled.
|
||||
Direct engine.scan calls and failed/interrupted CLI scans retain flat exports.
|
||||
|
||||
ADD YOUR OWN RULES OR IOCS
|
||||
|
||||
Bundled public references are loaded automatically. They are a dated, attributed
|
||||
collection, not a claim to cover every known attack. No private case addresses,
|
||||
appliance identities or environment-specific assumptions are included in defaults.
|
||||
References are not downloaded during a scan.
|
||||
|
||||
netscaler-triage ./logs --config examples/rules.yml
|
||||
netscaler-triage ./logs --ioc-file indicators.txt
|
||||
netscaler-triage ./logs --pattern "(?i)your_search_expression"
|
||||
|
||||
Repeat --config or --ioc-file to combine packs. An explicit --config directory
|
||||
loads supported packs recursively in sorted relative-path order; no private or
|
||||
customer directory is loaded unless you select it.
|
||||
|
||||
netscaler-triage ./logs --config ./my-rules
|
||||
Sigma YAML is the primary rule
|
||||
format, parsed with pySigma and evaluated locally with the supported subset in
|
||||
RULES.txt. Unsupported features are rejected explicitly. Defaults include 47
|
||||
Sigma rules and 246 deduplicated public reference indicators.
|
||||
THREAT_INTELLIGENCE.txt indexes all 52 sources, uses and limitations;
|
||||
config/threat_intel.json is the complete machine-readable inventory.
|
||||
|
||||
JSON, CSV and one reference per line in TXT remain supported. TXT infers clear
|
||||
IP addresses, SHA-256 hashes, HTTP(S) URLs and absolute Unix paths; other text
|
||||
stays literal. Explicit CSV/JSON types are unchanged. An exported IOCS.csv
|
||||
can be loaded again. Examples contain synthetic values, not real intelligence.
|
||||
Keep local case rules under private/, which is gitignored and excluded from
|
||||
release builds; load them explicitly with --config private/your_rules.yml.
|
||||
Private packs are never discovered or enabled automatically.
|
||||
Example files are included in the source release.
|
||||
Rule packs are data files. Python plugin code is not loaded from them.
|
||||
|
||||
TIME AND DEVICE IDENTITY
|
||||
|
||||
Explicit timestamps are retained. A missing year or timezone is not guessed.
|
||||
For known time settings and explicit appliance names on prepared folders:
|
||||
netscaler-triage scan --input ADC-A=./logs --year 2026 --timezone ADC-A=UTC
|
||||
|
||||
Use your actual device settings; UTC and the year above are examples only.
|
||||
Ambiguous daylight-saving times remain unresolved. Time assumptions and per-file
|
||||
year overrides can be supplied using --time-config; see examples/time.json.
|
||||
|
||||
LIMITS AND INCOMPLETE RESULTS
|
||||
|
||||
Exit 0 means the configured processing completed, not that the appliance is clean.
|
||||
Exit 2 means an error or incomplete processing; inspect SUMMARY.txt and
|
||||
details/summary.json. If the scan failed before presentation, inspect
|
||||
summary.json at the result root instead.
|
||||
Invalid decoding candidates may be ordinary text, not malicious payloads.
|
||||
Unknown text formats remain searchable, even if no timestamp is parsed.
|
||||
Proprietary binary logs require an appropriate external decoder.
|
||||
|
||||
Unreadable files, damaged archives and regex timeouts are recorded while other
|
||||
valid inputs continue. Path traversal, unsafe links and exhausted archive quotas
|
||||
stop preparation. A controlled interruption preserves the processed database
|
||||
where possible. There is no automatic resume after interruption.
|
||||
|
||||
A single worker reads lines incrementally. SQLite stores events and large sorts
|
||||
on disk; small decoding caches are bounded. One CPU core can be busy. This is not
|
||||
a hard process-wide RAM or CPU quota: archive metadata, large rule lists and file
|
||||
inventories need additional memory. Allow disk space for preparation and results.
|
||||
|
||||
Defaults: 2 GiB preparation budget, 100,000 archive entries, nesting depth 3,
|
||||
1 MiB per line and 512 MiB expanded per log file. Limits never mean a silent
|
||||
successful scan. Advanced options can raise them for legitimate larger inputs.
|
||||
|
||||
HELP AND OPTIONAL COMMANDS
|
||||
|
||||
netscaler-triage --help
|
||||
netscaler-triage scan --help-all
|
||||
netscaler-triage extract logs.7z prepared
|
||||
netscaler-triage decode payload.txt
|
||||
netscaler-triage indicators text.txt
|
||||
netscaler-triage inventory ./logs
|
||||
|
||||
Short help shows common options. Advanced help includes resource limits, time
|
||||
assumptions and --no-default-patterns, which disables built-in rules and bundled
|
||||
public references; explicitly supplied packs still apply.
|
||||
|
||||
DEVELOPMENT AND RELEASE
|
||||
|
||||
uv run python -W error -m unittest discover -s tests
|
||||
uv run mypy
|
||||
uv run ruff check src tests typing_stubs
|
||||
uv run ruff format --check src tests typing_stubs
|
||||
uv build
|
||||
|
||||
Synthetic tests cover parsing, archive safety, decoding, matching, correlations
|
||||
and packaging hygiene. VALIDATION.txt records what was actually tested. Runtime
|
||||
environments and analysis outputs are not release files. Source distributions
|
||||
use an explicit allowlist to avoid bundling unrelated documents or case settings.
|
||||
|
||||
LICENSE
|
||||
|
||||
Project code is licensed under the MIT License; see LICENSE.txt.
|
||||
Third-party dependencies and referenced external materials retain their own
|
||||
licenses. The project does not relicense original external publications.
|
||||
|
||||
VALIDATE PACKS BEFORE A SCAN
|
||||
|
||||
netscaler-triage rules --config ./my-rules
|
||||
|
||||
This offline command loads and compiles the bundled defaults and selected packs,
|
||||
then prints the Sigma rule, regex rule and explicit active indicator counts. It
|
||||
does not prepare or scan logs, contact external services or create artifacts.
|
||||
Use --no-default-patterns to validate only the selected files; --json gives a
|
||||
machine-readable result. Invalid Sigma, unsupported rules and conflicting rule
|
||||
identifiers fail validation. Syntax validation does not prove detection quality.
|
||||
Directory discovery skips README.txt, LICENSE.txt and CHANGELOG.txt documents
|
||||
(case-insensitive); an explicitly named TXT file still loads as references using TXT type inference.
|
||||
+1
-1
@@ -27,7 +27,7 @@ netscaler-triage = "netscaler_triage.cli:main"
|
||||
packages = ["src/netscaler_triage"]
|
||||
|
||||
[tool.hatch.build.targets.sdist]
|
||||
include = ["src/", "tests/", "typing_stubs/", "examples/", "config/threat_intel.json", "README.md", "README.txt", "RULES.txt", "VALIDATION.txt", "THREAT_INTELLIGENCE.txt", "LICENSE.txt", "uv.lock", ".python-version", ".gitignore", "pyproject.toml"]
|
||||
include = ["src/", "tests/", "typing_stubs/", "examples/", "config/threat_intel.json", "README.md", "RULES.txt", "VALIDATION.txt", "THREAT_INTELLIGENCE.txt", "LICENSE.txt", "uv.lock", ".python-version", ".gitignore", "pyproject.toml"]
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
|
||||
@@ -40,7 +40,6 @@ class ReleaseHygieneTests(unittest.TestCase):
|
||||
"tests/",
|
||||
"typing_stubs/",
|
||||
"examples/",
|
||||
"README.txt",
|
||||
"README.md",
|
||||
"RULES.txt",
|
||||
"VALIDATION.txt",
|
||||
|
||||
Reference in new issue
Block a user