NetScaler_forensik/RULES.txt

RULE PACKS AND IOC LISTS

A pack is loaded with --config; repeat the option to combine packs. Matching is
offline. YAML files are genuine Sigma detection rules parsed by official pySigma.
The local evaluator supports the explicit subset below, not full Sigma semantics.
JSON, CSV and TXT remain supported as described in LEGACY JSON PACKS below.

Bundled data/public_sigma.yml provides 47 rules: 41 behavior rules and six IOC
groups containing 246 attributed public references. These are not local findings.
Use --no-default-patterns to disable bundled defaults. Private packs are opt-in;
keep them in the ignored private/ directory and load them explicitly:

netscaler-triage ./logs --config examples/rules.yml
netscaler-triage ./logs --config private/rules.yml --config private/more-rules.yml

SUPPORTED SIGMA SUBSET

examples/rules.yml is a synthetic example with a UUID and provenance. Separate
rules with ---. IDs, or titles when IDs are absent, must be unique within a pack.
Use stable unique IDs across packs. Rules supply detections, not executable code.

Required logsource product: netscaler. Optional service must equal the normalized
event kind: http_access, http_error, syslog, shell_log or unknown. Omit service to
include all kinds. Other products, categories, custom logsource attributes and
custom taxonomies are rejected. A descriptive definition adds no filtering logic.

Supported detection semantics:
    Keyword lists: substring matching against raw; list entries use OR.
    Field maps: fields use AND; values in a list normally use OR.
    Strings: case-insensitive; * and ? are Sigma wildcards with standard literal
    wildcard escaping. Field strings match the complete field value.
    Modifiers: contains, startswith, endswith, re and all, including combinations
    accepted by pySigma. all requires every listed value to match.
    Regex: search semantics; case-sensitive unless inline flags such as (?i)
    change it. The configured regex timeout applies.
    Null: matches a missing or null field. Numeric/Boolean comparisons preserve
    value types; integer 200 and string "200" differ.
    Conditions: and, or, not, parentheses, 1 of and all of selector patterns or
    them. A list of conditions is evaluated with OR.

Unsupported rules fail at loading, without silent reinterpretation. Rejected
constructs include aggregation, Sigma correlation rules, filters, collection
actions, timeframe, backend pipelines, placeholders and modifiers such as cidr,
exists, base64, fieldref or cased. This evaluator does not generate SIEM queries.
The separate event correlation stage does not implement Sigma correlation rules.

NORMALIZED FIELDS AND DECODED VIEWS

Parsed fields are flattened, then top-level event metadata is overlaid. Names are
case-insensitive. Common fields, available only when present in the input, are:
    raw, kind, appliance, source, line, timestamp
    source_ip, uri, user_agent, method, protocol, status, bytes, referer
    session_id, transaction_id, request_id, context, return_code, command_status
    sha256

source is the relative log filename. src/client_ip alias source_ip; _raw/message
alias raw. HTTP status is an integer. sha256 is a list of hashes of successfully
transformed payload bytes; it is not evidence of execution. List-valued string
fields match if any item matches. Other field names can be matched when supplied
by a caller, but are not automatically extracted. Decoded SHA-1 and MD5 fields
are not calculated by the engine. Missing fields are not invented.

Sigma is evaluated once per event against its original view and available
successfully decoded text views. Decoded views replace raw only, never uri,
user_agent, source_ip or other structured fields. A rule can therefore combine
decoded content with the original request context and explicit exclusions.

rule_details preserves ID/name/title/description/references/level/falsepositives,
source and basis. raw identifies the original event view, including structured
fields. For decoded:N, sigma_decoded_views[N] identifies the corresponding entry
in the event's decoding list. Structured-only matches are not attributed to
unrelated decoded views. A matching view is not evidence that a payload ran.

Optional custom indicators metadata contains type/value/status plus provenance.
It is retained only as provenance, not as unconditional IOC checks. To activate
an independent indicator, provide an explicit CSV, TXT or JSON indicator pack.
This prevents catalog values from bypassing a Sigma condition. Catalog arrays
are excluded from each hit's custom_attributes to avoid repeated large exports.
Metadata does not replace the actual Sigma detection/condition definition.

LIMITS AND COVERAGE

Per YAML file: 4 MiB UTF-8 input, at most 256 documents, bounded token count,
32 nesting levels and 10,000 nodes per compiled condition. Anchors, aliases,
custom YAML tags and duplicate mapping keys are rejected. Unsupported rules
cause a startup error. Per event: at most 64 decoded views and 1,048,576 characters
per evaluated string field or decoded view, in addition to ingestion byte limits.
Referenced list fields are limited to 10,000 elements. Unreferenced metadata is
not evaluated. Sigma IDs must be unique across all selected packs.
Timeouts and SigmaEvaluationLimit preserve the affected event, record incomplete
matching, mark coverage partial and allow later records to be processed. They are
neither negative findings nor proof of attacks.

Custom metadata repeated per hit is limited to 16 KiB serialized JSON. Each rule
may provide at most 10,000 catalog rows, each limited to 128 KiB; type/value strings
are limited to 65,536 characters. Pack-size limits also apply.

Sigma references:
https://sigmahq.io/sigma-specification/specification/sigma-rules-specification.html
https://sigmahq-pysigma.readthedocs.io/en/latest/reference/collection.html

LEGACY JSON PACKS

A UTF-8 JSON pack may contain patterns and indicators. Bundled defaults still
apply unless explicitly disabled. The following address is documentation-only.

Minimal example:

{
  "patterns": [
    {
      "id": "custom_download_command",
      "pattern": "(?i)\\b(?:curl|wget)\\s+https?://",
      "description": "A download command appears in the log. Verify its arguments, origin and whether it executed."
    }
  ],
  "indicators": [
    {"type": "ipv4", "value": "192.0.2.42", "status": "reference_only"}
  ]
}

The address is a documentation example, not an attack indicator.
A complete synthetic pack is available in examples/rules.json.

PATTERNS

patterns is a list of objects with pattern (a regex) and an optional stable id.
description explains why a match may matter without claiming an attack succeeded.
source/source_id/sources can identify provenance. Use distinct IDs across packs.
Matched rule metadata is retained in rule_details, including matches after safe
decoding. Keep rules specific: broad matches on ordinary IPs or login messages
can create large, low-value result sets. Inline flags such as (?i) control case.
Regex timeouts mean incomplete checks, not negative results.

INDICATORS

indicators is a list with type, value and optional status and source.
Common types: ipv4, ipv6, domain, url, path, sha256 and literal. Use original values,
not defanged display strings. IP boundaries are checked. SHA-256 values are checked
against original file bytes and successfully decoded bytes, as well as log text.
Other hash-shaped text can be extracted; file hashing computes SHA-256 only.

Status records context; it does not suppress the source event:
    reference_only       Published or supplied reference, not a local verdict
    authorized_test      An explicitly identified permitted test
    context_only         Environment/context value, not automatically hostile

Document sources in a top-level sources list with id, title and url. Preserve
dates, attribution and uncertainty. A reference match is a lead; establish its
local role from the actual log fields.

OTHER INPUT FORMATS

TXT: one reference per line; blank lines and lines starting with # are ignored.
Unambiguous IPv4/IPv6 addresses, 64-digit hexadecimal SHA-256 hashes, HTTP(S)
URLs and whitespace-free absolute Unix paths receive their matching type. IP
boundaries then prevent 192.0.2.1 from matching 192.0.2.10. Other text, including
multiword commands, stays literal. CSV/JSON explicit types remain unchanged.
CSV: type,value,status, or indicator_type,indicator,status. The tool's IOCS.csv
output can be used directly. A legacy ip column is also accepted.
Legacy rule CSV: ID,Muster, where Muster contains the regex expression.
JSON: the pack object above, or a list of pattern objects.

TIME CONFIGURATION

--time-config accepts JSON with year, timezones and file_years. timezones maps
explicit appliance labels to IANA timezones. file_years maps APPLIANCE/relative/log
to a year when a specific yearless file needs an override. Example:

{
  "year": 2026,
  "timezones": {"ADC-A": "UTC"},
  "file_years": {"ADC-A/messages.previous": 2025}
}

These values are examples, not inferred settings. Explicit timestamps take
precedence. Choose assumptions based on the actual evidence.

PUBLIC INTELLIGENCE PROVENANCE

The bundled threat_intel.json is the canonical inventory. Indicators are unique
by type and normalized value, with stable IDs. source_ids/source_urls enumerate
all reporting sources; observations retain each source's role, confidence, dates,
original status and any MISP direction/to_ids conflicts. The source field can
contain multiple source IDs after deduplication. Reference-only matching does
not turn shared infrastructure, scanner markers or historical IOCs into confirmed
attackers. context_only and observation_category retain those distinctions.

Sources list retrieval attempts, requested URLs, access limitations and linked
indicator/rule IDs. Excluded observables and external whole-file YARA libraries
remain inventoried with reasons, without claiming equivalent Sigma coverage.
THREAT_INTELLIGENCE.txt provides the readable index.

Percent decoding, Base64 (including padding-free candidates) and nested layers
feed both IOC and Sigma matching. Raw records remain unchanged. The candidate
gate uses encoding evidence; arbitrary obfuscation is not exhaustively decoded.
A plain indicator match, a decoded byte hash and a source-file hash are different
evidence types. Do not interpret a textual pathname as proof that a file exists.

ADDING RULES WITHOUT CHANGING CODE

Create a directory such as my-rules and put Sigma YAML, JSON, CSV or TXT packs
inside it. Load it explicitly:
    netscaler-triage ./logs --config ./my-rules
Repeat --config for additional files or directories. Directory discovery is
recursive and sorted by relative filename; repeated identical files load once.
Hidden entries, __pycache__ and node_modules directories are ignored. Symlinks
are rejected. Unsupported file extensions inside a directory are ignored.
An empty directory or a malformed supported pack produces a named error.
Discovery is bounded to 64 files, 4096 directories and 32 nested levels. Each
Sigma file is limited to 4 MiB; other formats to 16 MiB; the total is 32 MiB.
No customer or private pack is activated automatically.

For exact IP or path references, create my-rules/local.csv:
    type,value,status,source,description
    ipv4,192.0.2.9,authorized_test,local,Approved external test
    path,/example/suspicious-file,reference_only,local,Investigate file context

The addresses and paths above are synthetic examples. Extra CSV columns such as
source, description, confidence and evidence are preserved verbatim. This also
applies to legacy ID,Muster regex CSV files. Use JSON when metadata requires
structured lists or nested objects. Keep reference values in their original
form rather than defanged display form; matching does not establish compromise.

VALIDATE PACKS BEFORE A SCAN

    netscaler-triage rules --config ./my-rules

This offline command loads and compiles the bundled defaults and selected packs,
then prints the Sigma rule, regex rule and explicit active indicator counts. It
does not prepare or scan logs, contact external services or create artifacts.
Use --no-default-patterns to validate only the selected files; --json gives a
machine-readable result. Invalid Sigma, unsupported rules and conflicting rule
identifiers fail validation. Syntax validation does not prove detection quality.
Directory discovery skips README.txt, LICENSE.txt and CHANGELOG.txt documents
(case-insensitive); an explicitly named TXT file still loads as references using TXT type inference.

TYPED INDICATOR MATCHING

IP addresses and hashes require complete token boundaries. Domain indicators
also match subdomains, but not concatenated names or unrelated suffixes. Paths
are case-sensitive; a trailing slash denotes a directory prefix. URL schemes
and authorities are case-insensitive, while paths and queries are exact.
Typed indicators prevent incidental substrings from becoming matches. Use an
explicit regex rule when different matching semantics are needed. A reference
match or an authentication failure is not automatically malicious; preserve
status and provenance and evaluate the surrounding event.