RULE PACKS AND IOC LISTS A pack is loaded with --config; repeat the option to combine packs. Matching is offline. YAML files are genuine Sigma detection rules parsed by official pySigma. The local evaluator supports the explicit subset below, not full Sigma semantics. JSON, CSV and TXT remain supported as described in LEGACY JSON PACKS below. Bundled data/public_sigma.yml provides 47 rules: 41 behavior rules and six IOC groups containing 246 attributed public references. These are not local findings. Use --no-default-patterns to disable bundled defaults. Private packs are opt-in; keep them in the ignored private/ directory and load them explicitly: netscaler-triage ./logs --config examples/rules.yml netscaler-triage ./logs --config private/rules.yml --config private/more-rules.yml SUPPORTED SIGMA SUBSET examples/rules.yml is a synthetic example with a UUID and provenance. Separate rules with ---. IDs, or titles when IDs are absent, must be unique within a pack. Use stable unique IDs across packs. Rules supply detections, not executable code. Required logsource product: netscaler. Optional service must equal the normalized event kind: http_access, http_error, syslog, shell_log or unknown. Omit service to include all kinds. Other products, categories, custom logsource attributes and custom taxonomies are rejected. A descriptive definition adds no filtering logic. Supported detection semantics: Keyword lists: substring matching against raw; list entries use OR. Field maps: fields use AND; values in a list normally use OR. Strings: case-insensitive; * and ? are Sigma wildcards with standard literal wildcard escaping. Field strings match the complete field value. Modifiers: contains, startswith, endswith, re and all, including combinations accepted by pySigma. all requires every listed value to match. Regex: search semantics; case-sensitive unless inline flags such as (?i) change it. The configured regex timeout applies. Null: matches a missing or null field. Numeric/Boolean comparisons preserve value types; integer 200 and string "200" differ. Conditions: and, or, not, parentheses, 1 of and all of selector patterns or them. A list of conditions is evaluated with OR. Unsupported rules fail at loading, without silent reinterpretation. Rejected constructs include aggregation, Sigma correlation rules, filters, collection actions, timeframe, backend pipelines, placeholders and modifiers such as cidr, exists, base64, fieldref or cased. This evaluator does not generate SIEM queries. The separate event correlation stage does not implement Sigma correlation rules. NORMALIZED FIELDS AND DECODED VIEWS Parsed fields are flattened, then top-level event metadata is overlaid. Names are case-insensitive. Common fields, available only when present in the input, are: raw, kind, appliance, source, line, timestamp source_ip, uri, user_agent, method, protocol, status, bytes, referer session_id, transaction_id, request_id, context, return_code, command_status sha256 source is the relative log filename. src/client_ip alias source_ip; _raw/message alias raw. HTTP status is an integer. sha256 is a list of hashes of successfully transformed payload bytes; it is not evidence of execution. List-valued string fields match if any item matches. Other field names can be matched when supplied by a caller, but are not automatically extracted. Decoded SHA-1 and MD5 fields are not calculated by the engine. Missing fields are not invented. Sigma is evaluated once per event against its original view and available successfully decoded text views. Decoded views replace raw only, never uri, user_agent, source_ip or other structured fields. A rule can therefore combine decoded content with the original request context and explicit exclusions. rule_details preserves ID/name/title/description/references/level/falsepositives, source and basis. raw identifies the original event view, including structured fields. For decoded:N, sigma_decoded_views[N] identifies the corresponding entry in the event's decoding list. Structured-only matches are not attributed to unrelated decoded views. A matching view is not evidence that a payload ran. Optional custom indicators metadata contains type/value/status plus provenance. It is retained only as provenance, not as unconditional IOC checks. To activate an independent indicator, provide an explicit CSV, TXT or JSON indicator pack. This prevents catalog values from bypassing a Sigma condition. Catalog arrays are excluded from each hit's custom_attributes to avoid repeated large exports. Metadata does not replace the actual Sigma detection/condition definition. LIMITS AND COVERAGE Per YAML file: 4 MiB UTF-8 input, at most 256 documents, bounded token count, 32 nesting levels and 10,000 nodes per compiled condition. Anchors, aliases, custom YAML tags and duplicate mapping keys are rejected. Unsupported rules cause a startup error. Per event: at most 64 decoded views and 1,048,576 characters per evaluated string field or decoded view, in addition to ingestion byte limits. Referenced list fields are limited to 10,000 elements. Unreferenced metadata is not evaluated. Sigma IDs must be unique across all selected packs. Timeouts and SigmaEvaluationLimit preserve the affected event, record incomplete matching, mark coverage partial and allow later records to be processed. They are neither negative findings nor proof of attacks. Custom metadata repeated per hit is limited to 16 KiB serialized JSON. Each rule may provide at most 10,000 catalog rows, each limited to 128 KiB; type/value strings are limited to 65,536 characters. Pack-size limits also apply. Sigma references: https://sigmahq.io/sigma-specification/specification/sigma-rules-specification.html https://sigmahq-pysigma.readthedocs.io/en/latest/reference/collection.html LEGACY JSON PACKS A UTF-8 JSON pack may contain patterns and indicators. Bundled defaults still apply unless explicitly disabled. The following address is documentation-only. Minimal example: { "patterns": [ { "id": "custom_download_command", "pattern": "(?i)\\b(?:curl|wget)\\s+https?://", "description": "A download command appears in the log. Verify its arguments, origin and whether it executed." } ], "indicators": [ {"type": "ipv4", "value": "192.0.2.42", "status": "reference_only"} ] } The address is a documentation example, not an attack indicator. A complete synthetic pack is available in examples/rules.json. PATTERNS patterns is a list of objects with pattern (a regex) and an optional stable id. description explains why a match may matter without claiming an attack succeeded. source/source_id/sources can identify provenance. Use distinct IDs across packs. Matched rule metadata is retained in rule_details, including matches after safe decoding. Keep rules specific: broad matches on ordinary IPs or login messages can create large, low-value result sets. Inline flags such as (?i) control case. Regex timeouts mean incomplete checks, not negative results. INDICATORS indicators is a list with type, value and optional status and source. Common types: ipv4, ipv6, domain, url, path, sha256 and literal. Use original values, not defanged display strings. IP boundaries are checked. SHA-256 values are checked against original file bytes and successfully decoded bytes, as well as log text. Other hash-shaped text can be extracted; file hashing computes SHA-256 only. Status records context; it does not suppress the source event: reference_only Published or supplied reference, not a local verdict authorized_test An explicitly identified permitted test context_only Environment/context value, not automatically hostile Document sources in a top-level sources list with id, title and url. Preserve dates, attribution and uncertainty. A reference match is a lead; establish its local role from the actual log fields. OTHER INPUT FORMATS TXT: one reference per line; blank lines and lines starting with # are ignored. Unambiguous IPv4/IPv6 addresses, 64-digit hexadecimal SHA-256 hashes, HTTP(S) URLs and whitespace-free absolute Unix paths receive their matching type. IP boundaries then prevent 192.0.2.1 from matching 192.0.2.10. Other text, including multiword commands, stays literal. CSV/JSON explicit types remain unchanged. CSV: type,value,status, or indicator_type,indicator,status. The tool's IOCS.csv output can be used directly. A legacy ip column is also accepted. Legacy rule CSV: ID,Muster, where Muster contains the regex expression. JSON: the pack object above, or a list of pattern objects. TIME CONFIGURATION --time-config accepts JSON with year, timezones and file_years. timezones maps explicit appliance labels to IANA timezones. file_years maps APPLIANCE/relative/log to a year when a specific yearless file needs an override. Example: { "year": 2026, "timezones": {"ADC-A": "UTC"}, "file_years": {"ADC-A/messages.previous": 2025} } These values are examples, not inferred settings. Explicit timestamps take precedence. Choose assumptions based on the actual evidence. PUBLIC INTELLIGENCE PROVENANCE The bundled threat_intel.json is the canonical inventory. Indicators are unique by type and normalized value, with stable IDs. source_ids/source_urls enumerate all reporting sources; observations retain each source's role, confidence, dates, original status and any MISP direction/to_ids conflicts. The source field can contain multiple source IDs after deduplication. Reference-only matching does not turn shared infrastructure, scanner markers or historical IOCs into confirmed attackers. context_only and observation_category retain those distinctions. Sources list retrieval attempts, requested URLs, access limitations and linked indicator/rule IDs. Excluded observables and external whole-file YARA libraries remain inventoried with reasons, without claiming equivalent Sigma coverage. THREAT_INTELLIGENCE.txt provides the readable index. Percent decoding, Base64 (including padding-free candidates) and nested layers feed both IOC and Sigma matching. Raw records remain unchanged. The candidate gate uses encoding evidence; arbitrary obfuscation is not exhaustively decoded. A plain indicator match, a decoded byte hash and a source-file hash are different evidence types. Do not interpret a textual pathname as proof that a file exists. ADDING RULES WITHOUT CHANGING CODE Create a directory such as my-rules and put Sigma YAML, JSON, CSV or TXT packs inside it. Load it explicitly: netscaler-triage ./logs --config ./my-rules Repeat --config for additional files or directories. Directory discovery is recursive and sorted by relative filename; repeated identical files load once. Hidden entries, __pycache__ and node_modules directories are ignored. Symlinks are rejected. Unsupported file extensions inside a directory are ignored. An empty directory or a malformed supported pack produces a named error. Discovery is bounded to 64 files, 4096 directories and 32 nested levels. Each Sigma file is limited to 4 MiB; other formats to 16 MiB; the total is 32 MiB. No customer or private pack is activated automatically. For exact IP or path references, create my-rules/local.csv: type,value,status,source,description ipv4,192.0.2.9,authorized_test,local,Approved external test path,/example/suspicious-file,reference_only,local,Investigate file context The addresses and paths above are synthetic examples. Extra CSV columns such as source, description, confidence and evidence are preserved verbatim. This also applies to legacy ID,Muster regex CSV files. Use JSON when metadata requires structured lists or nested objects. Keep reference values in their original form rather than defanged display form; matching does not establish compromise. VALIDATE PACKS BEFORE A SCAN netscaler-triage rules --config ./my-rules This offline command loads and compiles the bundled defaults and selected packs, then prints the Sigma rule, regex rule and explicit active indicator counts. It does not prepare or scan logs, contact external services or create artifacts. Use --no-default-patterns to validate only the selected files; --json gives a machine-readable result. Invalid Sigma, unsupported rules and conflicting rule identifiers fail validation. Syntax validation does not prove detection quality. Directory discovery skips README.txt, LICENSE.txt and CHANGELOG.txt documents (case-insensitive); an explicitly named TXT file still loads as references using TXT type inference. TYPED INDICATOR MATCHING IP addresses and hashes require complete token boundaries. Domain indicators also match subdomains, but not concatenated names or unrelated suffixes. Paths are case-sensitive; a trailing slash denotes a directory prefix. URL schemes and authorities are case-insensitive, while paths and queries are exact. Typed indicators prevent incidental substrings from becoming matches. Use an explicit regex rule when different matching semantics are needed. A reference match or an authentication failure is not automatically malicious; preserve status and provenance and evaluate the surrounding event.