safetynet/docs/development-collector-retention.md

Protected development collector retention

The Windows development collector retains encrypted OTLP batches in bounded segments. This receiver is for debugging; its retention must never be used as the retention policy for security audit or forensic evidence.

The AES-256-GCM envelope, safetynet-debug-v1 associated data and DPAPI-bound key are unchanged. The collector directory grants access to the current Windows user and generated files inherit that restriction. No plaintext archive is created.

By default the active segment is at most 50 MiB. The collector retains two generated rolling retired segments, each at most 50 MiB. Before an append, it checks the complete encoded line's byte length and rotates if necessary. An individually oversized batch is rejected with HTTP 507. Storage or retention failures also return 507; successful export counters advance only after the encrypted append is flushed.

The first previous active archive is moved to signals.legacy.encrypted.ndjson. This segment is pinned and never automatically deleted, including after restart. Existing developer data therefore survives migration. The disk budget is the pinned legacy size plus 150 MiB of rolling data; normally this is at most 200 MiB. The pinned size can exceed 50 MiB if the older collector previously overshot its cap. Any manual disposal of the pinned segment requires a separately verified protected backup and an explicit operator decision.

Generated rolling segments are named signals.rolling.<timestamp>.<uuid>.encrypted.ndjson; oldest generated segments are removed once retention is exceeded. Directory ancestors, entries and mutation targets are checked for symlinks, junction resolution, non-files and hardlinks. Unexpected entries make storage fail closed, rather than permitting deletion of an unrelated file. This protects accidental path substitution; an administrator or process already controlling the same Windows account remains outside this developer receiver's threat boundary.

/health and /api/summary expose active (storedBytes/activeBytes), pinned (legacyBytes) and retired bytes, limits, rotations, dropped segment/byte counts and rejected writes. Request/signal counters and these operational counters start anew per process; retained files persist. Recent trace lookup remains a bounded in-memory view of this process and does not rehydrate retired batches.

Optional development configuration:

• DEBUG_COLLECTOR_DIRECTORY: isolated receiver directory, default data/observability.
• DEBUG_COLLECTOR_SEGMENT_BYTES: per-segment limit, default 52428800; minimum 1024.
• DEBUG_COLLECTOR_RETIRED_SEGMENTS: generated retired segment count, default 2; range 0–10.
• DEBUG_COLLECTOR_PORT: loopback port, default 4318.

The tests use a separate temporary owner-restricted Windows directory and alternate loopback port. They demonstrate real encrypted HTTP exports, rotation, retirement, preservation of the pinned archive, absent plaintext markers and HTTP 507 for oversized batches. They do not restart the live development collector.

Diagnostic indexing is validated before encryption or append: nested signal arrays, span identifiers, unsigned 64-bit timestamp strings and nonnegative durations within JavaScript's exact integer range. Empty SDK batches are accepted. Invalid batches return HTTP 400 without changing files, export counters or recent spans. The actual HTTP test confirms these invariants, then verifies a valid span's measured duration through trace lookup.