ctcloud: CertSpotter-backed CT cloud-resource monitor

Single stateless Go process that consumes CT logs via embedded (patched)
CertSpotter internals, classifies cloud/database/storage DNS identifiers
in-process with a zero-allocation classifier, enriches via round-robin DNS
resolvers with CNAME following, and batch-writes useful findings to PostgreSQL.

- third_party/certspotter: 2-hunk IdentifierSink patch, no stdout/JSON/files
- internal/classify: AWS/Azure/GCP/Firebase/Cloudflare R2/DigitalOcean/Supabase/OCI
- internal/dns: sharded bounded TTL cache, multi-resolver load balancing + failover
- internal/pipeline: bounded queues, dedup, direct-priority load shedding
- internal/store: pgx batch writer, Postgres CT checkpoints (no cert contents)
- internal/probe: optional non-destructive reachability checks (off by default)
- Docker: distroless non-root, read-only rootfs, no app volume, resource limits
- tests enforce 0 allocs/op on the classifier and sink hot path
This commit is contained in:
StasonJatham committed 2026-09-22 13:13:10 +02:00
1 parent 9e78a114ac
commit e8f7d6bfdf
118 files changed
+16473

No files matched your search

+16
View File
@@ -0,0 +1,16 @@
# Copy to .env and edit. .env is gitignored.
POSTGRES_USER=ctcloud
POSTGRES_PASSWORD=change-me-strong
POSTGRES_DB=ctcloud
# Local compose (private bridge network, no TLS):
DATABASE_URL=postgres://ctcloud:change-me-strong@postgres:5432/ctcloud?sslmode=disable
# Production: use a TLS-enabled database instead, e.g.
# DATABASE_URL=postgres://user:[email protected]:5432/ctcloud?sslmode=verify-full
DNS_SERVERS=1.1.1.1:53,8.8.8.8:53,1.0.0.1:53
DNS_WORKERS=16
LOG_LEVEL=info
ACTIVE_PROBING=false