ctcloud: CertSpotter-backed CT cloud-resource monitor
Single stateless Go process that consumes CT logs via embedded (patched) CertSpotter internals, classifies cloud/database/storage DNS identifiers in-process with a zero-allocation classifier, enriches via round-robin DNS resolvers with CNAME following, and batch-writes useful findings to PostgreSQL. - third_party/certspotter: 2-hunk IdentifierSink patch, no stdout/JSON/files - internal/classify: AWS/Azure/GCP/Firebase/Cloudflare R2/DigitalOcean/Supabase/OCI - internal/dns: sharded bounded TTL cache, multi-resolver load balancing + failover - internal/pipeline: bounded queues, dedup, direct-priority load shedding - internal/store: pgx batch writer, Postgres CT checkpoints (no cert contents) - internal/probe: optional non-destructive reachability checks (off by default) - Docker: distroless non-root, read-only rootfs, no app volume, resource limits - tests enforce 0 allocs/op on the classifier and sink hot path
This commit is contained in:
1 parent
9e78a114ac
commit
e8f7d6bfdf
118 files changed
+16473
No files matched your search
+53
@@ -0,0 +1,53 @@
|
||||
# ctcloud patch to upstream CertSpotter
|
||||
|
||||
This directory is a copy of upstream CertSpotter
|
||||
(https://github.com/SSLMate/certspotter), pulled in as a local module and
|
||||
wired up with a `replace` directive in the root `go.mod`.
|
||||
|
||||
It is intentionally a copy (not a git submodule) so the patched tree lives in
|
||||
this repository and is compiled by the Docker build with no network access to a
|
||||
separate fork. To rebase on a newer upstream release:
|
||||
|
||||
1. Replace this directory with the new upstream tree.
|
||||
2. Re-apply the two hunks below.
|
||||
|
||||
## Changes (only two hunks)
|
||||
|
||||
### `monitor/config.go`
|
||||
|
||||
Adds an exported `IdentifierSink` interface and a `Config.IdentifierSink`
|
||||
field:
|
||||
|
||||
```go
|
||||
type IdentifierSink interface {
|
||||
HandleIdentifiers(ctx context.Context, identifiers *certspotter.Identifiers)
|
||||
}
|
||||
```
|
||||
|
||||
### `monitor/process.go`
|
||||
|
||||
In `processCertificate`, immediately after `certInfo.ParseIdentifiers()`, hand
|
||||
the parsed identifiers to the sink and return, bypassing the watchlist,
|
||||
certificate-chain retrieval and notification paths:
|
||||
|
||||
```go
|
||||
if config.IdentifierSink != nil {
|
||||
config.IdentifierSink.HandleIdentifiers(ctx, identifiers)
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
## Why
|
||||
|
||||
The sink gives ctcloud parsed DNS identifiers via a direct in-process Go call:
|
||||
|
||||
```text
|
||||
CT log entry -> CertSpotter parse -> IdentifierSink.HandleIdentifiers -> classifier
|
||||
```
|
||||
|
||||
No stdout, no JSON, no files, no hooks, no subprocess. Setting the sink also
|
||||
skips issuer-chain fetching and the watchlist machinery, which is exactly the
|
||||
I/O and serialization overhead the design forbids.
|
||||
|
||||
Upstream is otherwise unmodified. License and notices are preserved in
|
||||
`LICENSE`.
|
||||
Reference in new issue
Block a user