- Store: skip identical CT checkpoint writes (CertSpotter calls StoreLogState
per batch across 64 logs; hashing the state avoids redundant writes).
- webui: cache /api/stats for 10s and raise API query timeouts; UI polls, so
repeated aggregate queries no longer queue behind checkpoint bursts.
- reword discovery detail (no 'name enumeration' for S3/GCS; only Azure uses a
container list).
Azure has two levels: the account maps to DNS (<account>.blob.core.windows.net)
and anonymous access is per container with no anonymous container enumeration.
Discovery now tests incoming labels as Azure account names (DNS existence) and,
for accounts that exist, probes a small set of common containers, indexing any
that list anonymously. CONTENT still never downloaded.
Also document that docker compose down -v wipes all discovered assets; use
stop/start to keep the pgdata volume.
Live: connectsystem.blob.core.windows.net indexed (33 objects).
Remove all fabricated candidate names (suffixes and the wordlist). Discovery now
only tests names that actually occur in the incoming hostname: the subdomain
label and the registrable-domain label (assets.example.com -> assets, example).
Nothing is permuted or drawn from a wordlist.
CMS SANs are custom domains, not bucket names, so matching SANs finds almost
no buckets. This adds internal/discover: from each non-matching hostname it
derives candidate names (domain label + subdomain label + ~40 suffixes) plus a
built-in wordlist, and anonymously tests listing on S3, GCS and (optionally)
DigitalOcean Spaces. Every 200 is indexed (keys/sizes/dates only).
Bounded hard: global BUCKET_SCAN_QPS rate limit, 12h per-name dedup, random
sampled domain queue, MaxPerDomain cap. Never downloads contents.
Fix: GCS candidates now use the virtual-host form
(<bucket>.storage.googleapis.com) so buckets no longer collide on the
cloud_assets unique key; previously they all shared storage.googleapis.com and
overwrote each other.
Live: 30 open buckets in ~5 min (e.g. estimator-dev, image-prod,
update-media, mos-images S3; several GCS buckets), 11k+ objects indexed.
StasonJatham