Bucketeer UI (dark mode, pagination, virtual scroll), readable+indexed resources, LLM-friendly logs

Logging:
- structured slog with LOG_FORMAT=json|text and a stable key vocabulary
- periodic stats record with queue depths, db pool, goroutines, memory, dns/probe
- no per-certificate logging (hot path is counters only)

UI (Bucketeer branding, DESIGN.md):
- light+dark themes (OS default, persisted header toggle, no first-paint flash)
- official mascot: lossless WebP light/dark logos + favicon, embedded and served
  with cache headers (1.2MB source PNGs -> ~27/22KB webp)
- Assets/Contents views, pagination (offset) and native virtual scrolling
  (content-visibility) so large result sets do not blow up the DOM
- search with debounce/abort + URL state, exposure chips, sort, summary counts

Readability + indexing:
- internal/index listers for S3-compatible, GCS, Azure, Firebase, OpenSearch
  (metadata only: keys/sizes/dates; never object contents)
- asset_entries table + store write/search; readable resources are indexed and
  searchable in the UI
- deep, unauthenticated, read-only DB probes (PROBE_DEEP): Redis PING,
  PostgreSQL StartupMessage, MongoDB OP_MSG listDatabases; OpenSearch/Neptune
  HTTP reads
- Azure account-level listing fixed: 403 is reported reachable, not
  auth_required, since anonymous access is per-container

DNS:
- curated reliable resolvers by default; DNS_SERVERS=fresh adds the compiled-in
  Fresh-Resolvers snapshot with a bounded startup sample (warming all 6.7k at
  once broke the container resolver)
- random healthy selection + failover + per-server cooldown

Other: prober Close fixes a shutdown hang; config/README updated.
This commit is contained in:
StasonJatham committed 2026-09-22 14:33:03 +02:00
1 parent 8fce55f8c9
commit a366905a43
28 files changed
+9744 -241

No files matched your search

+9 -1
View File
@@ -10,12 +10,20 @@ DATABASE_URL=postgres://ctcloud:change-me-strong@postgres:5432/ctcloud?sslmode=d
# Production: use a TLS-enabled database instead, e.g.
# DATABASE_URL=postgres://user:[email protected]:5432/ctcloud?sslmode=verify-full
DNS_SERVERS=1.1.1.1:53,8.8.8.8:53,1.0.0.1:53
# DNS_SERVERS unset = ~20 curated public resolvers. Options:
# DNS_SERVERS=fresh curated + compiled-in Fresh-Resolvers (sampled)
# DNS_SERVERS=system use the OS resolver
# DNS_SERVERS=1.1.1.1,8.8.8.8 your own comma-separated list
# DNS_SERVERS=system
DNS_WORKERS=16
DNS_CACHE_SIZE=262144
DEDUP_SIZE=262144
LOG_LEVEL=info
LOG_FORMAT=text # json for log pipelines / LLM analysis
ACTIVE_PROBING=false
PROBE_DEEP=true # unauthenticated read-only DB protocol checks
INDEX_ENABLED=true # index metadata of readable resources
INDEX_MAX_OBJECTS=1000
# Optional HTTP basic auth for the web UI (/ and /api/*):
# UI_USER=admin