Bucketeer UI (dark mode, pagination, virtual scroll), readable+indexed resources, LLM-friendly logs

Logging:
- structured slog with LOG_FORMAT=json|text and a stable key vocabulary
- periodic stats record with queue depths, db pool, goroutines, memory, dns/probe
- no per-certificate logging (hot path is counters only)

UI (Bucketeer branding, DESIGN.md):
- light+dark themes (OS default, persisted header toggle, no first-paint flash)
- official mascot: lossless WebP light/dark logos + favicon, embedded and served
  with cache headers (1.2MB source PNGs -> ~27/22KB webp)
- Assets/Contents views, pagination (offset) and native virtual scrolling
  (content-visibility) so large result sets do not blow up the DOM
- search with debounce/abort + URL state, exposure chips, sort, summary counts

Readability + indexing:
- internal/index listers for S3-compatible, GCS, Azure, Firebase, OpenSearch
  (metadata only: keys/sizes/dates; never object contents)
- asset_entries table + store write/search; readable resources are indexed and
  searchable in the UI
- deep, unauthenticated, read-only DB probes (PROBE_DEEP): Redis PING,
  PostgreSQL StartupMessage, MongoDB OP_MSG listDatabases; OpenSearch/Neptune
  HTTP reads
- Azure account-level listing fixed: 403 is reported reachable, not
  auth_required, since anonymous access is per-container

DNS:
- curated reliable resolvers by default; DNS_SERVERS=fresh adds the compiled-in
  Fresh-Resolvers snapshot with a bounded startup sample (warming all 6.7k at
  once broke the container resolver)
- random healthy selection + failover + per-server cooldown

Other: prober Close fixes a shutdown hang; config/README updated.
This commit is contained in:
StasonJatham committed 2026-09-22 14:33:03 +02:00
1 parent 8fce55f8c9
commit a366905a43
28 files changed
+9744 -241

No files matched your search

+8 -6
View File
@@ -33,12 +33,13 @@ type Metrics struct {
DBRows atomic.Uint64
DBErrors atomic.Uint64
QueueDrops atomic.Uint64
QueueFull atomic.Uint64
MalformedCT atomic.Uint64
CTErrors atomic.Uint64
ProbeAttempts atomic.Uint64
ProbePublic atomic.Uint64
QueueDrops atomic.Uint64
QueueFull atomic.Uint64
MalformedCT atomic.Uint64
CTErrors atomic.Uint64
ProbeAttempts atomic.Uint64
ProbePublic atomic.Uint64
EntriesIndexed atomic.Uint64
gauges []gauge
}
@@ -84,6 +85,7 @@ func (m *Metrics) Handler() http.Handler {
counter("ct_errors_total", "Non-fatal CT errors", m.CTErrors.Load())
counter("probe_attempts_total", "Optional reachability probes performed", m.ProbeAttempts.Load())
counter("probe_public_total", "Probes classified as public read", m.ProbePublic.Load())
counter("entries_indexed_total", "Metadata entries indexed from readable resources", m.EntriesIndexed.Load())
for _, g := range m.gauges {
fmt.Fprintf(w, "# HELP %s %s\n# TYPE %s gauge\n%s%s %g\n", g.name, g.help, g.name, g.name, g.extra, g.fn())
}