Bucketeer UI (dark mode, pagination, virtual scroll), readable+indexed resources, LLM-friendly logs

Logging:
- structured slog with LOG_FORMAT=json|text and a stable key vocabulary
- periodic stats record with queue depths, db pool, goroutines, memory, dns/probe
- no per-certificate logging (hot path is counters only)

UI (Bucketeer branding, DESIGN.md):
- light+dark themes (OS default, persisted header toggle, no first-paint flash)
- official mascot: lossless WebP light/dark logos + favicon, embedded and served
  with cache headers (1.2MB source PNGs -> ~27/22KB webp)
- Assets/Contents views, pagination (offset) and native virtual scrolling
  (content-visibility) so large result sets do not blow up the DOM
- search with debounce/abort + URL state, exposure chips, sort, summary counts

Readability + indexing:
- internal/index listers for S3-compatible, GCS, Azure, Firebase, OpenSearch
  (metadata only: keys/sizes/dates; never object contents)
- asset_entries table + store write/search; readable resources are indexed and
  searchable in the UI
- deep, unauthenticated, read-only DB probes (PROBE_DEEP): Redis PING,
  PostgreSQL StartupMessage, MongoDB OP_MSG listDatabases; OpenSearch/Neptune
  HTTP reads
- Azure account-level listing fixed: 403 is reported reachable, not
  auth_required, since anonymous access is per-container

DNS:
- curated reliable resolvers by default; DNS_SERVERS=fresh adds the compiled-in
  Fresh-Resolvers snapshot with a bounded startup sample (warming all 6.7k at
  once broke the container resolver)
- random healthy selection + failover + per-server cooldown

Other: prober Close fixes a shutdown hang; config/README updated.
This commit is contained in:
StasonJatham committed 2026-09-22 14:33:03 +02:00
1 parent 8fce55f8c9
commit a366905a43
28 files changed
+9744 -241

No files matched your search

+18
View File
@@ -39,6 +39,24 @@ CREATE TABLE IF NOT EXISTS observations (
CREATE INDEX IF NOT EXISTS observations_asset_idx ON observations (asset_id, observed_at DESC);
-- Metadata index of resources confirmed anonymously readable. Keys/sizes/dates
-- only; never object contents.
CREATE TABLE IF NOT EXISTS asset_entries (
id BIGSERIAL PRIMARY KEY,
asset_id BIGINT NOT NULL REFERENCES cloud_assets (id) ON DELETE CASCADE,
kind TEXT NOT NULL,
name TEXT NOT NULL,
size BIGINT NOT NULL DEFAULT 0,
updated TIMESTAMPTZ,
content_type TEXT NOT NULL DEFAULT '',
etag TEXT NOT NULL DEFAULT '',
indexed_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (asset_id, kind, name)
);
CREATE INDEX IF NOT EXISTS asset_entries_asset_idx ON asset_entries (asset_id);
CREATE INDEX IF NOT EXISTS asset_entries_name_idx ON asset_entries (name);
-- CT resume state. `state` is the JSON-encoded certspotter monitor.LogState
-- (download/verified Merkle positions and last verified STH). No certificate
-- contents are ever stored here.