fix memory blowup and keep direct matches from being dropped
- dns.TTLCache: stop preallocating every shard map to maxShard; this allocated the full theoretical cache capacity up front (hundreds of MB of empty buckets) and pushed the container over its memory limit, causing an OOM restart loop under Docker's default 1g cap. - pipeline: write direct provider matches immediately without a blocking DNS enrichment step, so they can no longer be starved by the optional CNAME queue. Add TestDirectNotStarvedByDNSBacklog. - config: add DEDUP_SIZE, lower DNS_CACHE_SIZE/DEDUP_SIZE defaults to 256k. - .env.example / README: document DEDUP_SIZE and cache sizing. Live: stable at ~250-300 MiB with no restarts, direct queue draining to 0.
This commit is contained in:
1 parent
e8f7d6bfdf
commit
9710efec1a
7 files changed
+75
-18
No files matched your search
@@ -114,6 +114,7 @@ func run() error {
|
||||
BatchSize: cfg.DBBatchSize,
|
||||
FlushInterval: cfg.DBFlushInterval,
|
||||
DedupTTL: cfg.DedupeTTL,
|
||||
DedupSize: cfg.DedupSize,
|
||||
Enricher: enricher,
|
||||
Write: write,
|
||||
Metrics: m,
|
||||
|
||||
Reference in new issue
Block a user