Bound crack memory: cap wordlist sort, chunk target loading

- prepare_candidates: use sort -S 1G --parallel 4 (was unbounded, OOM-prone
  on large --passwords-from datasets)
- crack: add --max-hashes to stream large hash files in bounded chunks, so the
  in-RAM target index no longer scales with the whole input
- docs + regression test for chunked cracking
This commit is contained in:
StasonJatham committed 2026-09-23 20:01:02 +02:00
1 parent 1ead58173c
commit 3b7011f433
4 files changed
+119 -36

No files matched your search

+5
View File
@@ -139,6 +139,11 @@ sources are merged through `sort -u`. Only unsalted MD5/SHA1/SHA256 are
attackable — bcrypt is ignored. Feed `cracked.txt` back into the index with
`split`/`merge`.
The **target index is held in RAM** (roughly 250 bytes per hash), so a
multi-hundred-million-hash file will exhaust memory. Pass `--max-hashes N`
(e.g. `--max-hashes 8000000`) to stream the input in bounded chunks at the cost
of rescanning the wordlist per chunk.
## Measured performance
Single large-file test on Collections 2-5 (3 files, 19.94 GB, 3 workers):