boring stuff

This commit is contained in:
StasonJatham committed 2026-09-19 14:56:03 +02:00
1 parent 8f92e64ffd
commit cbfa73c4b3
4 files changed
+62 -4

No files matched your search

+16 -2
View File
@@ -3,6 +3,10 @@
Command line tools to parse, deduplicate, and query multi-billion password
collections (Collections 1-5, AntiPublic, Breach Compilation).
> **Responsible use.** This project ships **no data**. Use it only on datasets
> you own or are authorized to process, for security research and defense.
> Do not commit datasets, hashes, or cracked credentials.
Processing produces a deduped `a/b/c` bucket tree that answers exact lookups in
O(1) and wildcard queries with a parallel ripgrep scan:
@@ -164,10 +168,20 @@ Run `collectionparser --help` or `collectionparser <command> --help`.
| `sort` | Sort and deduplicate every file in a query-friendly folder. |
| `merge` | Merge one query-friendly folder into another, then sort/dedup. |
## Credits
Derived from and inspired by
[philipperemy/3.7-billion-passwords-tools](https://github.com/philipperemy/3.7-billion-passwords-tools)
by [Philippe Remy](https://github.com/philipperemy) — the original parser and
`a/b/c` index. This project rewrites it for memory safety, resumable batched
builds, wildcard queries, hash collection/cracking and JSONL export.
Thanks, Philippe.
## Development
```bash
pip install -e '.[dev]'
ruff check . # lint
pytest # tests
ruff format . && ruff check . # style + lint
pytest # tests
```