boring stuff
This commit is contained in:
1 parent
8f92e64ffd
commit
cbfa73c4b3
4 files changed
+62
-4
No files matched your search
@@ -3,6 +3,10 @@
|
||||
Command line tools to parse, deduplicate, and query multi-billion password
|
||||
collections (Collections 1-5, AntiPublic, Breach Compilation).
|
||||
|
||||
> **Responsible use.** This project ships **no data**. Use it only on datasets
|
||||
> you own or are authorized to process, for security research and defense.
|
||||
> Do not commit datasets, hashes, or cracked credentials.
|
||||
|
||||
Processing produces a deduped `a/b/c` bucket tree that answers exact lookups in
|
||||
O(1) and wildcard queries with a parallel ripgrep scan:
|
||||
|
||||
@@ -164,10 +168,20 @@ Run `collectionparser --help` or `collectionparser <command> --help`.
|
||||
| `sort` | Sort and deduplicate every file in a query-friendly folder. |
|
||||
| `merge` | Merge one query-friendly folder into another, then sort/dedup. |
|
||||
|
||||
## Credits
|
||||
|
||||
Derived from and inspired by
|
||||
[philipperemy/3.7-billion-passwords-tools](https://github.com/philipperemy/3.7-billion-passwords-tools)
|
||||
by [Philippe Remy](https://github.com/philipperemy) — the original parser and
|
||||
`a/b/c` index. This project rewrites it for memory safety, resumable batched
|
||||
builds, wildcard queries, hash collection/cracking and JSONL export.
|
||||
|
||||
Thanks, Philippe.
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
pip install -e '.[dev]'
|
||||
ruff check . # lint
|
||||
pytest # tests
|
||||
ruff format . && ruff check . # style + lint
|
||||
pytest # tests
|
||||
```
|
||||
Reference in new issue
Block a user