Simplify documentation and add repository development skill
This commit is contained in:
Ops Agent committed
1 parent
3d8cf46a63
commit
5aaa31dc1a
10 files changed
+473
-239
No files matched your search
+67
-152
@@ -1,181 +1,96 @@
|
||||
# WhatsApp test suite
|
||||
# CLI reference
|
||||
|
||||
## Portable single-file bundle
|
||||
|
||||
Copy **`whatsapp.py`** to another writable folder. That one Python file includes
|
||||
all tools, the native helpers and the regression fixtures. No adjacent source
|
||||
package, DLLs, C++ compiler or build script is needed to run it.
|
||||
Use the portable file from a writable folder:
|
||||
|
||||
```powershell
|
||||
python whatsapp.py --help
|
||||
python whatsapp.py test
|
||||
python whatsapp.py <command> --help
|
||||
```
|
||||
|
||||
For development, replace `python whatsapp.py` with `python -m whatsapp_suite`. With no arguments, either entry point runs offline tests.
|
||||
|
||||
## Core commands
|
||||
|
||||
Live commands require signed-in Windows WhatsApp Desktop, x64 Windows and 64-bit Python 3.11+. The core path uses the Python standard library and bundled native helpers, without CDP or UI automation.
|
||||
|
||||
| Command | Result | Useful options |
|
||||
| --- | --- | --- |
|
||||
| `test` | Synthetic regression results | `--output`, explicit `--live` / `--capabilities` |
|
||||
| `contacts-export` | Selected fields from all local contact rows | `--output`, `--pid` |
|
||||
| `export-zip` | Local message records and manifest in ZIP | `--output`, `--batch-size`, `--redact-text`, `--pid` |
|
||||
| `live` | Fixed app probes and detailed report | `--suite`, `--output`, `--pid` |
|
||||
| `desktop-search QUERY` | App search results with message metadata | `--limit`, `--output`, `--pid` |
|
||||
| `send-self` | One actual self-message and verification receipt | `--output`, `--pid` |
|
||||
| `webhook` | Listener installation, status, delivery and restoration | See below |
|
||||
|
||||
```powershell
|
||||
python whatsapp.py contacts-export --output exports/contacts.json
|
||||
python whatsapp.py export-zip --output exports/messages.zip
|
||||
python whatsapp.py live --suite history-read
|
||||
python whatsapp.py contacts-export --output contacts.json
|
||||
python whatsapp.py export-zip --output messages.zip
|
||||
python whatsapp.py send-self
|
||||
python whatsapp.py desktop-search "keyword" --limit 10
|
||||
```
|
||||
|
||||
The bundle compresses the original modules and resources together with LZMA.
|
||||
It verifies its embedded payload and imports Python modules directly from memory.
|
||||
Only a native helper needed for a live command is materialized in a hash-verified
|
||||
cache beside the bundle. Keep the containing folder writable. Native helper
|
||||
bytes, native object checks, batch sizes and message read/send paths are unchanged.
|
||||
`live --suite` accepts `math`, `capabilities`, `all`, `message-json`, `contacts` and `history-read`. The latter reads the five oldest records after a bounded timestamp scan. It saves decoded body/caption text when present.
|
||||
|
||||
Core WebView commands use the standard library. Python 3.11+, Windows x64,
|
||||
64-bit Python are required for live Desktop operations. Other Desktop versions
|
||||
are attempted with a compatibility warning; only the recorded version was tested. Optional legacy CDP, Frida and QR tools retain the packages
|
||||
listed below. Node.js remains optional for JavaScript fixture tests.
|
||||
## Work with saved exports
|
||||
|
||||
The development package below remains the editable source. Rebuild the generated
|
||||
single-file distribution after changing it:
|
||||
These commands read existing JSON/JSONL files, without running WhatsApp:
|
||||
|
||||
```powershell
|
||||
python build_whatsapp_bundle.py
|
||||
python whatsapp.py stats messages.jsonl
|
||||
python whatsapp.py search messages.jsonl "keyword" --limit 20 --redact-text
|
||||
```
|
||||
|
||||
The build is deterministic and records size and SHA-256 in
|
||||
`analysis/whatsapp-suite/bundle-build.json`. The build file is not needed on a
|
||||
machine running the bundle.
|
||||
Extract `messages.jsonl` from a message ZIP before searching it; the search command does not read ZIP files directly. Search supports `--chat-id` and `--output`. `inventory` lists local storage files and sizes.
|
||||
|
||||
The WhatsApp scripts are consolidated in `whatsapp_suite/`, with one CLI and
|
||||
offline regression tests in `tests/`. Run commands from this workspace.
|
||||
## Webhook commands
|
||||
|
||||
```powershell
|
||||
python -m whatsapp_suite
|
||||
python -m whatsapp_suite --help
|
||||
python -m whatsapp_suite test --output analysis/whatsapp-suite/results.json
|
||||
python whatsapp.py webhook install --endpoint https://your-endpoint.example/events --self-only --auto-restore --config listener.json
|
||||
python whatsapp.py webhook status
|
||||
python whatsapp.py webhook stop --config listener.json
|
||||
```
|
||||
|
||||
`python whatsapp_tests.py` is an equivalent entry point. The default run uses
|
||||
fixtures and temporary files. It verifies saved-export parsing, Unicode search,
|
||||
chat filtering, text redaction, history pagination/decryption contracts, fixed
|
||||
probe responses, command dispatch, and bundled DLL hashes. Node.js runs the
|
||||
JavaScript history fixtures; that test is reported as skipped if Node is absent.
|
||||
Python 3.11 or newer is required.
|
||||
Set the endpoint token in the environment variable named by `--token-env` (default `WHATSAPP_INJECTOR_WEBHOOK_TOKEN`). Never place the token in the URL.
|
||||
|
||||
To test an already running WhatsApp Desktop installation:
|
||||
|
||||
```powershell
|
||||
python -m whatsapp_suite test --live
|
||||
python -m whatsapp_suite test --capabilities
|
||||
python -m whatsapp_suite live --suite math
|
||||
python -m whatsapp_suite live --suite capabilities
|
||||
python -m whatsapp_suite live --suite history-read --output analysis/whatsapp-suite/js-history-read.json
|
||||
python -m whatsapp_suite export-zip --output exports/whatsapp-messages.zip
|
||||
python -m whatsapp_suite contacts-export --output exports/contacts.json
|
||||
python -m whatsapp_suite send-self
|
||||
```
|
||||
|
||||
Live WebView probes require Windows x64 and 64-bit Python. Desktop build
|
||||
`2.2639.100.0` and its recorded WebView2 Core DLL were tested. Other versions or
|
||||
DLL hashes produce a warning and are attempted. The helper discovers function and vtable locations from binary fingerprints and checks native
|
||||
objects and methods before executing JavaScript; changed layouts can
|
||||
cause failure or process instability. The JSON report includes detected/tested
|
||||
compatibility metadata and cleanup status. `--pid` selects a Desktop host.
|
||||
|
||||
## Tools
|
||||
|
||||
Every command supports `--help` for its options.
|
||||
|
||||
| Command | Purpose |
|
||||
| Action | Behavior |
|
||||
| --- | --- |
|
||||
| `inventory` | List local Desktop storage files and sizes |
|
||||
| `stats EXPORT` | Summarize a saved JSON or JSONL export |
|
||||
| `search EXPORT QUERY` | Unicode-aware literal search of saved messages |
|
||||
| `desktop-search QUERY` | Bounded native app search through WebView2; metadata output |
|
||||
| `live --suite math\|capabilities\|all` | Fixed live WebView test suites |
|
||||
| `live --suite message-json\|contacts` | Explicit fixed message search/contact sample; saves private data |
|
||||
| `live --suite history-read` | Read the five oldest local message records through the JS database getter and decoder, without CDP |
|
||||
| `webview-test` | Arithmetic check in a separate WebView2 host; may download its SDK |
|
||||
| `analyze` | Storage analysis, optionally using an existing loopback CDP endpoint |
|
||||
| `history-export` | Export local history through Desktop's own database decryption |
|
||||
| `export-zip` | Export all local message rows to ZIP through JavaScript on WebView, without CDP or UI automation |
|
||||
| `webhook` | Install/status/stop, direct HTTPS or loopback relay, ensure/auto-restore and optional watch/serve; see [webhook guide](WEBHOOKS.md) |
|
||||
| `contacts-export` | Export all local contact rows to JSON through the app's JS database getter |
|
||||
| `send-self` | Send one fixed test message to the signed-in account itself and verify stored plaintext |
|
||||
| `ram-read` | Read loaded V8 message objects from process memory |
|
||||
| `ram-search QUERY` | Search RAM models with verified collection membership |
|
||||
| `native-search [QUERY]` | Experimental injected SQLite search, `--oldest --limit 5`, or `--export-index` |
|
||||
| `qr-memory` | Recover candidate Desktop pairing QR payloads from RAM |
|
||||
| `qr-window` | Save an already displayed Desktop linking QR |
|
||||
| `pairing` | Start an isolated browser and request an official linking QR |
|
||||
| `install` | Subscribe to message events and start the resident native relay; installer exits |
|
||||
| `install --restore --config FILE` | Reuse and re-enable an existing configuration |
|
||||
| `status` | Report listener and relay counters |
|
||||
| `stop --config FILE` | Stop the listener and disable restoration in that configuration |
|
||||
| `auto-restore --config FILE` | Register/enable the per-user minute task for a direct HTTPS listener |
|
||||
| `ensure --config FILE` | Check once and restore a missing configured listener |
|
||||
| `watch --config FILE` | Optional continuous restoration loop |
|
||||
| `serve --restore-config FILE --store FILE` | Run the optional loopback receiver with SQLite storage and restoration |
|
||||
|
||||
```powershell
|
||||
python -m whatsapp_suite search export.jsonl "keyword" --redact-text
|
||||
python -m whatsapp_suite desktop-search "keyword" --limit 10
|
||||
python -m whatsapp_suite history-export --help
|
||||
```
|
||||
Direct installation also accepts `--endpoint`, `--token-env`, `--self-only`, `--auto-restore`, `--config` and `--pid`. Loopback installation uses `--port` instead of `--endpoint`. Full setup, payload and task removal are in [the webhook guide](WEBHOOKS.md).
|
||||
|
||||
`history-read` uses the native WebView bridge to execute JavaScript in the running
|
||||
app. It scans at most 100,000 local record timestamps in a read-only transaction,
|
||||
then calls `WAWebSchemaMessage.getMessageTable().get` and
|
||||
`WAWebDBMessageSerialization.messageFromDbRow` for the five oldest records.
|
||||
It saves decoded body/caption text when present, including messages absent from
|
||||
the loaded conversation models. System events and media-only records can have
|
||||
no text. The live result includes page-state and native bridge cleanup status.
|
||||
This command requires neither CDP nor optional Python packages.
|
||||
## Output and failure behavior
|
||||
|
||||
`export-zip` uses the same native bridge and the app's JavaScript database getter
|
||||
and decoder. It paginates all local rows up to the initial upper primary key,
|
||||
streams them into `messages.jsonl`, and includes a `manifest.json` with row counts,
|
||||
decoding errors, scope, and bridge cleanup status. Rows include message IDs,
|
||||
chat IDs, timestamps, types, body and caption text; media files are not downloaded.
|
||||
This is a live scan, not an atomic database snapshot. Row decoding failures are
|
||||
preserved as error records and make the command return a failure status.
|
||||
`--batch-size` controls bounded reads (default 100); `--redact-text` omits body and
|
||||
caption text. Existing outputs are refused. Interrupted exports retain a
|
||||
`.zip.partial` file and a `.zip.report.json` diagnostic report. A completed ZIP
|
||||
is checked for CRC errors before publication.
|
||||
- Commands emit compact JSON summaries; exports and detailed reports are local files.
|
||||
- Success normally returns exit code 0; reported failures return a nonzero code. `webhook ensure` may exit 0 while waiting for WhatsApp: inspect its status file, not only the exit code.
|
||||
- ZIP exports contain `messages.jsonl` and `manifest.json`. They cover local rows up to the initial upper key, with no phone sync or media download.
|
||||
- The scan is not an atomic snapshot. Decode failures remain visible as error records and make the export fail.
|
||||
- Existing export outputs are refused. Interrupted ZIP exports retain `.zip.partial` and an adjacent `.zip.report.json`.
|
||||
- `send-self` reserves its receipt before sending. It verifies the recipient, stored plaintext and acknowledgement; an uncertain attempt is not retried automatically.
|
||||
- Live reports include compatibility and cleanup information. A version warning does not certify support for that build.
|
||||
|
||||
`contacts-export` reads the local contact table rather than the five-contact
|
||||
loaded-model sample. It exports selected identity, name, phone and business
|
||||
fields. Local records can include separate PN/LID entries and contacts outside
|
||||
the phone address book; this is not a deduplicated phone address-book export.
|
||||
Keep private outputs out of Git. See [troubleshooting](TROUBLESHOOTING.md) and [security](../SECURITY.md).
|
||||
|
||||
`send-self` resolves the signed-in account and checks both the selected chat and
|
||||
outgoing message recipient before invoking the app's text-send action. It sends
|
||||
one identifiable test message and reads it back through the message database
|
||||
decoder, recording ACK and exact plaintext equality in a receipt. An existing
|
||||
receipt is refused and sending is never retried automatically. Unlike the
|
||||
offline test suite, this command actually sends a WhatsApp message.
|
||||
## Optional research commands
|
||||
|
||||
The separate bulk `history-export` command requires an existing Desktop CDP endpoint. RAM tools only see
|
||||
loaded objects and cannot retrieve all history. Pairing commands are separate
|
||||
from the automated test run. The `runtime` module preserves the explicit
|
||||
temporary CDP context manager for research scripts; entering that context
|
||||
restarts Desktop and is never part of the default tests.
|
||||
These commands have distinct transports and are separate from the core no-CDP workflow. Always inspect their help before use.
|
||||
|
||||
## Optional dependencies
|
||||
|
||||
The default suite, saved-export tools, and native WebView probes need only the
|
||||
Python standard library (plus Node.js for the history fixture test).
|
||||
|
||||
| Tools | Additional Python packages |
|
||||
| Command | Purpose / dependency |
|
||||
| --- | --- |
|
||||
| CDP analysis and history export | `websockets` |
|
||||
| Injected SQLite search | `frida` |
|
||||
| QR from RAM | `qrcode[pil]` |
|
||||
| QR from Desktop window | `Pillow`, `zxing-cpp` |
|
||||
| Isolated browser pairing | `websockets`, `qrcode[pil]`; installed Chrome |
|
||||
| `webview-test` | Separate WebView2 host test; may download its SDK |
|
||||
| `analyze` | Storage inspection, optionally an existing CDP session; `websockets` for CDP |
|
||||
| `history-export` | Bulk history through an existing Desktop CDP endpoint; `websockets` |
|
||||
| `ram-read`, `ram-search` | Inspect loaded models in RAM; not complete history |
|
||||
| `native-search` | Experimental injected SQLite search/index export; `frida` |
|
||||
| `qr-memory` | Pairing QR candidates in RAM; `qrcode[pil]` |
|
||||
| `qr-window` | Capture an already displayed pairing QR; `Pillow`, `zxing-cpp` |
|
||||
| `pairing` | Isolated browser linking QR; Chrome, `websockets`, `qrcode[pil]` |
|
||||
|
||||
Install all optional packages into a virtual environment with:
|
||||
|
||||
```powershell
|
||||
python -m pip install -r requirements-whatsapp-optional.txt
|
||||
```
|
||||
|
||||
## Layout and migration
|
||||
|
||||
- `whatsapp_suite/`: canonical tool implementations, common normalization,
|
||||
storage paths, Desktop discovery, CLI, and test runner.
|
||||
- `whatsapp_suite/assets/`: verified native DLLs and the native SQLite agent.
|
||||
- `tests/`: synthetic Python regressions and the JavaScript history fixture.
|
||||
- `analysis/whatsapp-suite/results.json`: latest automated test result.
|
||||
|
||||
The former `whatsapp_tool.py` commands now use `python -m whatsapp_suite`;
|
||||
`whatsapp-search` is named `desktop-search`, and `whatsapp-test` is
|
||||
`live --suite math`. The former `whatsapp_math.py` options are available under
|
||||
`live`. Archived scripts have become package modules; the identical Desktop QR
|
||||
RAM scripts have one implementation (`qr_memory`). Research helpers now import
|
||||
these modules. Old script archive and portable script copies were removed.
|
||||
The clean Git project excludes research reports, extracted application code and browser profiles. The surrounding original workspace retains those local files.
|
||||
`requirements-whatsapp-optional.txt` lists optional Python packages. No optional research action, pairing operation or send is performed by default tests.
|
||||
Reference in new issue
Block a user