karl/ghostwire
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Simplify documentation and add repository development skill

Browse Source
This commit is contained in:
Ops Agent committed 2026-10-11 09:03:49 +02:00
1 parent 3d8cf46a63
commit 5aaa31dc1a
10 files changed
+473 -239
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/ARCHITECTURE.md
+84 -17
View File
@@ -1,35 +1,102 @@
# Architecture
## Execution path
## Two ways to run the same code
The portable launcher verifies and decompresses its embedded payload, then imports the Python package directly from the in-memory ZIP. It uses the same CLI and modules as `python -m whatsapp_suite`.
`python -m whatsapp_suite` runs the editable source package. `python whatsapp.py` runs its generated portable distribution through the same CLI.
For a core live operation, Python uses Windows APIs through `ctypes`. It verifies the helper DLL, records application/WebView compatibility, installs a temporary `WH_GETMESSAGE` hook on the Desktop UI thread, and passes a bounded JavaScript request through shared memory. The native helper discovers function/vtable locations from relocation-free binary fingerprints and locates the existing WebView object using those dynamically resolved addresses and calls its `ExecuteScriptAsync` interface. A thread message wakes the hook. Results return through the shared mapping; the hook, mapping and process handles are then released. Reports record cleanup outcomes.
The builder stores the maintained modules, fixtures and selected assets in a compressed archive. The launcher verifies its payload, imports Python modules from memory, and extracts only the native helper needed for a live operation into a hash-verified cache beside the bundle. A C++ compiler is needed to change native code, not to run the portable file.
This still uses native C++ DLLs. Python packages their bytes; it does not replace their implementation with pure Python. The C++ source is for maintenance and is unnecessary on a machine running the portable file. SHA-256 checks verify embedded asset integrity, not publisher identity or authenticity of a downloaded file.
## Live JavaScript execution
Message export uses the app's own JavaScript database getter and row decoder. It paginates a bounded range, streams decoded records into a ZIP and writes a manifest. The scan is not an atomic database snapshot. Failed decodes remain visible and cause failure status. Final publication refuses existing output and verifies ZIP CRCs.
```mermaid
sequenceDiagram
participant CLI as Python CLI
participant DLL as Native helper
participant WV as Existing WhatsApp WebView
CLI->>CLI: Detect host and UI thread; verify helper
CLI->>DLL: Temporary Windows hook + shared request
DLL->>DLL: Discover and validate native layout
DLL->>WV: ExecuteScriptAsync
WV-->>DLL: JavaScript result
DLL-->>CLI: Shared result packet
CLI->>CLI: Release hook, mappings and handles
```
Contact export reads selected public fields from the local contact table. Sending is restricted to a fixed self-message test with recipient checks and persisted plaintext readback. No general-purpose arbitrary-recipient send CLI is provided.
Python calls Windows APIs through `ctypes`. A `WH_GETMESSAGE` hook loads the helper on WhatsApp's UI thread. A thread message wakes it; a shared mapping carries a bounded request and result.
## Boundaries
The helper scans the loaded WebView component for relocation-free code fingerprints and a matching vtable layout, then locates the existing object and validates its script-execution method. This avoids fixed module-relative function addresses. Compiler/object-field and interface ABI assumptions remain.
Live native operations were tested with Desktop build `2.2639.100.0`. Other versions are attempted with a warning. The helper resolves function and vtable addresses dynamically, but retains function fingerprints, an object field offset and the exact interface ABI derived from the tested WebView DLL. A changed layout may fail discovery or cause process instability; dynamic address discovery does not make all compiler/object layouts compatible. Updates require new evidence and validation to be considered tested. Core bridge commands do not use CDP or UI automation; optional legacy commands have their own transports and dependencies. RAM readers only see loaded models. Exported message text does not include media file downloads. Password or passkey private-key extraction is not an implemented feature.
The hook is temporary. Normal exports and probes record cleanup of the hook, mapping, process handles and helper. Native failures are not evidence that a different layout is safe.
The repository includes only maintained source, synthetic fixtures and required runtime assets. Local profiles, extracted application code, credentials, messages, contacts and research dumps are excluded.
## Read, search and send
## Security mechanism classification
| Operation | Data path | Boundary |
| --- | --- | --- |
| ZIP export | App database getter → app row decoder → paginated records → ZIP | Local rows; no atomic snapshot, phone sync or media download |
| Old-message read | Timestamp scan → five oldest rows → app decoder | Bounded scan; not limited to loaded chat models |
| Contact export | Local contact table → selected identity/name/business fields | May contain separate PN/LID records |
| Desktop search | Existing app message search → bounded metadata | Search scope comes from the app |
| Saved-export search | Python JSON/JSONL parsing and normalization | Does not need the app |
| Self-message | Resolve own identity → guarded send → stored readback | One fixed test message; no arbitrary-recipient CLI |
The observed implementation uses documented Windows hook loading and WebView script execution APIs. It does not demonstrate a memory-corruption exploit, privilege escalation or a cryptographic break. It obtains the running application's existing decrypted data through that application's internal getters under the current signed-in session.
Database text is decoded through WhatsApp's own JavaScript methods under the signed-in session. Failed row decodes remain explicit. Export publication refuses existing outputs and checks ZIP CRCs.
However, loading a helper into another process and locating internal objects by memory layout is reverse-engineered process instrumentation. WhatsApp does not expose the internal module calls used here as a documented external automation API. Calling documented OS/WebView APIs does not establish that this overall integration is supported or intended by WhatsApp, nor is this observation a complete security audit.
## Listener and delivery
References: [Microsoft SetWindowsHookExW](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setwindowshookexw), [Microsoft WebView2 JavaScript execution](https://learn.microsoft.com/en-us/microsoft-edge/webview2/how-to/javascript).
```mermaid
flowchart LR
M["Loaded message collection"] --> J["JS observer and decoded queue"]
J --> N["UI-thread native timer"]
N --> H["WinHTTP worker"]
H --> T["HTTPS target or loopback receiver"]
T --> A["HTTP 2xx"]
A --> Q["Remove acknowledged queue entries"]
R["Optional Windows minute task"] --> I["Restore after app restart"]
I --> J
```
## Autonomous webhook relay
The JavaScript producer observes message collection events, obtains stored rows through the app decoder, and queues bounded message events. It does not contain transport credentials. A UI-thread timer executes short queue scripts; a worker performs HTTP delivery outside that UI thread.
The explicit `webhook install` command registers JavaScript message collection observers and installs a separate resident helper. A UI-thread timer runs short queue-peek scripts; a WinHTTP worker posts bounded batches to the configured HTTPS endpoint or `127.0.0.1:port/whatsapp-webhook`, without proxies or redirects. HTTPS uses certificate validation and a token header. Queue entries are removed after HTTP 2xx. The built-in loopback receiver acknowledges after SQLite commit and deduplicates message IDs.
Direct HTTPS uses certificate validation and a token header. Loopback mode posts to a local receiver. Proxies and redirects are disabled. Native WinHTTP supplies the outbound transport; direct JavaScript localhost requests were blocked by WhatsApp's CSP during testing, and the implementation does not weaken CSP.
The installer exits and releases its temporary hooks. The webhook helper retains one module reference so native callbacks remain valid; stop cancels the timer and releases WebView references but leaves an idle DLL loaded until app exit. This is intentional and distinct from the short-lived test/export helpers' unload checks.
HTTP 2xx triggers queue acknowledgement. Failures leave records for retry. A consumer must deduplicate IDs; this is not exactly-once delivery. The built-in local receiver commits SQLite data before acknowledgement.
App process exit destroys both the injected JavaScript and resident relay. For direct HTTPS, `--auto-restore` registers a per-user Windows task that runs a short `webhook ensure` check every minute. It uses a DPAPI-protected token and does not keep Python or a local server running between checks. Alternatively, a loopback receiver started with `--restore-config` checks every five seconds. Saved configuration persists; injected process memory does not. See [webhook guide](WEBHOOKS.md) for limits.
The installer exits after setup. The native relay retains a module reference so timer/worker callbacks remain valid. Stop cancels observation/timers and releases WebView references; an idle DLL remains loaded until app exit.
## Restoration and configuration
WhatsApp process exit destroys the listener and native relay. Automatic restoration is a separate capability:
| Mode | Bootstrap | Lifetime |
| --- | --- | --- |
| Direct HTTPS with `--auto-restore` | Per-user Windows task invokes `webhook ensure` every minute | Short Python checks; no continuous local server |
| Loopback receiver with `--restore-config` | Receiver checks every five seconds | Receiver must remain running |
| Optional `watch` | Python restoration loop | Watcher remains running |
The task runs as the logged-in user with limited privileges. It stores paths to Python, the bundle and configuration; moving them breaks that registration. Direct restoration recovers a current-user DPAPI-protected token from the configuration. The bundle is portable, but that protected configuration is tied to the Windows user/machine.
Reinstallation reconciles qualifying loaded models since the saved start timestamp; it does not perform complete database catch-up. In-memory queues can be lost on exit, and messages in startup/polling gaps are not guaranteed to be delivered. See [the webhook guide](WEBHOOKS.md) for schema and limits.
## Source map
| Layer | Files |
| --- | --- |
| Command dispatch and tests | `cli.py`, `runner.py`, `tests/` |
| Shared resources/cache | `common.py` |
| Windows orchestration | `desktop.py`, `live.py`, `native_search.py` |
| Data operations | `history.py`, `zip_export.py`, `contacts_export.py`, `exports.py`, `self_message.py` |
| Listener orchestration | `webhook.py`, `assets/webhook_listener.js` |
| Native implementations | `native/math_probe.cpp`, `search_probe.cpp`, `webhook_probe.cpp`, `core_discovery.h` |
| Portable packaging | `build_whatsapp_bundle.py`, generated `whatsapp.py` |
All Python filenames above are under `whatsapp_suite/`. See [contributing](../CONTRIBUTING.md) for extension steps and hash updates.
## Compatibility and security boundaries
The tested Desktop build is `2.2639.100.0`. Other builds produce advisory metadata/warnings and are attempted, but dynamic address discovery does not imply compatibility with every future compiler/object layout. Changed layouts can fail or cause process instability.
The observed bridge uses documented Windows loading and WebView execution APIs. It does not demonstrate memory corruption, privilege escalation or broken encryption. The overall integration remains reverse-engineered process instrumentation: WhatsApp does not document these internal module calls as an external automation API. This is not a complete security audit.
Passwords/passkey private keys and media download are not implemented features. Core commands use no CDP or UI automation; optional research tools have separate transports. Hashes establish integrity, not publisher identity.
See [validation](VALIDATION.md) for evidence, [security](../SECURITY.md) for private-data handling, and [Microsoft's Windows hook](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setwindowshookexw) and [WebView JavaScript](https://learn.microsoft.com/en-us/microsoft-edge/webview2/how-to/javascript) documentation for the public APIs.
docs/CLI.md
+67 -152
View File
@@ -1,181 +1,96 @@
# WhatsApp test suite
# CLI reference
## Portable single-file bundle
Copy **`whatsapp.py`** to another writable folder. That one Python file includes
all tools, the native helpers and the regression fixtures. No adjacent source
package, DLLs, C++ compiler or build script is needed to run it.
Use the portable file from a writable folder:
```powershell
python whatsapp.py --help
python whatsapp.py test
python whatsapp.py <command> --help
```
For development, replace `python whatsapp.py` with `python -m whatsapp_suite`. With no arguments, either entry point runs offline tests.
## Core commands
Live commands require signed-in Windows WhatsApp Desktop, x64 Windows and 64-bit Python 3.11+. The core path uses the Python standard library and bundled native helpers, without CDP or UI automation.
| Command | Result | Useful options |
| --- | --- | --- |
| `test` | Synthetic regression results | `--output`, explicit `--live` / `--capabilities` |
| `contacts-export` | Selected fields from all local contact rows | `--output`, `--pid` |
| `export-zip` | Local message records and manifest in ZIP | `--output`, `--batch-size`, `--redact-text`, `--pid` |
| `live` | Fixed app probes and detailed report | `--suite`, `--output`, `--pid` |
| `desktop-search QUERY` | App search results with message metadata | `--limit`, `--output`, `--pid` |
| `send-self` | One actual self-message and verification receipt | `--output`, `--pid` |
| `webhook` | Listener installation, status, delivery and restoration | See below |
```powershell
python whatsapp.py contacts-export --output exports/contacts.json
python whatsapp.py export-zip --output exports/messages.zip
python whatsapp.py live --suite history-read
python whatsapp.py contacts-export --output contacts.json
python whatsapp.py export-zip --output messages.zip
python whatsapp.py send-self
python whatsapp.py desktop-search "keyword" --limit 10
```
The bundle compresses the original modules and resources together with LZMA.
It verifies its embedded payload and imports Python modules directly from memory.
Only a native helper needed for a live command is materialized in a hash-verified
cache beside the bundle. Keep the containing folder writable. Native helper
bytes, native object checks, batch sizes and message read/send paths are unchanged.
`live --suite` accepts `math`, `capabilities`, `all`, `message-json`, `contacts` and `history-read`. The latter reads the five oldest records after a bounded timestamp scan. It saves decoded body/caption text when present.
Core WebView commands use the standard library. Python 3.11+, Windows x64,
64-bit Python are required for live Desktop operations. Other Desktop versions
are attempted with a compatibility warning; only the recorded version was tested. Optional legacy CDP, Frida and QR tools retain the packages
listed below. Node.js remains optional for JavaScript fixture tests.
## Work with saved exports
The development package below remains the editable source. Rebuild the generated
single-file distribution after changing it:
These commands read existing JSON/JSONL files, without running WhatsApp:
```powershell
python build_whatsapp_bundle.py
python whatsapp.py stats messages.jsonl
python whatsapp.py search messages.jsonl "keyword" --limit 20 --redact-text
```
The build is deterministic and records size and SHA-256 in
`analysis/whatsapp-suite/bundle-build.json`. The build file is not needed on a
machine running the bundle.
Extract `messages.jsonl` from a message ZIP before searching it; the search command does not read ZIP files directly. Search supports `--chat-id` and `--output`. `inventory` lists local storage files and sizes.
The WhatsApp scripts are consolidated in `whatsapp_suite/`, with one CLI and
offline regression tests in `tests/`. Run commands from this workspace.
## Webhook commands
```powershell
python -m whatsapp_suite
python -m whatsapp_suite --help
python -m whatsapp_suite test --output analysis/whatsapp-suite/results.json
python whatsapp.py webhook install --endpoint https://your-endpoint.example/events --self-only --auto-restore --config listener.json
python whatsapp.py webhook status
python whatsapp.py webhook stop --config listener.json
```
`python whatsapp_tests.py` is an equivalent entry point. The default run uses
fixtures and temporary files. It verifies saved-export parsing, Unicode search,
chat filtering, text redaction, history pagination/decryption contracts, fixed
probe responses, command dispatch, and bundled DLL hashes. Node.js runs the
JavaScript history fixtures; that test is reported as skipped if Node is absent.
Python 3.11 or newer is required.
Set the endpoint token in the environment variable named by `--token-env` (default `WHATSAPP_INJECTOR_WEBHOOK_TOKEN`). Never place the token in the URL.
To test an already running WhatsApp Desktop installation:
```powershell
python -m whatsapp_suite test --live
python -m whatsapp_suite test --capabilities
python -m whatsapp_suite live --suite math
python -m whatsapp_suite live --suite capabilities
python -m whatsapp_suite live --suite history-read --output analysis/whatsapp-suite/js-history-read.json
python -m whatsapp_suite export-zip --output exports/whatsapp-messages.zip
python -m whatsapp_suite contacts-export --output exports/contacts.json
python -m whatsapp_suite send-self
```
Live WebView probes require Windows x64 and 64-bit Python. Desktop build
`2.2639.100.0` and its recorded WebView2 Core DLL were tested. Other versions or
DLL hashes produce a warning and are attempted. The helper discovers function and vtable locations from binary fingerprints and checks native
objects and methods before executing JavaScript; changed layouts can
cause failure or process instability. The JSON report includes detected/tested
compatibility metadata and cleanup status. `--pid` selects a Desktop host.
## Tools
Every command supports `--help` for its options.
| Command | Purpose |
| Action | Behavior |
| --- | --- |
| `inventory` | List local Desktop storage files and sizes |
| `stats EXPORT` | Summarize a saved JSON or JSONL export |
| `search EXPORT QUERY` | Unicode-aware literal search of saved messages |
| `desktop-search QUERY` | Bounded native app search through WebView2; metadata output |
| `live --suite math\|capabilities\|all` | Fixed live WebView test suites |
| `live --suite message-json\|contacts` | Explicit fixed message search/contact sample; saves private data |
| `live --suite history-read` | Read the five oldest local message records through the JS database getter and decoder, without CDP |
| `webview-test` | Arithmetic check in a separate WebView2 host; may download its SDK |
| `analyze` | Storage analysis, optionally using an existing loopback CDP endpoint |
| `history-export` | Export local history through Desktop's own database decryption |
| `export-zip` | Export all local message rows to ZIP through JavaScript on WebView, without CDP or UI automation |
| `webhook` | Install/status/stop, direct HTTPS or loopback relay, ensure/auto-restore and optional watch/serve; see [webhook guide](WEBHOOKS.md) |
| `contacts-export` | Export all local contact rows to JSON through the app's JS database getter |
| `send-self` | Send one fixed test message to the signed-in account itself and verify stored plaintext |
| `ram-read` | Read loaded V8 message objects from process memory |
| `ram-search QUERY` | Search RAM models with verified collection membership |
| `native-search [QUERY]` | Experimental injected SQLite search, `--oldest --limit 5`, or `--export-index` |
| `qr-memory` | Recover candidate Desktop pairing QR payloads from RAM |
| `qr-window` | Save an already displayed Desktop linking QR |
| `pairing` | Start an isolated browser and request an official linking QR |
| `install` | Subscribe to message events and start the resident native relay; installer exits |
| `install --restore --config FILE` | Reuse and re-enable an existing configuration |
| `status` | Report listener and relay counters |
| `stop --config FILE` | Stop the listener and disable restoration in that configuration |
| `auto-restore --config FILE` | Register/enable the per-user minute task for a direct HTTPS listener |
| `ensure --config FILE` | Check once and restore a missing configured listener |
| `watch --config FILE` | Optional continuous restoration loop |
| `serve --restore-config FILE --store FILE` | Run the optional loopback receiver with SQLite storage and restoration |
```powershell
python -m whatsapp_suite search export.jsonl "keyword" --redact-text
python -m whatsapp_suite desktop-search "keyword" --limit 10
python -m whatsapp_suite history-export --help
```
Direct installation also accepts `--endpoint`, `--token-env`, `--self-only`, `--auto-restore`, `--config` and `--pid`. Loopback installation uses `--port` instead of `--endpoint`. Full setup, payload and task removal are in [the webhook guide](WEBHOOKS.md).
`history-read` uses the native WebView bridge to execute JavaScript in the running
app. It scans at most 100,000 local record timestamps in a read-only transaction,
then calls `WAWebSchemaMessage.getMessageTable().get` and
`WAWebDBMessageSerialization.messageFromDbRow` for the five oldest records.
It saves decoded body/caption text when present, including messages absent from
the loaded conversation models. System events and media-only records can have
no text. The live result includes page-state and native bridge cleanup status.
This command requires neither CDP nor optional Python packages.
## Output and failure behavior
`export-zip` uses the same native bridge and the app's JavaScript database getter
and decoder. It paginates all local rows up to the initial upper primary key,
streams them into `messages.jsonl`, and includes a `manifest.json` with row counts,
decoding errors, scope, and bridge cleanup status. Rows include message IDs,
chat IDs, timestamps, types, body and caption text; media files are not downloaded.
This is a live scan, not an atomic database snapshot. Row decoding failures are
preserved as error records and make the command return a failure status.
`--batch-size` controls bounded reads (default 100); `--redact-text` omits body and
caption text. Existing outputs are refused. Interrupted exports retain a
`.zip.partial` file and a `.zip.report.json` diagnostic report. A completed ZIP
is checked for CRC errors before publication.
- Commands emit compact JSON summaries; exports and detailed reports are local files.
- Success normally returns exit code 0; reported failures return a nonzero code. `webhook ensure` may exit 0 while waiting for WhatsApp: inspect its status file, not only the exit code.
- ZIP exports contain `messages.jsonl` and `manifest.json`. They cover local rows up to the initial upper key, with no phone sync or media download.
- The scan is not an atomic snapshot. Decode failures remain visible as error records and make the export fail.
- Existing export outputs are refused. Interrupted ZIP exports retain `.zip.partial` and an adjacent `.zip.report.json`.
- `send-self` reserves its receipt before sending. It verifies the recipient, stored plaintext and acknowledgement; an uncertain attempt is not retried automatically.
- Live reports include compatibility and cleanup information. A version warning does not certify support for that build.
`contacts-export` reads the local contact table rather than the five-contact
loaded-model sample. It exports selected identity, name, phone and business
fields. Local records can include separate PN/LID entries and contacts outside
the phone address book; this is not a deduplicated phone address-book export.
Keep private outputs out of Git. See [troubleshooting](TROUBLESHOOTING.md) and [security](../SECURITY.md).
`send-self` resolves the signed-in account and checks both the selected chat and
outgoing message recipient before invoking the app's text-send action. It sends
one identifiable test message and reads it back through the message database
decoder, recording ACK and exact plaintext equality in a receipt. An existing
receipt is refused and sending is never retried automatically. Unlike the
offline test suite, this command actually sends a WhatsApp message.
## Optional research commands
The separate bulk `history-export` command requires an existing Desktop CDP endpoint. RAM tools only see
loaded objects and cannot retrieve all history. Pairing commands are separate
from the automated test run. The `runtime` module preserves the explicit
temporary CDP context manager for research scripts; entering that context
restarts Desktop and is never part of the default tests.
These commands have distinct transports and are separate from the core no-CDP workflow. Always inspect their help before use.
## Optional dependencies
The default suite, saved-export tools, and native WebView probes need only the
Python standard library (plus Node.js for the history fixture test).
| Tools | Additional Python packages |
| Command | Purpose / dependency |
| --- | --- |
| CDP analysis and history export | `websockets` |
| Injected SQLite search | `frida` |
| QR from RAM | `qrcode[pil]` |
| QR from Desktop window | `Pillow`, `zxing-cpp` |
| Isolated browser pairing | `websockets`, `qrcode[pil]`; installed Chrome |
| `webview-test` | Separate WebView2 host test; may download its SDK |
| `analyze` | Storage inspection, optionally an existing CDP session; `websockets` for CDP |
| `history-export` | Bulk history through an existing Desktop CDP endpoint; `websockets` |
| `ram-read`, `ram-search` | Inspect loaded models in RAM; not complete history |
| `native-search` | Experimental injected SQLite search/index export; `frida` |
| `qr-memory` | Pairing QR candidates in RAM; `qrcode[pil]` |
| `qr-window` | Capture an already displayed pairing QR; `Pillow`, `zxing-cpp` |
| `pairing` | Isolated browser linking QR; Chrome, `websockets`, `qrcode[pil]` |
Install all optional packages into a virtual environment with:
```powershell
python -m pip install -r requirements-whatsapp-optional.txt
```
## Layout and migration
- `whatsapp_suite/`: canonical tool implementations, common normalization,
storage paths, Desktop discovery, CLI, and test runner.
- `whatsapp_suite/assets/`: verified native DLLs and the native SQLite agent.
- `tests/`: synthetic Python regressions and the JavaScript history fixture.
- `analysis/whatsapp-suite/results.json`: latest automated test result.
The former `whatsapp_tool.py` commands now use `python -m whatsapp_suite`;
`whatsapp-search` is named `desktop-search`, and `whatsapp-test` is
`live --suite math`. The former `whatsapp_math.py` options are available under
`live`. Archived scripts have become package modules; the identical Desktop QR
RAM scripts have one implementation (`qr_memory`). Research helpers now import
these modules. Old script archive and portable script copies were removed.
The clean Git project excludes research reports, extracted application code and browser profiles. The surrounding original workspace retains those local files.
`requirements-whatsapp-optional.txt` lists optional Python packages. No optional research action, pairing operation or send is performed by default tests.
docs/PUBLISHING.md
+19 -10
View File
@@ -1,17 +1,26 @@
# Publish to tea.karlcom.de
# Publishing
This directory is the clean repository root. The surrounding research workspace is not the repository and must not be uploaded.
The canonical repository is [karl/whatsapp_injector](https://tea.karlcom.de/karl/whatsapp_injector). Publish from its clean Git root, not the surrounding research workspace.
1. The configured repository is `https://tea.karlcom.de/karl/whatsapp_injector.git`.
2. Copy the SSH or HTTPS clone URL supplied by the server. Use your existing credential manager or SSH configuration; never embed tokens in the URL.
3. From this directory, inspect the tracked file list and configure the remote:
## Upload a reviewed change
Rebuild and test when source, assets or tests changed; use the [development workflow](../CONTRIBUTING.md). Documentation-only changes need accurate examples and valid links.
```powershell
git ls-files
git remote add origin https://tea.karlcom.de/karl/whatsapp_injector.git
git push -u origin main
git status --short
git diff --cached
git remote -v
git push origin main
```
The owner and remote URL were supplied by Karl. Repository visibility is controlled on the server. Never push the surrounding research workspace.
Stage only maintained source, documentation, synthetic fixtures and required runtime assets. Review staged files before committing. Exports, profiles, credentials, protected listener configurations and research captures belong outside Git.
For a release, rebuild and test the bundle, then attach only `whatsapp.py` and a SHA-256 checksum to a server release. State the tested Desktop build and optional dependencies. Do not attach test exports or research reports. No server-specific CI workflow is included because runner availability and configuration have not been verified.
An existing clone already has its remote configured. If moving a repository to a different server, configure the intended URL explicitly; never embed a token in it. Use an existing credential manager or SSH setup.
## Portable release
Build and test `whatsapp.py`, then attach that file and its SHA-256 checksum to a release. Include the tested Desktop build, Python/platform requirements, relevant live validation and untested cases.
The runtime file contains the native DLLs; users need no compiler. It must run from a writable folder. Do not attach private test outputs. Embedded hashes establish integrity, not a publisher signature.
Repository visibility is controlled on the server. No open-source license has been selected; see [licensing status](../LICENSE.md). No server-specific CI workflow is included because runner availability has not been verified.
docs/TROUBLESHOOTING.md
+22
View File
@@ -0,0 +1,22 @@
# Troubleshooting
Start with `python whatsapp.py --help` or the specific command's `--help`. Commands report compact results; private detailed reports are written locally.
| Symptom | What to check |
| --- | --- |
| No running Desktop host found | Start the Windows WhatsApp app normally, sign in, and wait until it has loaded. Live commands require x64 Windows and 64-bit Python. |
| Unsupported layout or compatibility warning | A warning means the build is untested; discovery still attempts it. A changed fingerprint/ABI can fail. Keep the detected version and a redacted report for investigation; do not force guessed offsets. |
| Access denied or helper extraction fails | Keep the portable file in a writable folder. Check local security-software diagnostics before changing permissions. Running as administrator is not a routine requirement. |
| Existing output/receipt/configuration | Choose a new export filename. Use `webhook install --restore --config listener.json` only when reusing that listener configuration. Inspect an uncertain send receipt before retrying. |
| JavaScript tests skipped | Install Node.js and rerun the suite if complete fixture coverage is needed. A passing test run can still contain skips. |
| Webhook token rejected | The direct token must contain 32–128 printable ASCII characters. Supply the named environment variable and ensure the target validates `X-Webhook-Token`. |
| Direct webhook queue grows | Check HTTPS connectivity, certificates, token authentication and the target's HTTP status. Non-2xx responses leave records queued for retry. The relay disables proxies and redirects. |
| Listener disappears after WhatsApp restart | Process exit destroys injected code. Enable `--auto-restore`, keep registered file paths stable, and allow up to about a minute after the app has loaded. |
| Restore fails after moving configuration | DPAPI protection is tied to the Windows user/machine. Reconfigure with the endpoint token on the intended user/machine. |
| Stop seems temporary | A restoring configuration can re-enable a listener. Use `webhook stop --config listener.json`; for full removal follow the task-removal steps in the [webhook guide](WEBHOOKS.md). |
| ZIP contains fewer messages than expected | The export covers locally available rows, not guaranteed full account history or media downloads. Inspect `manifest.json` and the adjacent report. |
| Export is interrupted | Keep the `.partial` and report for diagnosis. A new attempt requires a new filename; partial files are not resumed. |
For an automatically restored listener, the configuration records the exact task name. The adjacent `listener.status.json` reports the last scheduled check; Task Scheduler exposes its last result and next run. A successful scheduler exit alone does not prove message delivery.
When reporting a problem, include the command, Python/app versions and a redacted error/cleanup summary. Do not attach real exports or protected configuration. See [security](../SECURITY.md).
docs/WEBHOOKS.md
+52
View File
@@ -1,5 +1,14 @@
# Message webhooks
Use this guide to connect a message listener to your HTTP consumer. Start with direct HTTPS if you already have a webhook endpoint; the local receiver is optional. The listener observes messages surfaced in the Desktop app, rather than subscribing to WhatsApp's server history.
## Choose a delivery mode
| Mode | What runs locally | Use it for |
| --- | --- | --- |
| Direct HTTPS | Listener/relay in WhatsApp; optional brief scheduled Python checks | Your existing webhook service or Node-RED |
| Loopback HTTP | Listener/relay plus a running Python receiver | Local testing and durable SQLite capture |
## Execution and restart behavior
`webhook install` is a one-shot command. JavaScript subscribes to message collection events inside WhatsApp and queues decoded records. A resident native helper posts them directly to an HTTPS endpoint or to a loopback HTTP receiver. Direct HTTPS needs no local server and no continuously running Python process. No CDP or UI automation is used. WhatsApp's CSP blocks direct JavaScript HTTP requests to localhost, so delivery uses native WinHTTP without weakening CSP.
@@ -10,8 +19,17 @@ A WhatsApp process restart destroys injected code. For direct HTTPS, optional `-
For direct HTTPS, supply a 32–128 character printable ASCII token through the named environment variable using your secret manager, then run:
If you are setting it manually, this PowerShell prompt hides the input and avoids placing the token in the command text:
```powershell
$env:WHATSAPP_INJECTOR_WEBHOOK_TOKEN = [System.Net.NetworkCredential]::new('', (Read-Host 'Webhook token' -AsSecureString)).Password
```
Install the listener:
```powershell
python whatsapp.py webhook install --endpoint https://your-endpoint.example/events --token-env WHATSAPP_INJECTOR_WEBHOOK_TOKEN --self-only --auto-restore --config listener.json
Remove-Item Env:\WHATSAPP_INJECTOR_WEBHOOK_TOKEN
```
The endpoint must authenticate `X-Webhook-Token` and accept the batch schema below. Certificate validation is enabled; credentials in URLs, query strings, fragments and redirects are rejected. Remove `--self-only` only when you intend to forward events from other chats. `webhook auto-restore --config listener.json` registers/enables restoration separately; `webhook ensure --config listener.json` performs one check. `webhook watch --config listener.json` is an optional continuously running alternative.
@@ -60,6 +78,40 @@ The loopback endpoint is `POST /whatsapp-webhook` on `127.0.0.1`; direct HTTPS u
HTTP 2xx means the batch was accepted. The built-in receiver commits before acknowledgement and deduplicates message IDs. Failed HTTP delivery leaves queued records available for retry. Consumer side effects such as AI jobs should also deduplicate IDs. There is no exactly-once guarantee.
### Example batch
This is synthetic data. Message IDs are opaque strings; use `event_id` for deduplication rather than parsing its format.
```json
{
"listener_id": "91b17841-2106-4e73-97db-c38be6e9b269",
"seqs": [1],
"events": [{
"seq": 1,
"event_id": "synthetic-message-1",
"type": "message",
"message": {
"id": "synthetic-message-1",
"chat_id": "synthetic-chat",
"timestamp": 1700000000,
"from_me": true,
"message_type": "chat",
"body": "Hello",
"caption": null,
"text_truncated": false,
"body_length": 5,
"caption_length": 0
}
}]
}
```
`timestamp` is Unix seconds; sequence numbers identify queue entries within the current listener installation. Body/caption may be null. Text lengths count JavaScript UTF-16 code units. Batches contain up to 16 events and are additionally bounded by serialized size.
Your endpoint should authenticate `X-Webhook-Token`, validate the batch, deduplicate IDs, persist or enqueue accepted work, and only then return HTTP 2xx. No response-body schema is required by the relay. Use a non-2xx status when a retry is needed. TLS certificates must validate, and redirects are not followed.
For a summarizer, pass newly accepted message events to your own processing queue. This repository supplies delivery, not an AI provider, prompt, response policy or automatic message sender.
## Bounds and limitations
- This listens to the app's loaded message collection, not a server-side subscription. Messages not surfaced there can be missed.