142 lines
9.6 KiB
Markdown
142 lines
9.6 KiB
Markdown
# Development observability
|
|
|
|
Run `npm run debug:collector` and verify `http://127.0.0.1:4318/health`.
|
|
Start the management service with `OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318`.
|
|
Enable browser diagnostics through a normally published signed policy: `enabled: true`,
|
|
`mode: debug`, endpoint above, and debug sampling of 1 for controlled reproduction.
|
|
Administrators can disable diagnostics centrally; mandatory review still applies.
|
|
|
|
Reproduce the feature, inspect `/api/summary` for slow stages and inspect
|
|
`/api/trace/<traceId>` for parent/child timings. Browser management requests carry
|
|
W3C trace context into backend HTTP spans. AI inspection, provider requests,
|
|
response reading/decoding and library organization have separate backend stages.
|
|
Use the native tabs smoke fixture to verify real Electron exports and correlation.
|
|
|
|
The receiver listens only on numeric loopback, rejects foreign origins and hosts,
|
|
and stores encrypted batches under ignored `data/observability`. AES-256-GCM keys
|
|
are protected by Windows DPAPI. The summary contains bounded timing metadata;
|
|
it is a local developer tool accessible to the same machine's users, not a
|
|
production SIEM or an authenticated remote service. Recent trace lookup is held
|
|
in memory and resets on restart. Each encrypted segment has a 50 MiB development
|
|
limit. The collector retains an active segment and at most two generated retired
|
|
segments, plus a separately pinned previous archive. Inspect `activeBytes`,
|
|
`legacyBytes`, `retiredBytes` and `rejectedWrites`; investigate collection failures
|
|
instead of assuming exports succeeded. See [retention](development-collector-retention.md).
|
|
|
|
Ordinary traces never contain passwords, tokens, prompt bodies or document text.
|
|
Security audit and restricted encrypted forensic evidence are separate pipelines.
|
|
Debug mode now exports eight internal inspector stages: inspect, parse, OCR, OCR model initialization, YARA, macro extraction, semantic inference and semantic model initialization. The worker sends bounded in-memory timing records through its existing private pipe; it gains no network capability. Browser/backend import creates child spans with original timestamps under the active request trace. Invalid/off/stale metadata is discarded without changing the verdict. These named stages do not trace every function or native-model kernel. Add named stages when investigating further gaps, without weakening DLP or retrying protected actions.
|
|
|
|
The repository's AGENTS.md requires this workflow for subsequent development.
|
|
|
|
## Coverage verification, 2026-10-13
|
|
|
|
The live collector reported 3,147 spans, 2,482 log records and 36 metric records,
|
|
with zero rejected storage writes. Twenty focused diagnostics, OTLP export,
|
|
inspector timing, collector validation/retention and failure-path tests passed.
|
|
These counts are a process snapshot, not a completeness claim.
|
|
|
|
Agent RPC instrumentation now wraps each request in `agent.request`; decrypt,
|
|
authorization, audit, dispatch, projection and encryption run beneath that parent.
|
|
The browser diagnostic operation allowlist accepts the new name. A real protected
|
|
RPC test verifies asynchronous parent/child nesting without recording arguments
|
|
or response content. The combined transport/diagnostics suite passes 38 tests;
|
|
A subsequent actual Electron run verifies exported parent/child phases linked to an attributable durable agent event; Windows packaged verification remains a separate gate.
|
|
|
|
The actual native agent audit proof links `browser.agent` to
|
|
`browser.management` and `server.request` under trace
|
|
`f8e35fd442e718995d9274c646fb8e08`. The native upload helper-failure proof links a
|
|
recording `browser.upload.fault` span, its lease parent, two logs and the durable
|
|
audit event under trace `e6d26d4edf738a7b4f6f562ab2112f5b`. Older traces may be in
|
|
the protected pinned archive rather than the current in-memory index.
|
|
|
|
A sampled slow `browser.parser` scan lasted 1,049 ms but contained only the outer
|
|
span in the live index. Do not infer which parser or model stage caused that
|
|
duration from an outer span alone: reproduce with signed debug mode enabled and
|
|
verify the expected inspector children before attributing a performance issue.
|
|
|
|
The current individual native proof links principal `reader-a` to
|
|
trace `3a0c31cbf53d7808e39a91daf2aaa3d7`. Its request took 81.019 ms,
|
|
the two durable audit phases totaled 77.598 ms and adapter dispatch took
|
|
0.313 ms. This single local run points to audit persistence as the dominant
|
|
stage; it is not a throughput benchmark. Preserve pre-execution durable audit
|
|
when investigating latency. Do not optimize by removing an audit gate or retrying
|
|
an action. The proof is in `artifacts/individual-agent-native-proof.json`.
|
|
The final Windows package verifier checks current source and runs both shipped
|
|
legacy and individual DPAPI/PQ clients successfully.
|
|
|
|
## Latest interaction and session measurements
|
|
|
|
The actual protected interaction proof uses trace
|
|
`32cbbb58088d193301c5f6ac530284af`: 30 spans, verified parent correlation,
|
|
capture 0.3353 ms, challenge 8.1735 ms, encryption 0.4718 ms, send 62.1817 ms
|
|
and acknowledgement 0.4234 ms. The backend store stage took 54.2629 ms.
|
|
Collector rejected writes were zero. These are one local reproduction's
|
|
measurements, not performance guarantees.
|
|
|
|
After session-title classification was added, the actual native session inventory
|
|
proof uses trace `64013e27cfc37bc3792983bc06149c87`: request 85.675 ms,
|
|
session adapter 1.0637 ms and durable audit phases 33.3975 and 47.2472 ms.
|
|
This fixture did not enable the semantic model; it cannot establish semantic-model
|
|
latency. The native proof confirms a confidential title is masked while preserving
|
|
its opaque session ID.
|
|
|
|
A parallel fixture timed out before enrollment and therefore before signed
|
|
diagnostics activation. Its isolated rerun and restart reached the shell in
|
|
794/806 ms. Safe local startup snapshots were added; the original timeout's cause
|
|
is unresolved. Do not describe that failure as a traced DLP/model regression.
|
|
The current Windows package proof and 515-test suite passed before the subsequent
|
|
export-counter refinement. Ignored proof artifacts remain local.
|
|
|
|
After the refinement and policy child stages, the final suite passes 522 tests.
|
|
Real signed policy/bookmark fixtures verify actual OTLP child spans and disabled
|
|
transitions. Windows build, packaged-runtime verification and the native tabs
|
|
proof pass. The native receiver attributes exports by actual `service.name`:
|
|
browser exports stay at 18 across revocation while backend collection remains
|
|
independent. The earlier aggregate assertion conflated the two services; its
|
|
failed run is retained rather than presented as successful. Do not retroactively
|
|
assign the original extra batch to a service without its original payload.
|
|
|
|
Foreground gesture development used actual traces to find a second attribution
|
|
gap: a queue flusher started by one event delivered the next event under the first
|
|
async context. `withCorrelation` now validates each durable record's own IDs and
|
|
`audit.deliver` creates a recording delivery child under that source parent.
|
|
Malformed IDs clear unrelated context, and current signed sampling/disable still
|
|
applies. Real OTLP regression tests and three actual human-event traces prove
|
|
distinct parents and complete evidence phase correlation. Trace
|
|
`b7783f42197d0cff83957b0ee96e8296` has 17 spans, capture 19.50 ms and send 13.20 ms;
|
|
`62e09413b9a42f4a58bd7f99485176ac` and
|
|
`545ddac025113e50effed4e9a2ac87ee` each also have 17 spans. Collector rejected writes
|
|
were zero. These are local reproduction timings, not throughput guarantees.
|
|
|
|
On 2026-10-13, the running protected collector and the actual exported trace
|
|
`fd47bcfc8b259712101ad3c81c1d95c1` were inspected again. The request took
|
|
724.3133 ms, its first audit phase 682.6577 ms, and dispatch 2.0827 ms.
|
|
Its correlated audit delivery took 52.7383 ms. The audit phase alone does not
|
|
establish whether serialization, DPAPI, filesystem synchronization or scheduling
|
|
caused the delay. New synchronous persistence spans separate `audit.serialize`,
|
|
`audit.encrypt`, `audit.write`, `audit.fsync` and `audit.commit`, without changing
|
|
the synchronous persistence boundary or retrying a write. Only fixed operation
|
|
identifiers and timings enter diagnostic attributes; no audit payload or paths.
|
|
Previously collected traces cannot supply these newly added stages.
|
|
|
|
The subsequent actual Electron vault fixture passed with the added stages.
|
|
Trace `e98cac3e18c6bd7b0e0442f033a7fa2d` exports all five persistence
|
|
children under `agent.audit`: serialization 0.0544 ms, encryption 0.0501 ms,
|
|
write 7.2461 ms, fsync 2.3703 ms and close/rename 14.7423 ms. These timings
|
|
describe this new run; they do not resolve the previous 683 ms observation.
|
|
The five-stage parentage was checked against the live collector for both list
|
|
and fill requests, with zero rejected writes. Twelve existing diagnostics tests
|
|
and four dedicated real OTLP tests pass, including central disable, secret
|
|
redaction, synchronous return/throw preservation and no replay on instrumentation
|
|
failure. At that stage, session adapter exports were proven in a fixture;
|
|
native browser restoration required the subsequent integration proof below.
|
|
|
|
The actual two-process Windows session proof now exports save/list/restore stages
|
|
and their audit persistence children. A delayed HTTPS receiver also verifies
|
|
that manually switching away and back aborts restoration, preserves existing tabs
|
|
and never requests the second saved destination. Trace
|
|
`583a1d4dbc80fb2cbcb135b1622a4993` records the failed native stage at 1410.0137 ms
|
|
and owned-tab rollback at 0.7242 ms. The delay was intentional fixture behavior;
|
|
it is not a production performance result. Collector rejected writes were zero.
|