Expand description
Utilities for the wire-conformance harness.
Intentionally separate from forge-sdk so the SDK crate surface
stays focused on what library consumers use. Nothing here is public
SDK API.
§Two testing modes
Each scenario runs in one of two modes:
- Live mode (
FORGE_WIRE_CAPTURE=1): spawn realclaude, drive forge-sdk through the scenario, capture the full stdin/stdout trace totarget/wire-traces/. Burns API tokens. Updates the committed baseline on request. - Replay mode (default): load the committed baseline trace
fixture for the scenario, feed every inbound line through
forge_sdk::transport::codec::decode_dispatch, assert everything decodes cleanly (including noDecodedLine::Unknownvariants unless explicitly expected). Runs on everycargo check/just check- no API cost.
§Baselines
Committed baselines live under crates/forge-test-harness/baselines/sdk/ <cli-version>/<scenario>.jsonl. Each scenario knows its own name;
the cli-version dir rotates when we bump the pinned CLI version
through the upgrade ritual.
Re-exports§
use std::sync::Arc;🔒use forge_sdk::Error;🔒use forge_sdk::OptionsBuilder;🔒use forge_sdk::transport::codec::DecodedLine;🔒use forge_sdk::transport::codec::decode_dispatch;🔒use parking_lot::Mutex;🔒
Modules§
- session_
redact - Session-persistence → stream-json transformer + PII redactor.
Structs§
- Decode
Report - Outcome of running every inbound line in a log through
decode_dispatch. Split across known decode categories plus Unknowns and outright errors. - Trace
Log - One captured line in a trace.
Enums§
- Dump
Kind 🔒 - What a dump is for, deciding its filename prefix AND whether it is
redacted together, so the two cannot disagree:
README.mdpromotes withcp target/wire-traces/capture-*.jsonl, so anything that glob reaches must be redacted.
Constants§
- CATCH_
ALL_ 🔒MARKERS - Serialised forms of the catch-all variants that stand in for a wire
value the decoder does not model (
#[serde(other)]). - EXPECTED_
GENERIC_ SYSTEM_ SUBTYPES systemsubtypes the decoder deliberately leaves generic. Everything else reachingMessage::Systemis a subtype nobody has modelled yet.- PINNED_
CLI_ VERSION - Pinned CLI version these baselines were captured against.
Functions§
- attach_
recording - Install inbound + outbound wire-tee callbacks on a builder so the
resulting
forge_sdk::Optionscaptures every stream-json line the SDK exchanges with theclaudesubprocess. Returns the configured builder + a shared handle to the trace log. Wire capture is a spawn-time configuration concern, not a transport-injection one. - baseline_
dir - Directory holding the committed trace baselines for the pinned CLI
version. Resolves to
crates/forge-test-harness/baselines/sdk/<PINNED_CLI_VERSION>/. - collect_
catch_ 🔒all_ drift - Walk
rawandreencodedtogether, recording every shared key whose value the decoder replaced with a catch-all marker. - decode_
all_ inbound - Run every inbound line from
logthroughdecode_dispatch, returning a categorised report. - legacy_
baseline_ dir - Directory holding the legacy-surface corpus for the pinned CLI
version: the same scenarios captured with the CLI’s ambient routing
on a model id it does not recognize. The CLI tailors its tool surface
per model, so this corpus records the surface a proxy-backed account
actually gets. That surface is neither identical to the recognized
one nor a superset of it: measure both rather than assuming the older
“unrecognized ids keep the full legacy surface” story still holds.
Resolves to
baseline_dir()’slegacy-surface/subdirectory, which keeps the two corpora out of each other’s diffs. - load_
baseline - Load a trace fixture by scenario name.
- load_
baseline_ from load_baselineagainst an explicit baseline directory - the legacy-surface corpus lives in a sibling directory of the pinned one and is loaded the same way.- record_
fallbacks 🔒 - Record every fallback the decoder took on
msg, which decoded fromline. - run_
live_ scenario - Run a live-capture scenario end-to-end: build options, spawn a recorded
claude, drive the scenario to aResultframe, dump the trace totarget/wire-traces/, assert every inbound line decodes cleanly.