Http Toolkit Intercept

HTTP Toolkit Intercept

Use this skill when you need authoritative evidence of what your program sent to a remote API and what it received back while verifying a code change. Works across any runtime — Node.js, Bun, Deno, Python, Go, Ruby, Java/JVM, .NET, PHP, Rust, shell scripts, etc. — as long as the process respects a proxy.

The reliable pattern is:

1. Start HTTP Toolkit correctly.
2. Run the program through the proxy in a mode that produces a machine-readable log (e.g. --output-format json, debug logging, or structured stdout).
3. Export outbound HTTP requests from HTTP Toolkit.
4. Pair the outbound HTTP export with the inbound program session log.

Do not rely on TUI screenshots alone when the question is about request payloads, auth headers, or wire-level behavior.

Prerequisites / Known-Good Launch

Start HTTP Toolkit

On Linux/headless environments, plain httptoolkit often fails due to sandbox/X11 issues. Prefer:

bashCopy code
xvfb-run --auto-servernum httptoolkit --no-sandbox

If a stale server is already running on ports 45456/45457, stop it first:

bashCopy code
pkill -f "HTTP Toolkit Server|httptoolkit|xvfb-run --auto-servernum httptoolkit" || true

Verify the proxy is reachable

bashCopy code
# HTTP Toolkit's admin API lives on port 45456/45457 by default.
# Treat 200/401/403 as "reachable"; only connection failure means the server is dead.
curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:45456/config

Quick Start

1. Launch your program with the proxy env vars set

The canonical env vars most runtimes honor:

bashCopy code
HTTP_PROXY="http://127.0.0.1:8000" \
HTTPS_PROXY="http://127.0.0.1:8000" \
ALL_PROXY="http://127.0.0.1:8000" \
NO_PROXY="" \
<your-program> <args>

Some runtimes require an extra env var or flag — see the "Runtime proxy matrix" below.

If the HTTP Toolkit CA is not trusted by your runtime, TLS verification will fail. See "TLS Safety" below. Disabling TLS verification is appropriate only for controlled local debugging.

2. Capture the inbound session log

If your program supports a machine-readable output mode (e.g. --output-format stream-json, --json, --log-level debug, structured stdout, or writing to a file), pipe it to a file:

bashCopy code
<your-program> exec --output-format stream-json "your input" \
  > /tmp/session-stdout.log 2> /tmp/session-stderr.log

3. Export outbound HTTP from HTTP Toolkit

Either:

• Use the HTTP Toolkit GUI export (File → Export → JSON / HAR), or
• Hit the admin API directly. The exact endpoint depends on your HTTP Toolkit version; inspect DevTools in the HTTP Toolkit UI to see the requests it makes.

4. Cross-reference the two streams

• Outbound HTTP (from HTTP Toolkit): authoritative for request bodies, headers, auth tokens, retry timing.
• Inbound session log (from the program): authoritative for how your code reacted to the responses.

Together they answer: "what did we send?" and "what did we do with the response?"

What finally worked for payload verification

The critical correct pathways that proved reliable were:

1. Use a non-interactive / exec mode, not the TUI, when verifying payloads

   • Interactive TUIs are slower and much harder to analyze.
   • Use whatever your program has for scripting (--output-format, --json, --headless, --batch).

2. Treat outbound and inbound as separate evidence sources

   • HTTP Toolkit gives outbound HTTP requests.
   • The program's session log gives inbound assistant/tool/application behavior.
   • You need both to answer: "what did the remote return?" and "what did the program actually do with it?"

3. Set the proxy env vars your specific runtime honors

   • HTTP_PROXY/HTTPS_PROXY cover most runtimes, but some (e.g. Bun, Java) require extra vars or flags.
   • See the "Runtime proxy matrix" below.

4. Disable TLS verification only for controlled local debugging when needed

   • If the HTTP Toolkit CA is not trusted locally, use the runtime-specific escape hatch to skip verification.
   • Prefer trusting the CA in your OS / language trust store instead.
   • Never disable TLS in production repros.

5. Keep runs bounded

   • Long network-heavy sessions can take time.
   • If you only need to prove request shape, export after the relevant request is observed — you do not always need to wait for full completion.

Key Facts

• Proxy env vars are runtime-specific — know which ones your runtime honors
• HTTP Toolkit admin API is request-oriented — outbound HTTP comes from HTTP Toolkit, inbound behavior comes from the program log
• TLS is verified by default — runtime-specific skip flags are local-dev escape hatches only

Runtime proxy matrix

Prefer trusting the HTTP Toolkit CA in your runtime/OS trust store over disabling verification. Bypass flags should be local-dev only.

TLS Safety Guardrails

• Keep TLS verification enabled whenever possible.
• Prefer trusting the HTTP Toolkit CA in your local trust store instead of disabling verification.
• Use runtime-specific TLS-bypass flags only for controlled local debugging in development.
• Never disable TLS when intercepting production traffic.

Inspecting Captured Logs

Filter to relevant events only

If your program emits newline-delimited JSON, use jq:

bashCopy code
jq -c 'select(.type == "tool_call" or .type == "message")' /tmp/session-stdout.log

Adjust the filter to match your program's event schema. For plain-text logs, use rg/grep patterns.

Match outbound HTTP to inbound events

Sort both streams by timestamp, then interleave them. The sequence usually looks like:

Copy code
outbound POST /api/endpoint        (from HTTP Toolkit)
inbound  event received             (from program log)
inbound  follow-up action           (from program log)
outbound POST /api/endpoint        (next request)

If a program log shows an outbound request that HTTP Toolkit didn't capture, that's a proxy-config bug.

Troubleshooting