Ring:Reviewing Operational Risk

Operational Risk Review

When to use

• Preparing a service for production and want to know what breaks when a flow gets stuck
• After a dev-cycle: pressure-test the newly built flows for recovery gaps
• Incident retro: formalize which failure modes have a rescue path and which do not
• You need operational runbooks or a backlog of "missing rescue mechanism" gap specs

Skip when

• Prototype / throwaway PoC not heading to production
• Pure library or SDK with no integration boundaries (no external calls, queues, or webhooks)
• Single-question check (use a targeted read instead of the full review)

Related

Complementary: ring:auditing-production-readiness (broad readiness scoring), ring:mapping-service-resources (resource inventory), ring:running-dev-cycle (optional end-of-cycle hook)

What this produces

For every failure scenario, a tier and an actionable artifact:

Audience

The output is always written for the developer running the skill (tech lead or engineer). Runbooks assume operator access; gap specs assume backlog ownership.

How this skill runs: a hybrid (mechanical + judgement) flow

The review is split into two phases so the deterministic work is not left to the LLM:

Phase 1 — mechanical (scan-integration-points.mjs, run by the dev): A zero-dependency Node.js script traverses the target repo and finds integration boundaries (external HTTP calls, queue consumers, event publishers, outbound webhooks). For each point it heuristically records whether retry, DLQ, timeout, rollback/compensation, and idempotency patterns appear nearby. It emits a structured JSON report. It is generic: it runs on any Go or TypeScript/Node.js Lerian repo. This phase is repeatable and produces the same map every time.

Phase 2 — judgement (this agent, from here on): The agent takes the JSON as structured context, runs the confirmation dialogue, simulates stuck states, classifies Tier 1/2/3, and writes runbooks (T2) and gap specs (T3). This is the analysis that needs a human-in-the-loop and cannot be reduced to regex.

The developer runs the .mjs first and pastes/attaches its JSON output to the agent before the dialogue begins. In Mode A the agent uses that JSON as the boundary map instead of re-deriving it by hand. See Step 1 (Mode A).

Step 0: Determine entry mode

Ask the developer (or infer from context):

• Mode A — Codebase explore: review an existing service by scanning its integration boundaries.
• Mode B — Plan context: review flows just built in a dev-cycle by reading plan.md and the current cycle's epic artifacts, without exploring the whole repo.

If a plan.md (ring:writing-plans format) with an active cycle is present and the developer wants to review what was just built, prefer Mode B. Otherwise use Mode A.

Step 1 (Mode A): Map integration boundaries — run the scanner first

The boundary map is produced mechanically by the script, not by hand. The developer runs it against the target repo and gives the JSON to the agent:

bashCopy code
# From the target repo root (any Go or TS/Node.js Lerian service):
node /path/to/reviewing-operational-risk/scan-integration-points.mjs . --out ops-risk-scan.json
# then paste/attach ops-risk-scan.json to the agent before the dialogue.

Monorepo / hexagonal Go — do NOT scan only the service subdir. In a hexagonal layout the outbound boundaries live in the imported pkg/* (or shared adapter) packages, not under apps/<svc>. Scanning only apps/<svc> returns a false 0. The scanner defends against this: given a subdir it walks up to the repo/module root, scans the whole tree, and attributes each boundary to its service via top-level dir. Always run from the repo root (or let the scanner expand to it) so pkg/* adapters and ports/out methods are included. If you must scan a single subdir, pass --no-repo-root and expect the map to miss shared adapters. Whenever integration_points == 0 for a service that imports ports/out, the scanner emits an explicit warnings[] entry — treat it as a scope error, not a clean bill of health.

The scanner is port-aware and adapter-aware: each method of a ports/out interface is treated as a boundary candidate (category: "outbound_port"), and its resilience is inferred over the whole concrete adapter file — so an SDK adapter (e.g. midazsdk.WithTimeout, DoWithRetry) that never calls net/http directly is still detected. Resilience for Go line-matches is inferred over the enclosing function, not a fixed ±N-line window, so a retry/timeout wrapper elsewhere in the function still counts.

Optional per-repo config (.ops-risk.json) at the repo root tunes detection: shared_adapter_dirs, http_wrapper_packages, outbound_port_globs, and exclude_handler_globs. The scanner auto-generates a default on first run if none exists (pass --no-gen-config to skip). Edit it to match the repo's layout before a serious review.

The script emits ring.ops-risk.integration-scan.v1 JSON:

jsoncCopy code
{
  "scan_root": "/repo", "requested_target": "/repo/apps/svc",
  "warnings": [ { "level": "warn", "code": "zero_boundaries_in_port_service",
                 "service": "apps/svc", "message": "0 fronteiras num serviço que importa ports/out ..." } ],
  "summary": {
    "files_scanned": 210, "integration_points": 102,
    "by_category": {...}, "by_service": {...}, "files_by_top_dir": {...},
    "services_importing_ports_out": ["apps/svc"], "http_wrapper_packages_detected": ["httpclient"]
  },
  "integration_points": [
    { "category": "http_outbound", "direction": "outbound", "language": "go",
      "service": "apps/svc", "file": "internal/x.go", "line": 156, "snippet": "...",
      "resilience": { "retry": false, "dlq": false, "timeout": true,
                      "rollback": false, "idempotency": false } },
    { "category": "outbound_port", "direction": "outbound", "language": "go",
      "service": "pkg", "file": "pkg/midazadapter/adapter.go", "line": 42,
      "port": { "interface": "PaymentPort", "method": "Charge", "declared_in": "apps/svc/ports/out/gateway.go" },
      "resilience": { "retry": true, "timeout": true, "dlq": false, "rollback": false, "idempotency": false } }
  ],
  "resilience_gaps": [ { "file": "...", "line": 156, "category": "...", "service": "...", "missing": ["retry","dlq"] } ]
}

Always read warnings[] first: a zero_integration_points or zero_boundaries_in_port_service warning means the map is probably incomplete (scope error) and must not be treated as "no boundaries exist".

The agent consumes this JSON as the starting boundary map. Treat every hit as a candidate and every false resilience flag as a prompt to verify, not a confirmed gap — the regex scan is deterministic but heuristic. The focus stays on what the service expects to receive and how it reacts when it does not — do NOT leave the repo to inspect dependencies.

If the script cannot be run (no Node.js, restricted env), fall back to manual greps for the same boundaries — run these from the repo root and cover the same root set as the scanner (internal/ components/ pkg/ src/ apps/ cmd/ services/ plugins/), not just the service subdir:

bashCopy code
grep -rn "http.NewRequest\|http.Client\|resty\|req.Get\|req.Post\|Do(ctx\|httpclient.\|DoWithRetry\|New.*Client(" internal/ components/ pkg/ apps/ cmd/ services/ plugins/ 2>/dev/null
grep -rn "axios\|fetch(\|got(\|undici" internal/ components/ pkg/ src/ apps/ cmd/ services/ plugins/ 2>/dev/null      # TS
grep -rn "Consume(\|Subscribe(\|HandleDelivery\|amqp\|rabbitmq\|sqs\|kafka" internal/ components/ pkg/ src/ apps/ cmd/ services/ plugins/ 2>/dev/null
grep -rn "Publish(\|Produce(\|NotifyURL\|callbackURL\|webhookURL" internal/ components/ pkg/ src/ apps/ cmd/ services/ plugins/ 2>/dev/null
grep -rln "ports/out\|port.[A-Z].*Port" internal/ components/ apps/ cmd/ services/ plugins/ 2>/dev/null   # find hexagonal outbound ports, then read their adapters in pkg/

For each integration point, confirm the resilience posture (the script pre-fills these flags; verify them against the code):

Produce a boundary map: {integration_point, direction, entities_touched, resilience: {retry, rollback, dlq, timeout, idempotency}}.

Step 1 (Mode B): Extract the flow from the plan

Read the cycle's plan.md and the epic artifacts of the current cycle only:

• ## Phase Overview + the active phase's ### Epic N.M: sections → the flows built this cycle
• Each epic's task blocks → the entities created/mutated and the transitions between them
• Any linked design docs (data model, API contracts) referenced by the epics

Produce, per flow: {flow_name, entry_point, entities[], state_transitions[], terminal_state}. Do not scan the whole repo — the plan is the source.

Step 2: Confirmation dialogue with the developer

Before analysing failures, confirm the model out loud and get agreement. In Mode A, drive this dialogue from the scanner JSON — walk the developer through the integration_points and the resilience_gaps the script surfaced, and let them correct false positives/negatives before you classify anything:

Copy code
For flow "<flow_name>":
  Entry point:      <e.g. POST /transfers>
  Terminal state:   <e.g. transfer.status = SETTLED>
  Entities & states: <entity: [state1 → state2 → state3]>
  Dependencies:     <external calls / queues / webhooks identified>

Is this correct? Anything missing or misidentified?

Incorporate corrections before proceeding. This gate prevents analysing a wrong model.

Step 3: Simulate stuck intermediate states

For each intermediate state of each entity in the flow, simulate a failure of progression and trace downstream impact:

Copy code
For entity E, transition Sn → Sn+1:
  1. Assume E is stuck in Sn (the transition never completes).
  2. What triggers Sn → Sn+1? (a consumer, an HTTP response, a scheduled job, a user action)
  3. If that trigger never fires or fails:
     - What downstream entities/flows are blocked or left inconsistent?
     - Is there money / data / a user commitment left in limbo?
  4. What, if anything, moves E forward or unwinds it?

Record one scenario per stuck state.

Step 4: Classify each scenario into a tier

Downgrade honestly: if the "retry" only works when the DB is hand-edited first, it's Tier 3, not Tier 2.

Step 5: Emit outputs

Write to docs/ops-risk/<flow-or-service>-<YYYY-MM-DD>.md.

Tier 2 → Operational runbook

Copy code
### Runbook: <scenario>
Symptom:        <how an operator recognises the stuck state>
Detection:      <query / metric / log to confirm>
Trigger:        <the exact API call / endpoint / Console action that unblocks it>
Steps:          1. ... 2. ... 3. ...
Verification:   <how to confirm the entity reached terminal state>
Blast radius:   <what else is affected while stuck>

Tier 3 → Gap spec

Copy code
### Gap: <scenario>
What's missing:     <the rescue mechanism that does not exist>
Impact if hit:      <blast radius, data/money at risk, frequency estimate>
Who can act today:  <e.g. only a DBA with prod write access>
What should exist:   <new API / endpoint / Console UI action / automated job>
Suggested owner:    <team / epic to carry it>

Summary table (top of file)

Copy code
| Flow | Entity | Stuck state | Tier | Artifact |
|------|--------|-------------|------|----------|

Step 6: Present to the developer

Summarize: mode used, flows reviewed, scenario count per tier, the count of Tier 3 gaps (the important number), and the path to the generated file. Offer to open issues for Tier 3 gaps if the developer wants a backlog.

Red Flags — STOP

• Classifying a scenario Tier 2 when the "trigger" only works after a manual DB edit → it is Tier 3.
• Leaving the repo in Mode A to audit a dependency's internals → out of scope; review only how THIS service reacts.
• Skipping the Step 2 confirmation dialogue → you may be analysing the wrong flow model.
• Trusting the scanner's resilience flags as ground truth → they are heuristic; a true is a hint and a false is a prompt to verify, never a final verdict.
• Starting the analysis in Mode A without the scan-integration-points.mjs JSON when Node.js is available → run the mechanical phase first, then reason over its output.
• Scanning only apps/<svc> in a hexagonal monorepo and trusting a 0 result → the boundaries live in imported pkg/* adapters; a subdir-only scan is a false 0. Run from the repo root (the scanner auto-expands) and never treat a warnings[] scope alert as "no boundaries exist".
• Reporting only Tier 3 counts without the concrete "what should exist" → a gap without a spec is not actionable.

Common Mistakes

• Analysing happy path only. The whole point is the stuck intermediate states, not the terminal success.
• Treating a DLQ as Tier 1 automatically. A DLQ with no replay path is really a Tier 2 (needs a re-drive) or Tier 3 (nothing drains it).
• Mode B scope creep. In plan-context mode, read the plan and epic artifacts — do not fall back into full-repo exploration.