External Tools

External Tools Skill

Purpose

This skill delegates implementation and review tasks to external AI CLI tools (OpenAI Codex CLI, Google Gemini CLI), enabling cost savings through cheaper models and cross-model adversarial review that eliminates single-model blind spots. Three core principles govern every interaction: one job per invocation (an external tool implements OR reviews, never self-validates), minimal permissions (sandboxed execution scoped to the task's working directory with only the tool's own API key), and the orchestrator verifies independently (external tools report facts only, the orchestrator judges pass/fail).

Prerequisites

Required Tools

Neither tool is strictly required. The skill adapts based on what is available (see Escalation Model below).

Configuration

Per-project config at .metaswarm/external-tools.yaml (optional -- if absent, external tools are not used):

yamlCopy code
adapters:
  codex:
    enabled: true
    model: "gpt-5.3-codex"
    timeout_seconds: 300
    sandbox: docker          # docker | platform | none
  gemini:
    enabled: true
    model: "pro"
    timeout_seconds: 300
    sandbox: docker

routing:
  default_implementer: "cheapest-available"
  escalation_order: ["codex", "gemini", "claude"]

budget:
  per_task_usd: 2.00        # circuit breaker per task
  per_session_usd: 20.00    # circuit breaker per session

Fallback Behavior

• Both tools available: Full cross-model delegation and review
• One tool available: Reduced chain with mutual review between the tool and Claude
• No tools available: Existing metaswarm behavior unchanged; skill is a no-op

Quick Reference

Health Check

bashCopy code
# Check if Codex is installed, authenticated, and reachable
adapters/codex.sh health

# Check Gemini
adapters/gemini.sh health

Returns JSON with status: "ready|degraded|unavailable".

Implement

bashCopy code
adapters/codex.sh implement \
  --worktree "/tmp/ext-codex-task-42" \
  --prompt-file "/private/tmp/xt-abc123/prompt.md" \
  --attempt 1 \
  --timeout 300

Review

bashCopy code
adapters/gemini.sh review \
  --worktree "/tmp/ext-codex-task-42" \
  --rubric-file "./rubrics/external-tool-review-rubric.md" \
  --spec-file "/private/tmp/xt-abc123/spec.md" \
  --timeout 300

All commands return a uniform JSON envelope with facts only (exit code, files changed, cost, duration). The orchestrator interprets results -- adapters never self-judge.

Workflow: Routing & Dispatch

Phase 0: Tool Discovery (Health Checks)

Before dispatching any task, the orchestrator runs health checks to determine which tools are available:

bashCopy code
# Run health checks for all configured adapters
for adapter in adapters/*.sh; do
  result=$("$adapter" health)
  # Parse status: ready | degraded | unavailable
done

Health is checked per task dispatch, not just at session start. Auth tokens can expire mid-session, network conditions change, and rate limits reset. A tool that was ready five minutes ago may now be unavailable.

Build the available tools list:

pythonCopy code
available_tools = [t for t in adapters if t.health() == "ready"]

Phase 1: IMPLEMENT (External Tool or Claude)

If an external tool is selected as implementer:

1. Create worktree -- git worktree add "$WORKTREE_PATH" -b "external/$TOOL/$TASK_ID" for isolation
2. Package context -- Gather relevant files, spec, acceptance criteria into a prompt file in a secure temp dir (mode 700, trap cleanup). Token-budget the context to fit the model's window.
3. Invoke -- safe_invoke() with timeout wrapper; capture JSON output
4. Scope check -- Verify all file changes are within the declared file scope; revert any out-of-scope changes
5. Capture output -- Facts only: exit code, files changed, diff stats, cost, duration

If Claude is selected (escalation or no external tools), use the existing Task() mechanism unchanged.

Phase 2: VALIDATE (Always Orchestrator -- Unchanged)

The orchestrator independently runs all quality gates:

• npm test / npx vitest run
• npx tsc --noEmit
• npx eslint <changed-files>
• Coverage enforcement via .coverage-thresholds.json
• File scope verification via git diff --name-only

This phase is identical whether the implementer was an external tool or Claude. Never trust the implementer's self-report.

Phase 3: ADVERSARIAL REVIEW (Cross-Model)

The key advantage of external tools: the writer is always reviewed by a different model.

Review is invoked via the adapter's review command. The orchestrator reads the reviewer's raw log and evaluates it independently -- the adapter never returns a pass/fail verdict.

Phase 4: COMMIT (Unchanged)

After passing adversarial review:

1. Merge the worktree branch into the working branch
2. Clean up the worktree (git worktree remove)
3. Log the session for self-reflection
4. Commit with DoD verification in the commit message

Escalation Model (Availability-Aware)

The orchestrator adapts its escalation chain based on which tools passed their health checks.

Scenario 1: Two External Tools Available (5 attempts max)

Copy code
Model A implements (attempt 1)
  Orchestrator validates + cross-model review (B + Claude)
  FAIL -> feedback to Model A

Model A implements (attempt 2, with review feedback)
  Orchestrator validates + cross-model review
  FAIL -> escalate to Model B

Model B implements (attempt 1, with Model A's branch as reference)
  Orchestrator validates + cross-model review (A + Claude)
  FAIL -> feedback to Model B

Model B implements (attempt 2, with review feedback)
  Orchestrator validates + cross-model review
  FAIL -> escalate to Claude

Claude implements (with both branches as reference)
  Orchestrator validates + cross-model review (A + B)
  FAIL -> alert user with all branches, findings, CI results

Worst case: A(2) + B(2) + Claude(1) = 5 attempts before user alert.

Scenario 2: One External Tool Available (3 attempts max)

Copy code
Model A implements (attempt 1)
  Orchestrator validates + review (Claude reviews)
  FAIL -> feedback to Model A

Model A implements (attempt 2, with review feedback)
  Orchestrator validates + review (Claude reviews)
  FAIL -> escalate to Claude

Claude implements (with Model A's branch as reference)
  Orchestrator validates + review (Model A reviews)
  FAIL -> alert user

Worst case: A(2) + Claude(1) = 3 attempts before user alert.

Scenario 3: No External Tools Available

Copy code
Existing metaswarm behavior unchanged.
Claude implements via Task() mechanism.
Standard adversarial review (fresh Task() instance).

No change to the current workflow. The skill is a no-op.

Routing Logic

The orchestrator selects the implementer for each task using cheapest-available selection:

pythonCopy code
available_tools = [t for t in adapters if t.health() == "ready"]

if len(available_tools) == 2:
    implementer = cheapest(available_tools)
    reviewers = [other_tool, claude]
    escalation = [implementer, other_tool, claude, user]
elif len(available_tools) == 1:
    implementer = available_tools[0]
    reviewers = [implementer, claude]  # mutual review
    escalation = [implementer, claude, user]
else:
    # pure metaswarm, no change
    implementer = claude
    reviewers = [claude_fresh_task]
    escalation = [claude, user]

Selection criteria for cheapest():

1. Pick the tool with lower estimated cost per invocation (based on model pricing and average token usage from session logs)
2. On tie, prefer the tool with higher historical success rate for this task type
3. If no historical data, prefer Gemini (free tier) over Codex

Prompt File Format

The prompt file is the key interface between the orchestrator and external tools. It must be self-contained -- the external tool has no access to BEADS, knowledge base, or conversation history.

markdownCopy code
# Task: [task title]

## Acceptance Criteria
- [ ] criterion 1
- [ ] criterion 2
- [ ] criterion 3

## Context
[relevant file contents, token-budgeted]
[prioritized: changed files > test files > imports > surrounding context]

## Coding Standards
[extracted from project's CLAUDE.md / coding standards guide]
[language-specific rules, naming conventions, patterns]

## Test Expectations
[what tests should pass after implementation]
[coverage requirements from .coverage-thresholds.json]
[test runner command: e.g., npx vitest run]

## Previous Attempt (if retry)
[review feedback from last attempt -- what to fix]
[DO NOT include full prior output, only the actionable feedback summary]

## Prior Model's Attempt (if escalation)
[diff summary from previous model's branch -- what was tried, why it failed]
[DO NOT include full branch, only the relevant diff summary]

Token budgeting rules (handled by package_context() in _common.sh):

• Estimate tokens per file using chars/4 heuristic
• Prioritize context: changed files > test files > imports > surrounding context
• Truncate to the model's context budget (configurable per adapter)
• On retry: include review feedback summary, not full prior output
• On escalation: include prior branch diff summary, not full branch

Error Handling

When an adapter returns exit_code != 0, the error_type field determines the orchestrator's response:

General rules:

• "Skip adapter" means move to the next tool in the escalation chain, not abort the task
• All errors are logged to ~/.claude/sessions/ for self-reflection
• Auth errors always escalate to the user -- the orchestrator cannot fix credentials
• Network and rate-limit errors get transient-failure treatment (backoff + retry)

Budget Enforcement

Two circuit breakers prevent runaway costs:

Per-Task Budget (per_task_usd)

Before each adapter invocation, the orchestrator checks cumulative cost for the current task:

Copy code
if task_cost_so_far + estimated_cost > per_task_usd:
    error_type = "cost_limit_exceeded"
    skip adapter, try next in escalation chain (or Claude)

Default: $2.00 per task.

Per-Session Budget (per_session_usd)

Before each adapter invocation, the orchestrator checks cumulative cost across all tasks in the session:

Copy code
if session_cost_so_far + estimated_cost > per_session_usd:
    error_type = "cost_limit_exceeded"
    alert user: "Session budget exhausted. External tools disabled for remaining tasks."
    fall back to Claude for all remaining work

Default: $20.00 per session.

Cost Tracking

Cost is extracted post-execution from the adapter's JSON output (cost.input_tokens, cost.output_tokens) and converted to USD using known model pricing. Cost estimation is approximate (~15-30% variance due to different tokenizers per model). There is no pre-execution cost prediction in v1 -- cost is measured after execution, not capped before.

Anti-Patterns

Logging

All adapter invocations produce structured session log entries in ~/.claude/sessions/:

Copy code
Session log entry:
  - timestamp, tool, command, model, attempt
  - prompt content hash (what we asked)
  - output JSON (facts: exit code, files changed, cost)
  - CI gate results (from orchestrator's Phase 2)
  - review output (from cross-model Phase 3)
  - outcome: success | retry | escalated | user_alert
  - cost (tokens used, estimated USD)

These logs feed the existing /self-reflect pipeline. Over time, the knowledge base accumulates routing intelligence:

• Patterns: "Codex excels at single-file TypeScript implementations"
• Gotchas: "Gemini tends to skip error handling in async functions"
• Routing decisions: "Route database migration tasks directly to Claude (external tools fail 80%)"

Future routing decisions improve automatically as the knowledge base grows from session log analysis.