Evolving Loop

Self-Evolving Development Loop

Execute an autonomous development cycle that dynamically generates, validates, and evolves its own execution strategy. Integrates with Meta-Engineering memory system for pattern learning and tool evolution.

Architecture Details: See docs/EVOLVING-LOOP-ARCHITECTURE.md

Usage

bashCopy code
# Start new task
/evolving-loop "Your task description

Acceptance Criteria:
- [ ] Criterion 1
- [ ] Criterion 2
"

# Flags
/evolving-loop --resume    # Resume interrupted session
/evolving-loop --status    # Check status
/evolving-loop --force     # Clear and restart
/evolving-loop --evolve    # Trigger manual evolution
/evolving-loop --memory    # Show memory system status

How It Works

Copy code
┌──────────────────────────────────────────────────────┐
│  8-Phase Self-Evolving Loop                          │
├──────────────────────────────────────────────────────┤
│                                                      │
│  Phase -2: CONTEXT_CHECK  → Check token pressure     │
│  Phase -1A: PATTERN_LOOKUP → Match task patterns     │
│                                                      │
│  ┌─────────────── Main Loop ───────────────┐        │
│  │ Phase 1: ANALYZE   → Extract AC         │        │
│  │ Phase 2: GENERATE  → Create skills      │        │
│  │ Phase 3: EXECUTE   → TDD implementation │        │
│  │ Phase 4: VALIDATE  → Score 0-100        │        │
│  │ Phase 5: DECIDE    → SHIP/FIX/EVOLVE    │        │
│  │ Phase 6: LEARN     → Extract patterns   │        │
│  │ Phase 7: EVOLVE    → Improve skills     │        │
│  │ Phase 8: SHIP      → Deliver result     │        │
│  └──────────────────────────────────────────┘        │
│                                                      │
│  Phase -1C: EVOLUTION → Update memory (on SHIP)     │
│                                                      │
└──────────────────────────────────────────────────────┘

Execution

When user runs /evolving-loop "$ARGUMENTS":

1. Handle Flags

bashCopy code
STATE_DIR=".self-evolving-loop"
MEMORY_DIR=".claude/memory/meta-engineering"
CHECKPOINT="$STATE_DIR/state/checkpoint.json"

# --status: Show current state
if [[ "$ARGUMENTS" == *"--status"* ]]; then
    /evolving-status
    exit 0
fi

# --memory: Show memory system status
if [[ "$ARGUMENTS" == *"--memory"* ]]; then
    echo "Memory System Status:"
    if [ -d "$MEMORY_DIR" ]; then
        echo "Tool Usage: $(jq '.tools | length' "$MEMORY_DIR/tool-usage.json" 2>/dev/null || echo "0") tools"
        echo "Patterns: $(jq '.task_patterns | keys | length' "$MEMORY_DIR/patterns.json" 2>/dev/null || echo "0") patterns"
        echo "Evolution: v$(jq -r '.version' "$MEMORY_DIR/evolution.json" 2>/dev/null || echo "0")"
    else
        echo "(Not initialized - will create on first run)"
    fi
    exit 0
fi

# --resume: Continue from checkpoint
if [[ "$ARGUMENTS" == *"--resume"* ]]; then
    if [ ! -f "$CHECKPOINT" ] || [ "$(jq -r '.status' "$CHECKPOINT")" == "idle" ]; then
        echo "No active session to resume."
        exit 1
    fi
fi

# --force: Clear old state
if [[ "$ARGUMENTS" == *"--force"* ]]; then
    rm -rf "$STATE_DIR/state/*" "$STATE_DIR/reports/*" "$STATE_DIR/generated-skills/*"
fi

2. Initialize (First-Run Safe)

bashCopy code
# Create directories (first-run safe)
mkdir -p "$MEMORY_DIR"
mkdir -p "$STATE_DIR"/{state,reports,generated-skills,history,backups}

# Helper: Read JSON with fallback
read_json_safe() {
    local file="$1"
    local default="$2"
    if [ -f "$file" ]; then
        cat "$file" 2>/dev/null || echo "$default"
    else
        echo "$default"
    fi
}

# Detect first run
IS_FIRST_RUN=false
if [ ! -f "$MEMORY_DIR/patterns.json" ]; then
    IS_FIRST_RUN=true
    echo "📝 First run detected - initializing memory system..."
fi

# Initialize memory files if missing (see docs for full schema)

3. Delegate to Orchestrator

CRITICAL: Use context isolation - orchestrator runs in fork context.

Before dispatch, preserve the request and checkpoint preimage. If Claude's Agent tool is unavailable or withheld, or Codex's spawn_agent interface is unavailable or withheld, the maximum depth is reached, or a concurrency limit rejects the call, do not retry or execute the loop inline. Return an incomplete dispatch_request and do not advance or mutate loop state.

Choose exactly one provider-native form:

• Claude Code: Agent(subagent_type="evolving-orchestrator", prompt="<bounded prompt>")
• Codex CLI: spawn_agent(agent_type="evolving-orchestrator", task_name="evolving_loop", message="<bounded prompt>")

markdownCopy code
Agent(subagent_type="evolving-orchestrator", prompt="""
Request: $ARGUMENTS
Task Type: $TASK_TYPE (from pattern matching)

Execute phases in sequence, each in fork context.
Return only brief status updates (1 line per phase).
Store ALL detailed output in files.

Return format:
📊 CONTEXT: [OK/Warning] - [N]% usage
🔍 PATTERNS: Matched [type], [N] recommendations
✅ ANALYZE: [N] AC identified
✅ GENERATE: Created v[N] skills
🔄 EXECUTE: Iter [N] - [status]
✅ VALIDATE: Score [N]/100
➡️ DECIDE: [SHIP/FIX/EVOLVE]
""")

In Codex, pass the same bounded request and return contract through spawn_agent(...); do not invoke the Claude example as a second dispatch.

Output Example

Copy code
🚀 Starting Self-Evolving Loop (Meta-Engineering v2.0)...

📊 CONTEXT: OK - 15% usage
🔍 PATTERNS: Matched 'auth', 3 recommendations
✅ ANALYZE: 5 acceptance criteria identified
✅ GENERATE: Created executor-v1, validator-v1, fixer-v1
🔄 EXECUTE: Iteration 1 - 4 files modified, 3/5 tests passing
✅ VALIDATE: Score 72/100
➡️ DECIDE: FIX (minor test failures)
🔄 EXECUTE: Iteration 2 - 2 files modified, 5/5 tests passing
✅ VALIDATE: Score 94/100
➡️ DECIDE: SHIP
📚 LEARN: 2 patterns identified
🧬 EVOLUTION: Updated memory
✅ SHIP: All criteria met!

📊 Summary: 2 iterations, 6 files changed, 5/5 AC complete

Phase Agents

State Files

Copy code
.self-evolving-loop/          ← Session state (temporary)
├── state/checkpoint.json     ← Current state
├── reports/*.json            ← Phase outputs
├── generated-skills/*.md     ← Dynamic skills
└── history/*.jsonl           ← Event logs

.claude/memory/meta-engineering/  ← Persistent memory
├── tool-usage.json           ← Usage statistics
├── patterns.json             ← Learned patterns
└── evolution.json            ← Evolution history

Stop / Resume

bashCopy code
# Stop after current phase
touch .self-evolving-loop/state/stop

# Resume later
/evolving-loop --resume

Hook-Driven Continuation (optional)

By default the loop runs through the evolving-orchestrator agent in fork context (see Delegate to Orchestrator above) — each phase stays in an isolated context.

If you instead want the loop driven by a Stop hook — so it resumes phase-by-phase in your main context and survives context exhaustion — merge the shipped hook config into .claude/settings.local.json:

bashCopy code
# Activate: append the loop's Stop + PostToolUse hooks
python3 - <<'PY'
import json
s   = json.load(open('.claude/settings.local.json'))
add = json.load(open('.self-evolving-loop/hooks/settings-hooks.json'))
h   = s.setdefault('hooks', {})
for event, entries in add['hooks'].items():
    h.setdefault(event, []).extend(entries)
json.dump(s, open('.claude/settings.local.json', 'w'), indent=2)
PY

To deactivate, remove those Stop/PostToolUse entries from .claude/settings.local.json again (or restore a pre-merge backup).

Tradeoff: hook mode runs each phase in your main context (visible, but consumes context as the task grows); agent mode (default) keeps each phase in an isolated fork context.

Related

• /evolving-status - View status and memory
• evolving-orchestrator - Phase coordinator
• Architecture Details - Full technical docs