Autonomous Agent Harness

Autonomous Agent Harness

Turn Claude Code into a persistent, self-directing agent system using only native features and MCP servers.

Consent and Safety Boundaries

Autonomous operation must be explicitly requested and scoped by the user. Do not create schedules, dispatch remote agents, write persistent memory, use computer control, post externally, modify third-party resources, or act on private communications unless the user has approved that capability and the target workspace for the current setup.

Prefer dry-run plans and local queue files before enabling recurring or event-driven actions. Keep credentials, private workspace exports, personal datasets, and account-specific automations out of reusable EGC artifacts.

When to Activate

• User wants an agent that runs continuously or on a schedule
• Setting up automated workflows that trigger periodically
• Building a personal AI assistant that remembers context across sessions
• User says "run this every day", "check on this regularly", "keep monitoring"
• Wants to replicate functionality from Hermes, AutoGPT, or similar autonomous agent frameworks
• Needs computer use combined with scheduled execution

Architecture

Copy code
┌──────────────────────────────────────────────────────────────┐
│                    AI coding tool runtime                     │
│                                                              │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌─────────────┐ │
│  │  Crons   │  │ Dispatch │  │ Memory   │  │ Computer    │ │
│  │ Schedule │  │ Remote   │  │ Store    │  │ Use         │ │
│  │ Tasks    │  │ Agents   │  │          │  │             │ │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └──────┬──────┘ │
│       │              │             │                │        │
│       ▼              ▼             ▼                ▼        │
│  ┌──────────────────────────────────────────────────────┐    │
│  │              EGC Skill + Agent Layer                  │    │
│  │                                                      │    │
│  │  skills/     agents/     commands/     hooks/        │    │
│  └──────────────────────────────────────────────────────┘    │
│       │              │             │                │        │
│       ▼              ▼             ▼                ▼        │
│  ┌──────────────────────────────────────────────────────┐    │
│  │              MCP Server Layer                        │    │
│  │                                                      │    │
│  │  memory    github    exa    supabase    browser-use  │    │
│  └──────────────────────────────────────────────────────┘    │
└──────────────────────────────────────────────────────────────┘

Core Components

1. Persistent Memory

Use Claude Code's built-in memory system enhanced with MCP memory server for structured data.

Built-in memory (~/.gemini/projects/*/memory/):

• User preferences, feedback, project context
• Stored as markdown files with frontmatter
• Automatically loaded at session start

MCP memory server (structured knowledge graph):

• Entities, relations, observations
• Queryable graph structure
• Cross-session persistence

Memory patterns:

Copy code
# Short-term: current session context
Use TodoWrite for in-session task tracking

# Medium-term: project memory files
Write to ~/.gemini/projects/*/memory/ for cross-session recall

# Long-term: MCP knowledge graph
Use mcp__memory__create_entities for permanent structured data
Use mcp__memory__create_relations for relationship mapping
Use mcp__memory__add_observations for new facts about known entities

2. Scheduled Operations (Crons)

Use Claude Code's scheduled tasks to create recurring agent operations.

Setting up a cron:

Copy code
# Via MCP tool
mcp__scheduled-tasks__create_scheduled_task({
  name: "daily-pr-review",
  schedule: "0 9 * * 1-5",  # 9 AM weekdays
  prompt: "Review all open PRs in Fmarzochi/EGC. For each: check CI status, review changes, flag issues. Post summary to memory.",
  project_dir: "/path/to/repo"
})

# Via egc -p (programmatic mode)
echo "Review open PRs and summarize" | egc -p --project /path/to/repo

Useful cron patterns:

3. Dispatch / Remote Agents

Trigger Claude Code agents remotely for event-driven workflows.

Dispatch patterns:

bashCopy code
# Trigger from CI/CD
curl -X POST "https://api.anthropic.com/dispatch" \
  -H "Authorization: Bearer $ANTHROPIC_API_KEY" \
  -d '{"prompt": "Build failed on main. Diagnose and fix.", "project": "/repo"}'

# Trigger from webhook
# GitHub webhook → dispatch → Gemini agent → fix → PR

# Trigger from another agent
egc -p "Analyze the output of the security scan and create issues for findings"

4. Computer Use

Leverage Gemini's computer-use MCP for physical world interaction.

Capabilities:

• Browser automation (navigate, click, fill forms, screenshot)
• Desktop control (open apps, type, mouse control)
• File system operations beyond CLI

Use cases within the harness:

• Automated testing of web UIs
• Form filling and data entry
• Screenshot-based monitoring
• Multi-app workflows

5. Task Queue

Manage a persistent queue of tasks that survive session boundaries.

Implementation:

Copy code
# Task persistence via memory
Write task queue to ~/.gemini/projects/*/memory/task-queue.md

# Task format
---
name: task-queue
type: project
description: Persistent task queue for autonomous operation
---

## Active Tasks
- [ ] PR #123: Review and approve if CI green
- [ ] Monitor deploy: check /health every 30 min for 2 hours
- [ ] Research: Find 5 leads in AI tooling space

## Completed
- [x] Daily standup: reviewed 3 PRs, 2 issues

Replacing Hermes

Setup Guide

Step 1: Configure MCP Servers

Ensure these are in ~/.gemini.json:

jsonCopy code
{
  "mcpServers": {
    "memory": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-memory"]
    },
    "scheduled-tasks": {
      "command": "<the scheduled-tasks server you verified and installed>"
    },
    "computer-use": {
      "command": "<the computer-use server you verified and installed>"
    }
  }
}

The memory entry names a published package. The scheduled-tasks and computer-use entries are placeholders: no first-party package exists for them at the time of writing, so point each at a server you have audited and installed yourself (the tool names used below, mcp__scheduled-tasks__* and the computer-use tools, are whatever that server exposes). Only register servers you have verified on the registry you install from (npm view <name> shows the publisher and the versions): a name that looks plausible but is not published is the shape of a slopsquatting attack, and the day someone registers it, npx -y runs their code.

Step 2: Create Base Crons

bashCopy code
# Daily morning briefing
egc -p "Create a scheduled task: every weekday at 9am, review my GitHub notifications, open PRs, and calendar. Write a morning briefing to memory."

# Continuous learning
egc -p "Create a scheduled task: every Sunday at 8pm, extract patterns from this week's sessions and update the learned skills."

Step 3: Initialize Memory Graph

bashCopy code
# Bootstrap your identity and context
egc -p "Create memory entities for: me (user profile), my projects, my key contacts. Add observations about current priorities."

Step 4: Enable Computer Use (Optional)

Grant computer-use MCP the necessary permissions for browser and desktop control.

Example Workflows

Autonomous PR Reviewer

Copy code
Cron: every 30 min during work hours
1. Check for new PRs on watched repos
2. For each new PR:
   - Pull branch locally
   - Run tests
   - Review changes with code-reviewer agent
   - Post review comments via GitHub MCP
3. Update memory with review status

Personal Research Agent

Copy code
Cron: daily at 6 AM
1. Check saved search queries in memory
2. Run Exa searches for each query
3. Summarize new findings
4. Compare against yesterday's results
5. Write digest to memory
6. Flag high-priority items for morning review

Meeting Prep Agent

Copy code
Trigger: 30 min before each calendar event
1. Read calendar event details
2. Search memory for context on attendees
3. Pull recent email/Slack threads with attendees
4. Prepare talking points and agenda suggestions
5. Write prep doc to memory

Constraints

• Cron tasks run in isolated sessions: they don't share context with interactive sessions unless through memory.
• Computer use requires explicit permission grants. Don't assume access.
• Remote dispatch may have rate limits. Design crons with appropriate intervals.
• Memory files should be kept concise. Archive old data rather than letting files grow unbounded.
• Always verify that scheduled tasks completed successfully. Add error handling to cron prompts.