Self Improving Agent

Self-Improving Agent

Turn evidence from completed work into a small, auditable behavior change. The default result is a candidate or no change—not an automatic rewrite of skills.

Use This Skill When

• A tool or workflow failed in a way that may recur.
• The user corrected an assumption, requirement, or operating rule.
• The same workaround appeared more than once.
• A focused test proved a better reusable method.
• The user asks to review or consolidate learning candidates.

Do not use it for routine session summaries, raw transcript storage, speculative ideas without evidence, or project facts that belong in project documentation.

Required Outcome

Every run ends in exactly one state:

1. candidate: reusable but not yet validated.
2. validated: representative evidence supports the lesson, but no owner change is claimed yet.
3. applied: the validated lesson was installed in one named durable owner with a change reference.
4. rejected: disproved, unsafe, too specific, or obsolete.
5. superseded or rolled_back: an applied/validated lesson was replaced or reverted.
6. no-delta: no reusable behavior change was found.
7. open-question: evidence is insufficient and the missing proof is named.

An artifact is not proof of improvement. An applied lesson must change future behavior and have a representative check that demonstrates the change.

Start Packet

Before editing durable guidance, state:

• Future behavior: what the agent should do differently next time.
• Representative task: one concrete scenario that should now succeed.
• Evidence: current source, failure output, user correction, or focused test.
• Owner: the one skill, instruction file, script, or runtime component that owns it.
• Write boundary: files allowed to change and information that must remain local.
• Proof: the command, eval, or review that confirms the new behavior.

If any item is unknown, capture a candidate and stop before validation or application.

Lifecycle

1. Capture the Signal

Prefer facts over interpretation. Record only the minimum reusable summary; do not copy transcripts, tool inputs, credentials, private paths, or customer data.

Claude Code failure hooks explicitly enabled with apb init --hooks can call:

bashCopy code
agent-playbook self-improve

Manual corrections or successes use an explicit summary and evidence label:

bashCopy code
apb self-improve capture \
  --kind correction \
  --summary "Verify the current source before relying on cached state" \
  --evidence "focused-test"

The CLI stores redacted events and deduplicated candidates under ~/.agent-playbook/self-improvement/. Override the root with AGENT_PLAYBOOK_DATA_DIR or --data-dir.

2. Assess Reusability

Keep a candidate only when all are true:

• It describes future behavior, not just what happened.
• It is useful beyond one private task or repository.
• It does not conflict with a current authoritative source.
• A narrow owner and a realistic validation path exist.

Use apb behavior inbox to inspect the prioritized queue. Repeated evidence increases occurrence count; it does not automatically increase truth. Use apb behavior owners <candidate-id> --repo . for local suggestions, but treat every result as a review candidate rather than an ownership decision.

3. Validate

Choose the smallest proof that can falsify the candidate, encode it as an executable artifact, and run it with apb self-improve eval. See references/eval-artifact.md for the schema and safety boundary.

Separate facts, hypotheses, and missing evidence. Structural validation alone does not prove that guidance is semantically current or executable by the host.

4. Validate, Apply, or Reject

Run the artifact first. A baseline scenario is recommended when the previous behavior can be reproduced safely; at least one candidate scenario is required:

bashCopy code
apb self-improve eval cand-123 --artifact behavior-eval.json

apb self-improve review cand-123 \
  --decision validate \
  --reason "baseline reproduced and candidate scenarios passed" \
  --eval-result /path/printed/by/the/eval/command.json

Validation accepts only a passing CLI-generated eval result for the same candidate. It does not claim runtime behavior changed.

Generate a local Behavior Change Proposal before editing the owner:

bashCopy code
apb behavior proposal cand-123 \
  --owner "skill:self-improving-agent" \
  --output behavior-proposal.md

The proposal contains the behavior diff intent, eval proof, acceptance criteria, privacy boundary, and rollback plan. It does not edit the owner or create a remote pull request.

After changing exactly one durable owner, record the application separately:

bashCopy code
apb self-improve review cand-123 \
  --decision apply \
  --reason "installed after the focused test passed" \
  --owner "skill:self-improving-agent" \
  --change-ref "commit:abc123"

Other decisions:

bashCopy code
apb self-improve review cand-123 --decision observe --reason "needs a second episode"
apb self-improve review cand-123 --decision reject --reason "project-specific exception"

Apply into the narrowest owner:

1. Executable test, script, or validator when behavior can be enforced.
2. The owning skill or its reference when agent judgment is required.
3. Project instructions only for project-wide constraints.
4. A knowledge notebook for durable facts that should be retrieved, not always loaded.

Never silently modify repository rules, publish packages, or trigger external actions as a side effect of capture.

5. Prove the Loop

Run the representative task after application. Report:

• candidate id and final state;
• evidence used and what remains uncertain;
• durable owner changed;
• executable eval result and artifact hash;
• rollback path.

If the new rule does not change the representative behavior, revert or reject it.

Knowledge Export

Export applied rules and open candidates as Markdown for Obsidian or another local knowledge system:

bashCopy code
apb self-improve export --output /path/to/vault/Agent/Learning.md

The export is a sink, not the source of truth. Candidate and active-rule state remain structured and auditable in the CLI data directory.

Host Boundary

Skills describe judgment; host adapters provide events and actions. Check the current host before claiming support:

• Claude Code: deterministic failure hook installed only by explicit apb init --hooks.
• Codex, Gemini, DeepSeek Harness: skill distribution is supported; learning event wiring depends on each host's current extension API.
• Unsupported hooks must remain manual or adapter-specific, never simulated by undocumented behavior.

Use apb conformance to inspect local-static contracts. A proven distribution or hook configuration does not prove host discovery or runtime invocation; those remain unverified until an observed host run supplies bounded evidence.

See references/learning-lifecycle.md for schemas and adapter contracts. Use evals/cases.json with evals/rubric.md when changing this skill.

Done Checklist

• Candidate/no-delta decision is explicit.
• Stored text is minimal, redacted, and portable.
• Current authoritative sources were checked when relevant.
• Validation uses a passing executable eval result for the same candidate.
• Application names one durable owner and a concrete change reference.
• Representative behavior was tested after application.
• No private project detail entered public skill assets.