Cli Skill Design

Designing a CLI skill

A CLI skill is two files that have to agree: the command surface (co <thing> ...) and the SKILL.md that tells an agent how to drive it. Design them together — the agent's whole world is what it types and what comes back.

co-browser is the worked example. Read it before you write anything: its exit-code table, its "read the output, not just the exit code" rule, and its Done checklist are what this methodology generalizes. (Building a skill that drives a website through co browser? That is the sibling skill browser-workflow-skill-builder — DOM, selectors, verification scripts. This one is about the command surface itself.)

Two properties. Each has a test you run and paste the result of — not a principle you assert in the PR.

(a) Tip-tested discoverability

Rule: every command execution — success and failure — ends by naming the next command, spelled out, with the argument shape filled in.

Copy code
Read one with: co gmail read <#>        ✅ names the command
See the docs for more options            ❌ names nothing

The tip test

A tip is good if an agent that has only that tip makes the right next call. That is testable, so test it:

pythonCopy code
from connectonion import llm_do

llm_do(
    f"You just ran a shell command. Its full output was:\n\n{out}\n\n"
    "Your goal: read the newest email. Reply with ONE shell command and nothing else.",
    model="co/gemini-3.8-flash",
)

Use a text-only call (llm_do), not co ai. An agent with a shell will run the command it picks — measured: the first attempt at this test executed co gmail read 1 and then co auth google against a real account. You are grading the reply, not the mailbox.

• Pass — the reply is a command that exists and advances the goal (co gmail read 1).
• Fail — it invents a name (co gmail open 1), asks for help, or replies with prose.

Rules for the harness, or the result means nothing:

• Give it the output only. No SKILL.md, no --help, no conversation history — those are exactly the crutches the tip exists to replace.

• Pin the model so a rerun compares like with like.

• Run it per command, not once. Score the whole surface in a table and paste it into the PR:

  command	tip printed	goal given to the fresh agent	it replied	pass

• Anything that fails: fix the tip, not the test.

What makes a tip pass

• It contains the literal command name, not a description of it.

• Placeholders say where the value comes from: <#> from this listing, not <id>.

• The tip survives piping. Agents always pipe. A tip inside if console.is_terminal: is invisible to every caller that needs it, and manual testing never catches it because a human runs in a terminal. Check every one:

  bashCopy code
  co <thing> <cmd> | cat        # the tip must still be there
  

• One next step. Two tips is a fork, and the agent resolves a fork by guessing.

• Failures get tips too, and the tip is the fix (see (b)).

Measured on the mail surface (8 tips, co/gemini-2.5-flash, 2026-08): 5 passed. The three failures are the three rules above, each in its pure form —

• a piped listing prints no tip, and the model invented readmail 18f2a;
• Retry the same command with --idempotency-key <key> never names the command, and the model replied !! --idempotency-key k-123;
• run co gmail to refresh stops one step short of the goal, and the model replied co gmail && co gmail 3 — a command that does not exist.

A tip that reads fine to a human fails this test. That is the point of running it.

(b) Self-diagnosing, self-correcting execution

Rule 1 — --help enumerates every capability

An agent that cannot find a command from --help will invent one, and an invented command name costs a round trip every time. So: no hidden commands, no capability that only SKILL.md knows about.

Check it, both directions:

bashCopy code
g=<thing>
# every subcommand the CLI has
co $g --help | sed -n '/─ Commands/,$p' | grep -oE '^│ [a-z-]+' | awk '{print $2}' | sort -u
# every command the skill mentions
grep -oE "co $g [a-z-]+" SKILL.md | awk '{print $3}' | sort -u

Diff the two lists. Every CLI command must be either documented or deliberately skipped (say which, and why, in the PR). Every command the skill mentions must exist — a skill naming a command that --help does not list is a documentation bug, and it is the failure mode this check exists to catch.

Repeat one level down for command groups (co outlook contact --help).

Rule 2 — every error path is a fix-it guide

The exit code says what kind of problem; the text says what to run. Both, every time. Follow co-browser's contract: a small, stable set of codes, and a table in SKILL.md whose right-hand column is a command, not an adjective.

Check it by producing each row. For every exit code your surface can return, write down the command that provokes it and run it:

bashCopy code
co <thing> <cmd-that-fails>; echo "exit=$?"

Then assert two things about the output: it names the cause, and it names a command to run next. Paste the reproduction table into the PR:

If a row cannot be provoked, you do not know that it behaves as documented — say so rather than documenting it.

Rule 3 — say so when failure exits 0

Some commands print ❌ Failed and still exit 0. That is fine as long as the skill says it loudly, because an agent that chains cmd && next on such a surface walks straight past the failure. Where any failure exits 0, SKILL.md opens with co-browser's rule:

Always read the output, not just the exit code.

and the exit-code table has a row for "exit 0, error text on stdout".

(c) A flag you repeat is a setting you never configured

Rule: anything a caller passes on every invocation belongs in configuration, and the flag is the override — not the only way in.

--engine onion was required on every paid browser call. A person testing the paid engine typed it a hundred times a day; an agent had to carry it through every call site, which is where it gets dropped, and the failure is silent — the free engine runs and reports success, so the thing under test was never tested. The flag was not the feature. The missing default was the defect.

Ask it of every flag you add: would someone pass this every time? If yes, it needs a place to live, and the flag becomes the way to differ from it once. Both directions have to work — --engine system must beat a configured paid default, so there is always a way to not spend money that needs no file edited.

Where a setting lives

A single name is not structure. Do not invent a config file for one string: two places to look is how a value gets set in one and read from the other.

Name the setting after the command that reads it (CO_BROWSER_ENGINE), and give the group a config verb that shows the current value and where it came from — then sets it. Showing the source is the whole point: "wtf, set in ~/.co/keys.env" and "wtf, set in your shell, which wins over the file" send a reader to different places.

When the setting costs money or does anything irreversible, the config verb says so before it writes, and the audit record distinguishes a run that a standing setting chose from one a flag asked for. Not as a brake — as legibility, because the first question about an unexpected charge is which invocations were a standing choice.

Progressive disclosure

The skill is read top to bottom by an agent that wants to act now.

1. Routing first — if several commands could serve the request, the first section is the table that picks one. Wrong-command errors are the expensive kind.
2. The 80% commands next, as copy-pasteable lines.
3. The gotchas that change a result — the ones that make an agent report something false if it doesn't know them (stale numbering, prefix-only search, silent export-on-download). Not trivia.
4. Errors and recovery last. By then the agent is only here because something broke.

Everything else belongs in --help. If the skill is restating --help, delete it from the skill: two copies drift, and the copy in the skill is the one that goes stale.

Honesty rule

Document only what you have run. Every command, flag, and exit code in a SKILL.md must have been verified against the code or --help on the branch you are writing against — not remembered, and not planned. Behavior that is designed but unshipped gets a dated "not yet — today it works like this" note, never a present-tense sentence. An agent cannot tell aspiration from fact, and it pays for the difference with a failed run.

Done checklist

• Routing table first, if more than one command could serve the request
• Every command in the skill exists in --help (diffed, both directions)
• Every command prints one next-step tip, and the tip survives | cat
• Tip test run per command, results table in the PR
• Exit-code table present, right column is a command
• Every exit code provoked at least once, reproduction table in the PR
• "Read the output, not just the exit code" stated if any failure exits 0
• Gotchas that change a reported result are written down
• Nothing documented that was not run
• No flag that a caller would pass every time (if there is one, it has a config verb and a home)

Shared env contract (1.8.4 implementation)

New command groups must use the shared selected env, never discover cwd .env. co --env-file PATH <group> ... is the explicit project selector; default reads and writes use global keys.env. Keep process credentials separate from loaded file values and resolve provider fields as whole account records. Test fresh processes from root, nested and unrelated directories, including a project-only non-OAuth variable that must stay absent by default.

A configuration failure names its source and co env, the command that shows the selected file and runs even when that file is broken: "not connected in ~/.co/keys.env … Run co env …" then Next: co auth google. A tip that says "set X in keys.env" names no command; write co env set X <value>.