Ops Desktop

OPS ► DESKTOP

Load ops-rules before acting. Public repo (no personal data). Outbound: one draft → one approval → one send. If AskUserQuestion / Workflow are missing, follow Rule 10 in ops-rules (Hermes: numbered options / two-turn Telegram card; delegate_task).

/ops:desktop drives a real GUI desktop (Linux noVNC pool, partial macOS / Windows support) through the desktop-act FastMCP server. Use it for tasks that browser-only automation can't reach: native dialogs, OS file pickers, multi-window flows, screen-recording verification.

The MCP tool schemas are deferred. Load them with ToolSearch on demand:

Copy code
ToolSearch select:mcp__desktop-act__acquire_desktop,mcp__desktop-act__screenshot,mcp__desktop-act__click,mcp__desktop-act__type_text,mcp__desktop-act__release_desktop

Runtime Context

Before any route runs, resolve:

1. Plugin root: ${CLAUDE_PLUGIN_ROOT} — used to locate the launcher.
2. MCP launcher: ${CLAUDE_PLUGIN_ROOT}/mcp-servers/desktop-act-launcher.py — auto-discovers an installed desktop-act marketplace or, if absent, clones from $DESKTOP_ACT_REPO (default https://github.com/Lifecycle-Innovations-Limited/desktop-act.git) and bootstraps a venv on first run.
3. noVNC default: http://<box-host>:6082 — primary desktop. acquire_desktop() provisions extra sessions on :6082+.
4. Mobile / SSH mode: if $SSH_CONNECTION, $SSH_CLIENT, or $SSH_TTY is set, emit compact text-only output per Rule 7 (no banners, no tables, plain lines). The noVNC URL is the one piece the user definitely needs — print it once on a copy-able line.

Routing table

Parse $ARGUMENTS and route. First token decides:

Anything else: treat the whole argument string as the goal.

Route — status (default)

1. Probe the MCP via mcp__desktop-act__status (auto-load schema first).
2. Print compact summary — plugin connected? venv ready? pool size? default URL?
3. If the MCP is not reachable, the launcher likely needs to bootstrap. Show:
   • one-line install command (/plugin marketplace add <repo> && /plugin install desktop-act)
   • or the env override (export DESKTOP_ACT_COMMAND=/path/to/run.sh)
   • or the bootstrap repo (export DESKTOP_ACT_REPO=https://...)
4. Mobile/SSH mode: drop banners, output 4–6 plain lines.

Route — list

Copy code
ToolSearch select:mcp__desktop-act__list_desktops

Call it. Format as session_id · display · vnc · last_used. Linux only renders full info; macOS/Windows fall back to a "limited platform" note.

Route — release <id> / release-all

Per Rule 5 (destructive actions): release_desktop tears down the X server. Ask before bulk release:

Copy code
AskUserQuestion: Release session abcd1234?
  [Release]  [Skip]

For release-all, batch confirm once.

Route — goal (default for free-form $ARGUMENTS)

The autonomous loop. Use this when the owner types /ops:desktop open the AWS console and screenshot the ECS cluster page.

Step 1 — Acquire

pythonCopy code
# Load schemas
ToolSearch select:mcp__desktop-act__acquire_desktop,mcp__desktop-act__screenshot,...

session = mcp__desktop-act__acquire_desktop()
session_id = session["session_id"]
display    = session["display"]      # e.g. ":51"
novnc_url  = session["novnc_url"]    # e.g. http://host:6082

Print the noVNC URL on a line by itself so the user can watch live. On mobile, the URL is the most useful thing — surface it FIRST.

Step 2 — Observe

pythonCopy code
shot = mcp__desktop-act__screenshot(session_id=session_id)
Read(file_path=shot["path"])   # let Claude see the screen

If you need window-level metadata before clicking blindly:

pythonCopy code
windows = mcp__desktop-act__list_windows(session_id=session_id)

Step 3 — Drive

Two modes:

A. Streaming primitives — full reasoning visibility, per-action transparency. Preferred for sensitive flows (anything touching credentials, billing pages, destructive UIs).

pythonCopy code
mcp__desktop-act__launch_app(session_id, app="firefox")
mcp__desktop-act__click(session_id, x=640, y=400)
mcp__desktop-act__type_text(session_id, text="ECS clusters")
mcp__desktop-act__keypress(session_id, key="Return")
mcp__desktop-act__scroll(session_id, direction="down", amount=3)
mcp__desktop-act__screenshot(session_id)   # verify

B. Autonomous loop — hands the goal to act() which runs claude-agent-sdk against the OAuth-bundled CLI (no API key needed; rides Claude Max). Faster but less observable.

pythonCopy code
result = mcp__desktop-act__act(
    session_id=session_id,
    goal="$ARGUMENTS",
    max_iterations=20,
)

For multi-step macros, prefer batch (single round-trip) or act_step (one micro-step at a time with explicit verification).

Step 4 — Verify

Always screenshot and Read the path once at the end before claiming the goal is done. If the verification screenshot doesn't show the expected state, iterate or escalate to the user.

Step 5 — Release

Always release in the same turn, even on failure:

pythonCopy code
mcp__desktop-act__release_desktop(session_id=session_id)

If the user wants to keep the session for follow-up actions, ask first via AskUserQuestion:

Copy code
Keep session abcd1234 alive for follow-up?
  [Keep]  [Release]

Cross-platform notes

On non-Linux platforms, prefer Kapture (mcp__kapture__*) or Playwright for browser-only tasks. The launcher prints a clear "limited platform" notice if it can't bring up a full session.

Safety

• Outbound comms (Rule 6): if a goal involves sending email/messages/forms, stage the draft and ask before pressing Send. Per-message approval, never batch.
• Destructive UI clicks: anything that says "Delete", "Terminate", "Force quit" routes through AskUserQuestion first per Rule 5.
• Credentials: never type passwords into a goal string — leave it for the owner to fill in the live noVNC viewer.
• Session isolation: each invocation of /ops:desktop gets its own session_id. Don't pass session IDs between concurrent agents or skills.

Examples

Copy code
/ops:desktop                                    # status
/ops:desktop list                               # pool state
/ops:desktop release abcd1234                   # free session
/ops:desktop open the AWS ECS console and screenshot the my-project-api cluster
/ops:desktop launch Firefox and navigate to https://example.com

$ARGUMENTS

$ARGUMENTS