Integrate Agent Email

Integrating @amd-gaia/agent-email

@amd-gaia/agent-email embeds the GAIA email agent in a JS/TS app. It triages, drafts, and sends email locally on AMD Ryzen AI — no cloud LLM. This package is the client: it downloads a frozen native sidecar binary, spawns it, and talks to it over local HTTP. There is no Python and no separate GAIA install.

Follow these steps to wire it into an app.

This file is NOT one of the agent's own skills. It is the integration playbook — how you wire this npm package into an app. The sidecar separately bundles six Agent Skills at gaia_agent_email/skills/<name>/SKILL.md, which are instructions the email agent itself would load into its own prompt at runtime — currently disabled, so none of them loads. Same filename, different artifact: don't load those into your assistant, and don't ship this one as an agent skill. See Skill sets below.

1. Install

bashCopy code
npm install @amd-gaia/agent-email

The package is ESM-only ("type": "module"). Use import, not require. From a CommonJS file, use await import("@amd-gaia/agent-email").

2. Pick the right entry point

• Node / main process → the default entry @amd-gaia/agent-email. It can fetch the binary and spawn/own the sidecar (uses node:fs, node:child_process).
• Browser / Electron renderer → the @amd-gaia/agent-email/client subpath. It has zero Node built-ins and only talks to an already-running sidecar over HTTP.

The desktop pattern: spawn the sidecar once from the Node/main process, then drive it from the renderer via ./client.

3. Fetch the binary and start the sidecar (Node)

tsCopy code
import { fetchBinary, startSidecar, shutdown } from "@amd-gaia/agent-email";

// Build time (or first run): download + SHA-256-verify the platform binary.
const { binaryPath } = await fetchBinary({ outDir: "resources" });

// Runtime: spawn -> wait for /health -> version-check, in one call.
const sidecar = await startSidecar({ binaryPath, port: 8131 });

// ... use sidecar.client ...

await shutdown(sidecar); // graceful stop — auto-cleanup also reaps on exit

• fetchBinary writes a verified binary into outDir. SHA-256 is mandatory; a bad download is rejected and not left on disk. Run it at build time or guard it to run once.
• startSidecar throws if the binary can't start, never becomes healthy, or the contract MAJOR version mismatches — and cleans up so a failed start leaks nothing. If the port is already taken (e.g. a playground still running) it throws PortInUseError without spawning; to reuse a running server, use connectSidecar({ baseUrl }) instead.
• The sidecar is auto-reaped when your process exits, crashes, or is signalled (default autoCleanup), so a missed shutdown won't orphan the frozen binary's child. shutdown(sidecar) is the graceful, awaited stop; autoCleanup: false opts out. If the sidecar survives the forced kill, shutdown rejects with an error naming the pid and the command to kill it — catch it and surface it rather than ignoring it.

4. Call the typed client

tsCopy code
const res = await sidecar.client.triage({
  payload: {
    kind: "single",
    principal: { email: "me@example.com" },
    message: {
      message_id: "m1",
      from: { name: "Sarah Chen", email: "sarah@example.com" },
      subject: "Prod incident follow-up",
      body: "Please review the report and reply by Friday.",
    },
  },
});
console.log(res.result.category, res.result.summary);

To classify many messages at once, use triageBatch — an items array (1–100) in, a parallel results array out, order-preserved. It's additive (the single triage above is unchanged). Per-item failures isolate, so an HTTP 200 can still carry errored items — inspect each results[].error, never just the status:

tsCopy code
const batch = await sidecar.client.triageBatch({
  items: [
    { kind: "single", principal: { email: "me@example.com" },
      message: { message_id: "m1", from: { email: "sarah@example.com" },
        subject: "Prod incident", body: "Reply by Friday." } },
  ],
});
for (const r of batch.results) {
  if (r.error) console.warn(`item ${r.index} failed: ${r.error.message}`);
  else console.log(`item ${r.index}:`, r.result!.category);
}

The interface:

Build the standalone surface (triage, draft, confirmAction, previewCalendarEvent) with zero connector setup. The read-only search and prescan read the live inbox (a connected mailbox, no token); send, the mailbox actions (archive / quarantine), and the calendar actions (view / create / respond) need a connected mailbox whose relevant scope was granted. Mint the gate token with draft (for send), confirmAction (for archive / quarantine), or previewCalendarEvent (for createCalendarEvent); archive and quarantine are reversible inside a 30s window via the ungated unarchive / unquarantine. Every non-2xx response throws HttpError (status, url, bodyText) — handle it; there is no silent null.

Scheduled daily briefing (#1608, REST-only): the sidecar can run prescan on a daily timer with no prompt. Off by default — launch with startSidecar({ env: { GAIA_EMAIL_BRIEFING_ENABLED: "true" } }) (fire time GAIA_EMAIL_BRIEFING_TIME, 24h local HH:MM, default 08:00), then pull the latest run from GET /v1/email/briefing with plain fetch (no client wrapper yet). 404 until the first scheduled run; an invalid env value fails sidecar startup loudly. The response is a cached run, so it also carries cache_age_seconds and stale (#2759) — stale is true once the briefing is at least 24 hours old. Read them and say the age before you show the contents; the endpoint labels an old briefing rather than refusing it or regenerating it, so a host that ignores those two fields presents last week's inbox as this morning's.

5. From a renderer (Electron / browser)

The sidecar serves same-origin only — no CORS. A renderer on a different origin cannot fetch http://127.0.0.1:8131 directly; the browser blocks it. So:

• Recommended: spawn the sidecar in the Electron main process (step 3) and expose triage/draft to the renderer over your own IPC. Don't call the sidecar from the renderer directly.
• The ./client entry (zero Node built-ins) is only usable from a same-origin or proxied page:

tsCopy code
import { EmailClient } from "@amd-gaia/agent-email/client";
// Pass the sidecar's session token (from sidecar.authToken in the main process,
// forwarded over IPC) — without it every /v1/email/* call is 401.
const client = new EmailClient({ baseUrl: "http://127.0.0.1:8131", authToken });

Canonical agent-loop query (POST /v1/email/query, schema 2.6)

The v2 keystone (#2016): NL request in, the agent reasons and chains its tools, the canonical Server-Sent Event types out — status / token / tool_call / tool_result / needs_confirmation / needs_input / final / error, terminated by exactly one final or error. This is the one loop every v2 front-door relays to. The host mints run_id and pushes the transcript slice in context, so the sidecar stays stateless. The typed client wraps it (#2097): query() returns an async iterator of typed QueryEvents; cancelQuery(runId) stops the run between steps:

tsCopy code
const runId = crypto.randomUUID(); // host-minted; also the cancel handle
for await (const ev of sidecar.client.query({
  query: "Triage my inbox",
  run_id: runId,
  context: [], // pushed transcript slice; [] for a fresh conversation
})) {
  switch (ev.type) {
    case "status":       console.log(ev.message); break;
    case "token":        process.stdout.write(ev.delta); break;
    case "tool_call":    console.log(`→ ${ev.tool}`, ev.args); break;
    case "tool_result":  console.log(`← ${ev.tool}`, ev.data); break;
    case "needs_confirmation": break; // run then ends with a final refusal (D1)
    case "needs_input":               // PAUSED — answer, then keep iterating
      await sidecar.client.respondToQuery(runId, ev.request_id, await askUser(ev));
      break;
    case "final":        console.log(ev.answer); break;   // terminal
    case "error":        console.error(ev.detail); break; // terminal, verbatim
    default:             console.warn("unsupported event", ev); // future additive type
  }
}
// Mid-run, from anywhere that knows runId:
// await sidecar.client.cancelQuery(runId);

Rules an integration must respect:

• Mint run_id yourself (crypto.randomUUID()) and keep it — it is the cancel handle, valid from the instant the request is sent.
• Exactly one terminal event. A terminal error is yielded (surface detail verbatim); transport/contract failures throw (HttpError non-2xx, QueryStreamError for a non-SSE response / malformed event / stream that closes without a terminal). Never treat iterator completion without a final as success — the client already throws for you.
• Gate can_answer_questions on the peer's version. Call version() first: a sidecar below apiVersion 2.6 does not know the field and answers 422 to every request carrying it — including false. Omit it below 2.6 and treat mid-run questions as unavailable.
• Declare can_answer_questions honestly. It defaults to false. Set it true only when a human is watching a UI that renders the question; a one-shot or batch job must leave it off, and then gets an immediate actionable refusal instead of a run parked on a question nobody can see.
• Answer needs_input, do not restart. The run is parked on the SAME stream. Call respondToQuery(runId, ev.request_id, value) and keep iterating the existing iterator — issuing a fresh query() abandons the paused run. value is an option's value (its label also works) or free text when allow_free_text. Render every option's description: the label alone does not tell the user what they are agreeing to. When sensitive is set, mask the input and never log it. Ignoring the question is safe but wasteful — the run ends with an error after timeout_seconds.
• Handle the default branch. A type outside the canonical vocabulary arrives as { type: "unknown", eventType, raw } — render an "unsupported event" placeholder or log it; it is never silently dropped.
• Long runs are normal. timeoutMs bounds time-to-first-response only. To abort from the client side pass query(req, { signal }) AND call cancelQuery so the sidecar stops the loop, not just the socket.

A confirmation-requiring step (a destructive tool such as send_now) emits needs_confirmation then ends with a final refusal pointing at the fixed-function route — mint a token via draft(), then send() (stateless stub, epic decision D1; confirm_url omitted). That is an approval and stays terminal and deny-by-default; a question (needs_input) is the resumable one. Do not treat them alike.

Mailbox setup is the agent's job now (#2469). When the agent has no usable mailbox — not connected, credentials broken, missing a scope, or connected-but-not-granted — it asks the user about that specific problem via needs_input and fixes it, rather than returning an error telling them to run a CLI command. Two cases are worth knowing: the connected-but-not-granted case needs no browser at all (a local permission write), and connecting Google still requires the user to supply their own OAuth client ID and secret, so expect a sensitive: true question on that path.

Mail-required, calendar-optional (#2730). Every setup/reconnect path — this self-repair flow included — requests the full mail + calendar scope union at consent time, but only the mail scopes gate whether the flow reports success. A user who declines calendar still ends up with a working mailbox; calendar tools raise their own actionable error, naming the exact scope, the first time one is actually called. Do not "fix" a self-repair flow that requests only mail scopes — that narrower request is the bug this issue removed, not a simplification to reintroduce.

Stateful agent surface (/v1/email/agent/*, 0.4.0)

Everything above is stateless — you send a payload, the sidecar analyzes it, no memory, no conversation. The sidecar also hosts a session-scoped, conversational agent that runs the full EmailTriageAgent (memory, personalization, every agent tool) over HTTP. This is the surface the Agent UI uses. It is not wrapped by the typed EmailClient yet — call it directly with fetch against the sidecar's baseUrl:

jsCopy code
const base = "http://127.0.0.1:8131";
// 1. Start a session (builds the agent; reports memory availability).
await fetch(`${base}/v1/email/agent/session`, {
  method: "POST", headers: { "content-type": "application/json" },
  body: JSON.stringify({ session_id: "s1" }),
});

// 2. Run a turn — the reply streams back as Server-Sent Events.
const res = await fetch(`${base}/v1/email/agent/query`, {
  method: "POST", headers: { "content-type": "application/json", accept: "text/event-stream" },
  body: JSON.stringify({ session_id: "s1", message: "Triage my inbox" }),
});
const reader = res.body.getReader(); const dec = new TextDecoder(); let buf = "";
for (;;) {
  const { value, done } = await reader.read(); if (done) break;
  buf += dec.decode(value, { stream: true });
  let i; while ((i = buf.indexOf("\n\n")) >= 0) {
    const line = buf.slice(0, i).split("\n").find(l => l.startsWith("data: "));
    buf = buf.slice(i + 2);
    if (!line) continue;
    const ev = JSON.parse(line.slice(6));           // {type: "thinking"|"step"|"permission_request"|"run_complete"|...}
    if (ev.type === "permission_request") {         // a gated tool (send/forward/delete/...) is waiting
      await fetch(`${base}/v1/email/agent/confirm-tool`, {
        method: "POST", headers: { "content-type": "application/json" },
        body: JSON.stringify({ session_id: "s1", approved: true }),
      });
    }
    if (ev.type === "run_complete") console.log("answer:", ev.answer);
  }
}

Other endpoints: POST /cancel, DELETE /session/{id}, GET /session/{id}/history, and the runtime memory toggle POST /memory + GET /memory/{id} (enabling memory that was never initialized returns 409, never a silent no-op). One turn at a time per session — an overlapping /query returns 409. See SPEC.md for the full table.

Full autonomy (/v1/email/agent/autonomy/*)

The agent can run proactively at the earn_trust level: it archives low-signal (promotional/spam) mail and marks FYI mail read on its own where your explicit preferences already sanction it (a low-priority sender, or a category you default to archive) or a sender/category has earned enough trust, and always asks before anything destructive (send / forward / RSVP / quarantine). There is no permanent-delete — the agent only ever moves mail to Trash, which is always reversible. Reply drafting is not yet wired into this proactive loop (the policy layer supports it, but no candidate reaches it today). Turn it on and inspect the earned trust:

jsCopy code
// Turn on full autonomy (levels: off | suggest | earn_trust | full; "off" = kill switch)
await fetch(`${base}/v1/email/agent/autonomy`, {
  method: "POST", headers: { "content-type": "application/json" },
  body: JSON.stringify({ session_id: "s1", level: "earn_trust" }),
});

// Run one observe→decide→act cycle now (the daemon/scheduler drives this in production)
const r = await fetch(`${base}/v1/email/agent/autonomy/run`, {
  method: "POST", headers: { "content-type": "application/json" },
  body: JSON.stringify({ session_id: "s1", max_messages: 25 }),
});
const report = await r.json();
// { level, executed:[…], proposals:[…], decisions:[…], skipped }
// decisions[] explains EVERY candidate considered: { message_id, tool, action, outcome, reason, sender }

// Inspect the earned-trust ledger — autonomy is never a black box
const status = await (await fetch(`${base}/v1/email/agent/autonomy/s1`)).json();
// { level, enabled, trust_min_samples, trust_threshold, trusted_scope_count, scopes:[…] }

The agent learns from your corrections: undoing an auto-executed action — POST /v1/email/agent/autonomy/undo with { session_id, action_id } from the executed[] entry, or the conversational undo_archive_batch tool for a batch archive — is captured as a negative outcome that pulls the sender/category back below the trust bar. (Positive-outcome accrual — trust rising as suggestions are accepted or left standing — is not yet wired, so today the ledger only ratchets trust down.) Every auto-action is reversible with undo. A bad level returns 400; an unknown session returns 404; undoing an unknown/expired action_id returns 409; /run while the level is off returns 409 too — it refuses rather than returning the same 200 shape a real, found-nothing cycle would (#2528).

The Python host also ships a thin-client CLI over this same surface: gaia email autonomy {status|set-level|pause|resume|run|trust|kill} (#2516).

Skill sets — disabled in this release

The sidecar bundles six Agent Skills (personal: inbox-triage, newsletter-digest, travel-itinerary; work: inbox-triage, meeting-scheduling, action-item-extraction, escalation-routing), but the agent's manifest currently declares no sets, so none of them loads. A personal and a work mailbox get identical behaviour. This is deliberate: the skills are held back until an eval run shows they improve triage.

What that means for your integration:

• Do not pass --skill-set or GAIA_EMAIL_SKILL_SET. Any value fails at startup with ... but this agent declares no skill sets — Agent Skills are switched off in this build. There is no working name. This is fail-loud behaviour, not a bug to work around.
• GAIA_EMAIL_ACCOUNT_TYPE still validates but selects nothing.
• Nothing in the API changes either way — same endpoints, same tools, same permissions. Re-enabling happens inside the agent's gaia-agent.yaml; your code does not change.

Running in a server / long-lived app

• fetchBinary is a build step, not per request (network + SHA verify). Run it once; resolveBinaryPath at runtime.
• Spawn once at boot, hold the Sidecar handle for the process lifetime — never per request.
• Low concurrency. One local Lemonade model slot, so parallel triage calls serialize. Cap inflight calls.
• Cleanup is automatic (default autoCleanup): the sidecar's child is reaped on exit/crash/signal. Call shutdown for a graceful stop (it rejects, naming the pid, if the sidecar survives the forced kill), or autoCleanup: false to wire signals yourself. The package does not restart a crashed sidecar.

Fast local iteration (when you need to fix the agent, not just call it)

The steps above spawn a frozen binary — you can't edit it. To debug or improve the agent, run its Python source and attach the same client. The frozen binary is that source frozen, so the contract is identical; only the base URL changes.

bashCopy code
pip install -e hub/agents/email/python     # editable install
gaia-agent-email serve --reload            # source server on 127.0.0.1:8131, auto-reload

tsCopy code
import { connectSidecar } from "@amd-gaia/agent-email";
// Attaches (health + version check), spawns nothing, token off in dev:
const dev = await connectSidecar({ baseUrl: "http://127.0.0.1:8131" });
await dev.client.triage({ payload: { /* … */ } });
// Edit the Python under gaia_agent_email/, save → reload → re-run. Seconds.

npx @amd-gaia/agent-email dev launches the serve process for you (--python <path> to use a specific venv). There's no child on the returned handle and nothing to shutdown() — you own the serve process (Ctrl+C). Switch back to production by using startSidecar (frozen binary) instead of connectSidecar; the client calls are unchanged.

Prerequisites — the agent needs a local model

The sidecar runs the LLM via Lemonade Server, which this package does not install. Before triage/draft/send succeed, the host must have:

1. A running Lemonade Server. GAIA's daemon starts and supervises one, so gaia daemon start is normally all that is needed.
2. The model pulled (gaia init installs Lemonade and downloads the default model).

Until then the binary boots, but the first triage returns HTTP 502.

Gotchas (read before debugging)

• Every /v1/email/* call needs the session token (#1706). sidecar.client carries it automatically; a client you construct yourself must pass authToken (from sidecar.authToken) or every call is 401. Absent or non-loopback Host → 400, non-loopback browser Origin → 403. /health · /version · /v1/email/spec · /v1/email/playground are exempt.
• health() is liveness-only. A green /health means the REST surface is up, NOT that triage will work. For real readiness call init() (GET /v1/email/init, #1795) — it probes Lemonade + the triage model and returns the InitResponse on both the ready (200) and not-ready (503) paths, so branch on .ready / read .hint. POST /v1/email/init streams a model-pull (no wrapper yet).
• HTTP 502 from triage → Lemonade isn't running/reachable, or the model isn't pulled. It is not a bug in this package.
• Addresses are objects, not strings. to (and triage's from / principal) are { email, name? }; to is a non-empty array of them. A plain string → 422.
• send needs the draft confirmation_token (missing/invalid → 403), but it takes no OAuth token — the mailbox is resolved from the host's GAIA connector store (no mailbox connected → 503). The read-only search / prescan resolve the mailbox the same way (503 with none, 400 with 2+). Triage and draft need no connector.
• Attachments bind to the token (schema 2.2). Re-send the exact attachments array you drafted with — the metadata-only draft echo has no content_base64, so spreading the echo into send loses the files. A swapped/extra/missing attachment → 403; bad base64, a malformed MIME type, or > 25 MB decoded → 422; Outlook additionally rejects files over 3 MB (Graph simple-attach limit).
• archive / quarantine are gated like send, but their token comes from confirmAction (not draft) and is bound to the (action, message_id) — a token for one can't authorize the other. Undo with unarchive (pass the returned batch_id) / unquarantine (pass the action_id) within 30s; past the window the reversal returns 409 (restore manually in the mail client). For Outlook, use the post_archive_id from the archive response — the folder move changes the id.
• Cleanup is automatic by default — the sidecar is reaped on exit/crash/signal; only autoCleanup: false (or a hard SIGKILL of your process) can orphan the child. shutdown stays the graceful stop.
• OAuth forward-out is daemon-only (sidecar contract 2.5, #2154). The /v1/connections/{provider} intake exists for the GAIA Agent UI daemon to forward short-lived access tokens to the sidecar (the sidecar never holds the refresh token). A standalone integrator using this package does not call it — keep resolving the mailbox from the host's GAIA connector store as before. There is no client.forwardConnection() method, by design.
• Some capabilities are agent-loop-only — no REST endpoint, no client method. Scheduled send / snooze (#1609), voice / style-matched drafting (#1607 — build_voice_profile learns a local style profile from Sent mail so drafts come out in the user's own voice), follow-up tracking (#1606 — check_followups flags sent mail still awaiting a reply, detection only), and waiting-on-you detection (#2581 — list_waiting_on_you flags INBOUND mail awaiting the user's reply; it only qualifies a message that has both a genuine ask/meeting-time signal and corroboration that it's real correspondence) all run in the agent tool loop. The REST contract has no routes for them yet, so don't look for client.scheduleSend() / client.snooze() / a voice, follow-up, or waiting-on-you method — they don't exist (and none of these moves SCHEMA_VERSION).
• --skill-set / GAIA_EMAIL_SKILL_SET always fail right now. Agent Skills are disabled in this release, so the agent declares no sets and every name is invalid. Don't wire either into your spawn options.
• ESM-only. require("@amd-gaia/agent-email") fails; use import / dynamic import().

Verify the integration

A green path looks like: fetchBinary succeeds → startSidecar resolves → client.triage(...) returns a result with a category and summary. If triage 502s, start Lemonade and pull the model, then retry — the rest of your integration is fine.

To eyeball the agent by hand without writing any code, run npx @amd-gaia/agent-email playground — it fetches the binary, starts the sidecar, and opens an interactive page where you can fire triage/draft and see a stack-health check.

For the full endpoint list, lifecycle internals, and connector details, see SPEC.md next to this file.