TypingMindTypingMindThe best frontend for AI
Regression Watch logo

Regression Watch

CommunityPopular
hoangsonww
regression-watch

Detect quality and efficiency regressions over time using Agent Monitor data — rising error rate (APIError events), falling cache hit rate, growing compaction frequency, and climbing cost-per-session. Splits history into an earlier baseline window and a recent window and reports which metrics are getting worse, by how much, and where. Use when checking whether things are degrading or trending in the wrong direction.

Overview

Publisherhoangsonww
RepositoryClaude-Code-Agent-Monitor
Skill nameregression-watch
Stars
1.1K
Forks
260
Bundled files
1
LicenseMIT
Links
Skill sourceRepositoryHomepage
  • Markdown instructions

    A SKILL.md file the model loads on demand, so it only costs tokens when a request actually matches.

  • Works with any LLM

    AI skills are plain Markdown, not provider-specific code, so this works with GPT, Claude, Gemini, Grok, or a local model.

  • 1 bundled files

    Scripts, templates, and references the model can read while it works. Files are read-only and never executed.

  • Open source

    Published by hoangsonww on GitHub. Read the source before you install it.

Installation

Install the Regression Watch AI skill in TypingMind to use it with any LLM, or drop it into another agent that reads SKILL.md.

1

Install in TypingMind

TypingMind installs a skill straight from its GitHub folder — it reads SKILL.md, bundles the resource files, and stores the result locally.

  1. Open the app and go to Plugins → Skills.
  2. Choose "Install from GitHub".
  3. Paste the skill folder URL below and confirm.
    https://github.com/hoangsonww/Claude-Code-Agent-Monitor/tree/master/plugins/ccam-insights/skills/regression-watch
  4. Enable the skill in any chat where you want it available.
Plugins → Skills → Add skill → From GitHub URL, then paste the folder URL and press Continue.
2

Install in another agent

Any agent that reads the Agent Skills format can use this skill — copy the folder into that agent's skills directory.

Claude Code — .claude/skills
git clone --depth 1 https://github.com/hoangsonww/Claude-Code-Agent-Monitor.git /tmp/Claude-Code-Agent-Monitor
mkdir -p .claude/skills
cp -r /tmp/Claude-Code-Agent-Monitor/plugins/ccam-insights/skills/regression-watch .claude/skills/regression-watch
Restart Claude Code after copying so it picks up the new skill.

Use it in TypingMind

Enable Regression Watch in any TypingMind chat and the model takes it from there. Its name and description sit in the system prompt, and the moment a request matches, the model loads the full instructions itself — you never invoke it by hand, and it costs no tokens until it is actually used.

The model loads Regression Watch on its own as soon as a request matches it.

Works with any AI model

AI skills are plain Markdown instructions rather than provider-specific code, so Regression Watch is not tied to the model it was written for. Install it once in TypingMind and use it with GPT-5, Claude, Gemini, Grok, DeepSeek, Mistral, Llama, or a local model you run yourself — all on your own API keys.

  • Loaded only when it is needed

    The system prompt carries just the name and description. The instructions are fetched on the first matching request, so an idle skill costs nothing.

  • Switch models mid-chat

    Because the skill is instructions rather than code, changing model does not break it — the next model reads the same SKILL.md.

Skill instructions

This is the SKILL.md content the model loads. Read it before installing — a skill is instructions your model will follow.

Regression Watch

Detect whether Claude Code sessions are getting worse over time across quality and efficiency metrics, using Agent Monitor data.

Input

The user provides: $ARGUMENTS

This may be:

  • empty or "all" — check every regression metric (default)
  • "errors" — error-rate regression only
  • "cache" — cache hit-rate regression only
  • "compaction" — compaction-frequency regression only
  • "cost" — cost-per-session regression only
  • A window like "last 30d" or "30 vs 90" — set the recent vs baseline window sizes

Data Sources

EndpointReturns
GET /api/analyticsdaily_events (365d), daily_sessions (365d), event_types, tokens (total_input, total_output, total_cache_read, total_cache_write — baselines pre-summed), avg_events_per_session
GET /api/events?session_id=XEvent stream incl. APIError, Compaction, PreToolUse/PostToolUse — used to localize regressions to specific sessions
GET /api/pricing/cost{ total_cost, breakdown[...] } — total cost to derive cost-per-session
GET /api/pricing/cost/{sessionId}Per-session cost — used to compare recent vs baseline session cost
GET /api/workflows/{sessionId}compaction (impact), errorPropagation (by depth), effectiveness — per-session quality signals
GET /api/sessions?limit=NSessions with started_at, cost, metadata — to bucket sessions into time windows

Report Sections

1. Windowing

Split history into a baseline window (older) and a recent window (newer). Default: recent = last 30 days, baseline = the 30–90 day range before it. Use daily_events/daily_sessions for series metrics and GET /api/sessions?limit=N to assign sessions to each window by started_at.

2. Error Rate Regression

  • Recent error rate = APIError count / total events in the recent window (from event_types and daily_events, or per-session GET /api/events).
  • Compare to the baseline rate. Flag if recent is higher.
  • Report the absolute and relative change and which sessions contributed most APIError events.

3. Cache Hit Rate Regression

  • Cache hit rate = total_cache_read / (total_cache_read + total_input).
  • Compute for each window (per-window input/cache_read from session metadata or the pricing breakdown). Flag a falling hit rate — that means more uncached input tokens and higher cost.

4. Compaction Frequency Regression

  • Compaction frequency = Compaction events / session per window (from event_types / daily_events, confirmed via per-session GET /api/workflows/{id} compaction). Flag a rising rate — context is overflowing more often.

5. Cost-per-Session Regression

  • Cost-per-session = window total cost / window session count, using GET /api/pricing/cost overall and GET /api/pricing/cost/{id} for the sessions in each window. Flag a climbing value.

6. Verdict

Roll up which metrics regressed, rank by relative worsening, and name the most likely driver (e.g., cache hit rate fell → cost per session climbed).

Output

  • A Markdown table: metric | baseline | recent | Δ | direction (▲ worse / ▼ better) | verdict.
  • Tag each regressed metric 🔴 (clear regression), 🟡 (mild/within noise), or 🟢 (improved).
  • Currency in USD to 4 decimals; rates as percentages to 2 decimals.
  • List the specific session IDs that contributed most to any regression.
  • End with the single highest-priority regression to address and a concrete next step.
  • Read-only: only report what the API returns; never fabricate baselines.

Bundled files

The model reads these on demand while the skill is loaded. They are exposed as readable files and are never executed.

Frequently asked questions

What does the Regression Watch AI skill do?

Detect quality and efficiency regressions over time using Agent Monitor data — rising error rate (APIError events), falling cache hit rate, growing compaction frequency, and climbing cost-per-session. Splits history into an earlier baseline window and a recent window and reports which metrics are getting worse, by how much, and where. Use when checking whether things are degrading or trending in the wrong direction.

Why use Regression Watch on TypingMind?

Because you install it once and use it with any model. Regression Watch is plain Markdown rather than provider-specific code, so the same skill runs on GPT-5, Claude, Gemini, Grok, or a local model — and you can switch model mid-chat without it breaking. TypingMind runs on your own API keys, so you pay providers directly instead of a per-seat subscription, and your skills and chats stay in your own storage.

How do I install Regression Watch in TypingMind?

Open Plugins → Skills → Install from GitHub in TypingMind and paste https://github.com/hoangsonww/Claude-Code-Agent-Monitor/tree/master/plugins/ccam-insights/skills/regression-watch. TypingMind reads its SKILL.md and bundles its files and installs it as a skill you can enable per chat.

Which AI models can use Regression Watch?

Any model you connect in TypingMind. AI skills are plain Markdown instructions rather than provider-specific code, so GPT, Claude, Gemini, Grok, and local models can all load this skill when a request matches it.

How many AI models can I use with Regression Watch?

As many as you like. As long as a model supports skills, you can use Regression Watch with it — GPT, Claude, Gemini, Grok, DeepSeek, Mistral, Llama and more — all on TypingMind with your own API keys.

Is the Regression Watch AI skill free?

Yes. It is published on GitHub by hoangsonww under the MIT license. You only pay your own AI provider for the tokens you use.

What are AI skills?

An AI skill is a reusable instruction bundle that teaches an AI model how to do one specific task. It follows the open Agent Skills format: a SKILL.md file with a name and description, plus any scripts, templates or reference files the model may need. The model reads the instructions only when your request matches the skill, so an installed skill costs nothing until it is used.

How are AI skills different from plugins or MCP servers?

A plugin or MCP server gives a model new tools to call — code that runs somewhere and returns a result. An AI skill gives the model knowledge and process instead: how to approach a task, which steps to follow, what good output looks like. Skills are plain Markdown, so they need no server, no API key and no runtime, and they work with any model.

Related AI Skills

View all
openclaw logo

Prototype Openclaw Tui

openclaw
OrganizationPopular

Build throwaway, fixture-driven OpenClaw Clack or Pi TUI prototypes and compare multiple interactive variants side by side in tmux without running the full application or touching live state.

82.2K
391K
2 files
View
openclaw logo

Slacrawl

openclaw
OrganizationPopular

Slack archive: search, sync freshness, threads/DMs, SQL counts, and Slacrawl repo work.

82.2K
391K
1 files
View
openclaw logo

Obsidian Vault Maintainer

openclaw
OrganizationPopular

Maintain an Obsidian-friendly memory wiki vault with wikilinks, frontmatter, and official Obsidian CLI awareness.

82.2K
391K
Instructions
View
openclaw logo

Control Ui

openclaw
OrganizationPopular

Operate and troubleshoot the OpenClaw Control UI: navigate connected clients, organize sessions, build session dashboards, and handle direct or Tailscale-hosted Gateways.

82.2K
391K
2 files
View

Set up your own AI workspace now

Get notified about new features and future giveaways by subscribing to our newsletter 👇