Paperclip Evals

OrganizationPopular

paperclipai

paperclip-evals

Choose, inspect, validate, and report Paperclip Runner or Product E2E evaluations while preserving evidence, provenance, cost, and failure classification.

Overview

Publisher	paperclipai
Repository	`paperclip`
Skill name	`paperclip-evals`
Stars	95.4K
Forks	16.2K
Bundled files	Instructions only
License	MIT
Links	Skill source Repository Homepage

Markdown instructions
A SKILL.md file the model loads on demand, so it only costs tokens when a request actually matches.
Works with any LLM
AI skills are plain Markdown, not provider-specific code, so this works with GPT, Claude, Gemini, Grok, or a local model.
Self-contained
Everything the model needs lives in the instructions — no extra files to sync.
Open source
Published by paperclipai on GitHub. Read the source before you install it.

Installation

Install the Paperclip Evals AI skill in TypingMind to use it with any LLM, or drop it into another agent that reads SKILL.md.

Install in TypingMind

TypingMind installs a skill straight from its GitHub folder — it reads SKILL.md, bundles the resource files, and stores the result locally.

Open the app and go to Plugins → Skills.
Choose "Install from GitHub".
Paste the skill folder URL below and confirm.
https://github.com/paperclipai/paperclip/tree/master/.agents/skills/paperclip-evals
Enable the skill in any chat where you want it available.

Plugins → Skills → Add skill → From GitHub URL, then paste the folder URL and press Continue.

Install in another agent

Any agent that reads the Agent Skills format can use this skill — copy the folder into that agent's skills directory.

Claude Code — .claude/skills

git clone --depth 1 https://github.com/paperclipai/paperclip.git /tmp/paperclip
mkdir -p .claude/skills
cp -r /tmp/paperclip/.agents/skills/paperclip-evals .claude/skills/paperclip-evals

Restart Claude Code after copying so it picks up the new skill.

Use it in TypingMind

Enable Paperclip Evals in any TypingMind chat and the model takes it from there. Its name and description sit in the system prompt, and the moment a request matches, the model loads the full instructions itself — you never invoke it by hand, and it costs no tokens until it is actually used.

The model loads Paperclip Evals on its own as soon as a request matches it.

Works with any AI model

AI skills are plain Markdown instructions rather than provider-specific code, so Paperclip Evals is not tied to the model it was written for. Install it once in TypingMind and use it with GPT-5, Claude, Gemini, Grok, DeepSeek, Mistral, Llama, or a local model you run yourself — all on your own API keys.

Loaded only when it is needed
The system prompt carries just the name and description. The instructions are fetched on the first matching request, so an idle skill costs nothing.
Switch models mid-chat
Because the skill is instructions rather than code, changing model does not break it — the next model reads the same SKILL.md.

Skill instructions

This is the SKILL.md content the model loads. Read it before installing — a skill is instructions your model will follow.

Paperclip evals

Use this skill when a request concerns Paperclip evaluation selection, interpretation, evidence, history, or a live run. Read doc/evals.md in the Paperclip repository first. It defines the two families and their boundaries.

Discover the repository

Do not assume the skill's installed location is inside a checkout. Locate the repo explicitly with git rev-parse --show-toplevel from the current directory, or inspect likely workspace roots and select the checkout containing package.json, tests/runner-e2e, and packages/paperclip-runner. Locate the private sibling paperclip-evals only when a Runner Eval needs its definitions; use an explicit PAPERCLIP_EVALS_ROOT or a discovered sibling checkout. Never invent a relative path from this copied skill into the repository.

Route the request

Choose Runner Evals for real runner/provider protocol behavior against the mock control plane. Authoritative details are in packages/paperclip-runner/docs/runner-protocol-live-evals.md and the sibling paperclip-evals/evals/paperclip-runner definitions.

Choose Product E2E Evals for real browser/server/database/runner/provider workflows, including local and Daytona environments. Read tests/runner-e2e/README.md, then FIXTURES.md, SECURITY.md, or EVERYDAY-WORKFLOWS.md as relevant. Everyday Workflows remain Product E2E even when imported into Evalbook. “Headless” is a browser mode, not a family.

Work safely

Start with read-only catalog inspection and credential-free validation. For Product E2E use pnpm test:e2e:runner:typecheck, pnpm test:e2e:runner:unit, and pnpm test:e2e:runner -- --list; run one explicit cell only when the user has authorized a live/paid run and the needed credentials and immutable Daytona image are configured. For Runner Evals use the pinned eval revision and the documented workflow/CLI. Never use a partial selector as evidence of full coverage.

Keep source revisions, definition/catalog fingerprints, model/profile, environment, selected cells, retries, timing, usage/cost coverage, and grader version attached to every interpretation. Preserve partial attempts and classify failures as product, model/provider behavior, grading/evidence, or infrastructure from the observed failure and supported cause. A usable completed behavior failure is not infrastructure; missing provider/profile, transport, startup, or evidence requires examining the evidence before choosing the cause.

Use the existing family generator and viewer. Public projections may contain sanitized fixture conversation and allowlisted tool outcomes/evidence; follow the family's projection and publisher checks. Do not expose raw trusted artifacts, credentials, secrets, private data, provider session IDs, or hidden reasoning. A refresh from retained evidence has zero provider calls and remains the original measurement with a new presentation. Link the public histories and hub from doc/evals.md when reporting results.

For adding a case or fixture, use the narrower add-runner-eval or add-product-e2e-eval skill.

Frequently asked questions

What does the Paperclip Evals AI skill do?

Choose, inspect, validate, and report Paperclip Runner or Product E2E evaluations while preserving evidence, provenance, cost, and failure classification.

Why use Paperclip Evals on TypingMind?

Because you install it once and use it with any model. Paperclip Evals is plain Markdown rather than provider-specific code, so the same skill runs on GPT-5, Claude, Gemini, Grok, or a local model — and you can switch model mid-chat without it breaking. TypingMind runs on your own API keys, so you pay providers directly instead of a per-seat subscription, and your skills and chats stay in your own storage.

How do I install Paperclip Evals in TypingMind?

Open Plugins → Skills → Install from GitHub in TypingMind and paste https://github.com/paperclipai/paperclip/tree/master/.agents/skills/paperclip-evals. TypingMind reads its SKILL.md and installs it as a skill you can enable per chat.

Which AI models can use Paperclip Evals?

Any model you connect in TypingMind. AI skills are plain Markdown instructions rather than provider-specific code, so GPT, Claude, Gemini, Grok, and local models can all load this skill when a request matches it.

How many AI models can I use with Paperclip Evals?

As many as you like. As long as a model supports skills, you can use Paperclip Evals with it — GPT, Claude, Gemini, Grok, DeepSeek, Mistral, Llama and more — all on TypingMind with your own API keys.

Is the Paperclip Evals AI skill free?

Yes. It is published on GitHub by paperclipai under the MIT license. You only pay your own AI provider for the tokens you use.

What are AI skills?

An AI skill is a reusable instruction bundle that teaches an AI model how to do one specific task. It follows the open Agent Skills format: a SKILL.md file with a name and description, plus any scripts, templates or reference files the model may need. The model reads the instructions only when your request matches the skill, so an installed skill costs nothing until it is used.

How are AI skills different from plugins or MCP servers?

A plugin or MCP server gives a model new tools to call — code that runs somewhere and returns a result. An AI skill gives the model knowledge and process instead: how to approach a task, which steps to follow, what good output looks like. Skills are plain Markdown, so they need no server, no API key and no runtime, and they work with any model.

View all

Openclaw Live Updater

openclaw

OrganizationPopular

Maintain the canonical live OpenClaw main checkout, macOS LaunchAgent-managed Gateway, local macOS app, exact-head main CI, and recurring full release validation. Use for fast-forward update…

Openclaw Pr Maintainer

openclaw

OrganizationPopular

Review, triage, repair, or land OpenClaw issues and pull requests with current-source evidence and the native maintainer workflow.

Openclaw Refactor Docs

openclaw

OrganizationPopular

Refactor an existing OpenClaw docs page with source-audited preservation, restructuring, and verification.

Openclaw Repair Sweep

openclaw

OrganizationPopular

Run scoped OpenClaw issue/PR repair campaigns: coordinate workers, prove root causes, and land or close verified work under the requested authority.

Overview

Installation

Install in TypingMind

Install in another agent

Use it in TypingMind

Works with any AI model

Skill instructions

Paperclip evals

Discover the repository

Route the request

Work safely

Frequently asked questions

Related AI Skills

Set up your own AI workspace now