Mutation Testing logo

Mutation Testing

OrganizationPopular
trailofbits
mutation-testing

Configures mewt or muton campaigns, analyzes surviving mutants, and investigates bugs exposed by testing gaps. Use when setting up mutation testing, reviewing campaign results, identifying equivalent mutants, or finding bugs from surviving mutations.

Overview

Publishertrailofbits
Repositoryskills
Skill namemutation-testing
Stars
7.1K
Forks
611
Bundled files
11
LicenseCC-BY-SA-4.0
Links
  • Markdown instructions

    A SKILL.md file the model loads on demand, so it only costs tokens when a request actually matches.

  • Works with any LLM

    AI skills are plain Markdown, not provider-specific code, so this works with GPT, Claude, Gemini, Grok, or a local model.

  • 11 bundled files

    Scripts, templates, and references the model can read while it works. Files are read-only and never executed.

  • Open source

    Published by trailofbits on GitHub. Read the source before you install it.

Installation

Install the Mutation Testing AI skill in TypingMind to use it with any LLM, or drop it into another agent that reads SKILL.md.

1

Install in TypingMind

TypingMind installs a skill straight from its GitHub folder — it reads SKILL.md, bundles the resource files, and stores the result locally.

  1. Open the app and go to Plugins → Skills.
  2. Choose "Install from GitHub".
  3. Paste the skill folder URL below and confirm.
  4. Enable the skill in any chat where you want it available.
Plugins → Skills → Add skill → From GitHub URL, then paste the folder URL and press Continue.
2

Install in another agent

Any agent that reads the Agent Skills format can use this skill — copy the folder into that agent's skills directory.

Claude Code — .claude/skills
git clone --depth 1 https://github.com/trailofbits/skills.git /tmp/skills
mkdir -p .claude/skills
cp -r /tmp/skills/plugins/mutation-testing/skills/mutation-testing .claude/skills/mutation-testing
Restart Claude Code after copying so it picks up the new skill.

Use it in TypingMind

Enable Mutation Testing in any TypingMind chat and the model takes it from there. Its name and description sit in the system prompt, and the moment a request matches, the model loads the full instructions itself — you never invoke it by hand, and it costs no tokens until it is actually used.

The model loads Mutation Testing on its own as soon as a request matches it.

Works with any AI model

AI skills are plain Markdown instructions rather than provider-specific code, so Mutation Testing is not tied to the model it was written for. Install it once in TypingMind and use it with GPT-5, Claude, Gemini, Grok, DeepSeek, Mistral, Llama, or a local model you run yourself — all on your own API keys.

  • Loaded only when it is needed

    The system prompt carries just the name and description. The instructions are fetched on the first matching request, so an idle skill costs nothing.

  • Switch models mid-chat

    Because the skill is instructions rather than code, changing model does not break it — the next model reads the same SKILL.md.

Skill instructions

This is the SKILL.md content the model loads. Read it before installing — a skill is instructions your model will follow.

Mutation Testing (mewt/muton)

Routes to the right mutation testing workflow and loads the references that workflow needs.

Note: muton and mewt share identical interfaces. Examples use mewt; substitute muton and its file names (muton.toml, muton.sqlite) for muton projects.

mewt --help and mewt <subcommand> --help are the source of truth for command-line behavior. Examples below reflect the mewt 4.x API; run --help when a flag looks unfamiliar or a command fails.

When to Use

Use this skill when the user:

  • Mentions "mewt", "muton", or "mutation testing"
  • Wants to configure, scope, or speed up a mutation testing campaign
  • Wants to analyze mutation results — surviving/uncaught mutants, equivalent mutants, kill rate
  • Wants to use mutation results to find bugs in the source code

When NOT to Use

Do not use this skill when the user asks about tests or line coverage without any mutation testing context.


Routing

Pick the workflow, then load it together with the references listed for it. Workflows and references do not load each other — that decision belongs here.

Setting up, scoping, or speeding up a campaignworkflows/configuration.md → Also load references/optimization-strategies.md when the campaign estimate is long enough to need trimming, or the user asks to make it faster.

Campaign finished, hunting for bugs in untested codeworkflows/bug-hunter.md

Turning results into a formal analysis reportworkflows/analyzing-results.md, plus:

Anything else → run mewt --help or mewt <subcommand> --help, then assist directly.


Essential Commands

bash
# Set up and run
mewt init                    # Create config and database
mewt mutate [paths]          # Generate mutants without testing them
mewt run [paths]             # Generate mutants and run the campaign

# Read results
mewt status                  # Overview with per-file breakdown
mewt results                 # Uncaught mutants (default view)
mewt results --all           # Every outcome, not just uncaught
mewt results --format json   # json | sarif | ids | table

# Narrow down (these filters work on both `results` and `print mutants`)
mewt results --target 'src/auth/**'   # Quote globs so the shell does not expand them
mewt results --severity high,medium
mewt results --mutation-types ER,CR
mewt results --status Uncaught        # Uncaught | TestFail | Skipped | Timeout
mewt results --line 42

# Investigate and re-test
mewt print mutant --id [id]              # View the mutated code
mewt test --ids [ids]                    # Re-test specific mutants
mewt test --ids-file uncaught_ids.txt    # Re-test IDs from a file, or '-' for stdin

# Inspect configuration
mewt print config                        # Effective config
mewt print targets                       # Files actually mutated
mewt print mutations --language [lang]   # Mutations and severities for a language

Language labels are canonical family or family/dialect values in mewt 4.x — for example rust, javascript/ts, move/sui, move/iota.


What Results Mean

  • Caught/TestFail: tests detected the mutation (good)
  • Uncaught: tests did not detect the change. Inspect the code to distinguish a testing gap from an equivalent mutation.
  • Timeout: tests took too long — inconclusive, not evidence of coverage
  • Skipped: a less severe mutant was skipped because a more severe mutant on the same line was uncaught

Interpreting Mutation Types

mewt print mutations --language [lang] lists every mutation slug, description, and severity for a language, and is authoritative — the operator set grows with each release. What that output does not tell you is what a survivor means, which is where prioritization comes from:

SeverityRepresentative slugsWhat an uncaught mutant tells you
HighER (Error Replacement)Tests tolerate the injected error. Investigate whether the path executes, whether error handling masks the change, and whether assertions check the outcome.
MediumCR (Comment Replacement)Removing the statement does not fail the tests. Check whether its effects matter and whether assertions observe them.
MediumIF/IT (If False/True), NR (Negation Removal)Tests do not distinguish the changed condition. Both constant replacements surviving can indicate an unexecuted condition or weak assertions on the branch outcomes.
LowOperator shuffles (AOS, COS, LOS, BOS, shift/assignment variants), BL, AS, LC, WFCheck boundary inputs, arithmetic assertions, and semantic equivalence. The mutation result alone does not establish whether the code executed.

Severity ranks the mutation, not the risk. A low-severity survivor in a fee calculation matters more than a high-severity survivor in a log line — weigh what the mutated code does. Filter with --severity to work through the results in priority order.

Bundled files

The model reads these on demand while the skill is loaded. They are exposed as readable files and are never executed.

Frequently asked questions

What does the Mutation Testing AI skill do?

Configures mewt or muton campaigns, analyzes surviving mutants, and investigates bugs exposed by testing gaps. Use when setting up mutation testing, reviewing campaign results, identifying equivalent mutants, or finding bugs from surviving mutations.

Why use Mutation Testing on TypingMind?

Because you install it once and use it with any model. Mutation Testing is plain Markdown rather than provider-specific code, so the same skill runs on GPT-5, Claude, Gemini, Grok, or a local model — and you can switch model mid-chat without it breaking. TypingMind runs on your own API keys, so you pay providers directly instead of a per-seat subscription, and your skills and chats stay in your own storage.

How do I install Mutation Testing in TypingMind?

Open Plugins → Skills → Install from GitHub in TypingMind and paste https://github.com/trailofbits/skills/tree/main/plugins/mutation-testing/skills/mutation-testing. TypingMind reads its SKILL.md and bundles its files and installs it as a skill you can enable per chat.

Which AI models can use Mutation Testing?

Any model you connect in TypingMind. AI skills are plain Markdown instructions rather than provider-specific code, so GPT, Claude, Gemini, Grok, and local models can all load this skill when a request matches it.

How many AI models can I use with Mutation Testing?

As many as you like. As long as a model supports skills, you can use Mutation Testing with it — GPT, Claude, Gemini, Grok, DeepSeek, Mistral, Llama and more — all on TypingMind with your own API keys.

Is the Mutation Testing AI skill free?

Yes. It is published on GitHub by trailofbits under the CC-BY-SA-4.0 license. You only pay your own AI provider for the tokens you use.

What are AI skills?

An AI skill is a reusable instruction bundle that teaches an AI model how to do one specific task. It follows the open Agent Skills format: a SKILL.md file with a name and description, plus any scripts, templates or reference files the model may need. The model reads the instructions only when your request matches the skill, so an installed skill costs nothing until it is used.

How are AI skills different from plugins or MCP servers?

A plugin or MCP server gives a model new tools to call — code that runs somewhere and returns a result. An AI skill gives the model knowledge and process instead: how to approach a task, which steps to follow, what good output looks like. Skills are plain Markdown, so they need no server, no API key and no runtime, and they work with any model.

View all

Set up your own AI workspace now

Get notified about new features and future giveaways by subscribing to our newsletter 👇