Subject Line Lab

Subject Line Lab

Stop A/B testing blind. This skill reads your actual last 100+ sends, finds the subject-line patterns that correlate with open rate on your list, and generates 20 tuned candidates for your next campaign.

Requires: Cogny MCP + a connected ESP (Klaviyo, Mailchimp, Rule, Get a Newsletter, Brevo, MailerLite, Salesforce Marketing Cloud, Symplify or Braze). Sign up

Usage

/subject-line-lab — mine patterns and generate generic candidates /subject-line-lab "Black Friday doors open" — mine patterns and generate candidates for a specific campaign topic

Step 0: Is Cogny connected?

If no mcp__cogny__* tools exist (no list_integrations, no cogny_status) and no Solo per-ESP tools (mcp__klaviyo__*, mcp__mailchimp__*, mcp__rule__*, mcp__get_a_newsletter__*) either, print this and stop:

Copy code
Subject Line Lab needs your marketing data through Cogny's MCP gateway.

1. Sign up at https://cogny.com and connect your ESP (Klaviyo, Mailchimp, Rule, Get a Newsletter, Brevo, MailerLite, Salesforce Marketing Cloud, Symplify or Braze).
2. Add Cogny to .mcp.json in your project:
   {"mcpServers": {"cogny": {"type": "http", "url": "https://app.cogny.com/mcp"}}}
3. Restart Claude Code; it opens a browser for the OAuth login on first use.

Nothing to run without Cogny.

Prerequisites Check

Detect which ESP is connected. Check both namespaces (Cloud-aggregated via mcp__cogny__<svc>__* and Solo/Lite per-ESP via mcp__<svc>__*):

• Klaviyo → mcp__cogny__mcp__Klaviyo__* or mcp__klaviyo__*
• Mailchimp → mcp__cogny__mcp__Mailchimp__* or mcp__mailchimp__*
• Rule → mcp__cogny__mcp__Rule__* or mcp__rule__*
• Get a Newsletter → mcp__cogny__mcp__Get_a_Newsletter__* or mcp__get_a_newsletter__*
• Brevo, MailerLite, Salesforce Marketing Cloud, Symplify, Braze → Cloud namespace only (mcp__cogny__mcp__<Prefix>__*)

On Cloud, mcp__cogny__list_integrations lists what is connected; treat any integration whose health_status isn't healthy as broken and name it.

Health check: health_status misses revoked tokens, so make one cheap read per ESP before relying on it (Klaviyo get_account, Mailchimp tool_get_account_info, Rule tool_list_tags, Get a Newsletter tool_list_senders, Brevo/MailerLite list_*campaigns with a limit of 1, SFMC list_email_sends for one day, Symplify get_campaign_overview for one day, Braze braze_list_campaigns). A 401, Forbidden or scope error means 🔴 "reconnect in Cogny": name it in the output, don't retry, and continue with the other ESPs.

Missing tool: if a tool named here isn't exposed by the workspace (older connections cache their tool list), call mcp__cogny__refresh_mcp_discovery once, then use the fallback named next to the tool for this run, mark that source ⚠ "needs reconnect", and tell the user to reconnect the Cogny MCP. Never loop on refresh. Get a Newsletter fallbacks: tool_get_report_have_not_opened → that report's list contacts minus the openers from tool_get_report_opens; tool_get_report_link_clicks → tool_get_report_opens (per-contact click counts) or tool_get_report_clicks (per-link totals); tool_get_report_opens_aggregated → the totals in tool_get_report. Mailchimp tool_get_campaign_content → tool_get_campaign for the headers, and ask the user for the body.

ESP connected but 0 sends in the window: skip it with a one-line note ("Mailchimp: connected, 0 broadcast sends in the last days — skipped") and run the rest.

If Cogny is connected but no ESP is, say: "Connect your ESP (Klaviyo, Mailchimp, Rule, Get a Newsletter, Brevo, MailerLite, Salesforce Marketing Cloud, Symplify or Braze) in Cogny", and stop.

If multiple are connected, prompt the user which one to analyze (or analyze all, labeled). Never pool sends from two ESPs into one baseline — open-rate definitions differ (see below). Run steps 2–5 per ESP and show the results side by side.

Works with

ESP tool adapter

Each ESP exposes subject-line history differently. Use the right tool per connected service:

Steps

1. Pull send history

For the connected ESP, pull the last 100 campaign sends (or as many as available). For each, capture:

• Subject line
• Preheader (if available)
• Send date + time (recipient local time if available)
• Recipient count
• Unique opens → open rate
• Unique clicks → CTR
• Segment / list name
• Campaign type (promo, newsletter, announcement, transactional-adjacent)

Skip transactional sends and automated flow emails — they distort the signal. Focus on broadcast campaigns. Exclude test sends too: recipient count under 10, a subject/name containing "test", "[TEST]", "testutskick" or "proof", or a send to a list/audience whose name contains "test", "intern", "internal" or "seed" (case-insensitive; internal seed lists can be dozens of people, so recipient count alone misses them). Report how many were excluded. (Brevo: type: "classic" only, never trigger; SFMC: never triggered sends; Braze: skip API-triggered and Canvas sends.)

2. Compute baseline

• Median open rate across the sample
• Mean open rate (flag skew if mean ≠ median by >20%)
• Sample size warning — if <30 sends, warn the user: "small sample, patterns may be noise"

3. Extract subject-line features

For each subject line, tag these features (use keyword/regex heuristics — no ML needed):

Structural:

• Length (chars): <30 / 30-50 / 50-70 / >70
• Word count
• Title Case / Sentence case / ALL CAPS / lowercase
• Ends in ? / ! / . / no punctuation

Content:

• Contains emoji (count, type)
• Contains number / digit
• Contains % or $ or currency (promo signal)
• Contains personalization token ({{ first_name }}, [Firstname])
• Question vs statement vs command
• First-person ("I", "we", "my") vs second-person ("you", "your")
• Urgency words: "today", "tonight", "last chance", "ends", "hours", "don't miss"
• FOMO / scarcity: "only", "limited", "few left", "selling fast"
• Curiosity gap: starts with "why", "how", "the truth about", "what I learned"
• Benefit-led: leads with a noun describing an outcome
• Social proof: "customers", "members", a number of people

Temporal:

• Day of week sent
• Hour of day (bucketed: early morning / morning / midday / afternoon / evening / late)

4. Compute lift per feature

For each feature, compute:

• Count in sample
• Mean open rate when feature is present
• Mean open rate when feature is absent
• Lift (% difference vs non-feature baseline)
• Significance note — flag as "strong" if n≥10 in both groups and lift is ≥10% relative; "weak" otherwise

Rank features by absolute lift.

5. Identify winning patterns

Output the top 5 positive patterns and top 3 negative patterns with specific numbers:

Copy code
Patterns that worked for your list (last 100 sends):

🟢 WINNERS
1. Subject lines ending in "?" → 34% open rate vs 21% baseline (+62%, n=14 strong)
2. Subject lines with numbers → 29% open rate vs 21% (+38%, n=22 strong)
3. Length 30-50 chars → 27% open rate vs 19% (+42%, n=31 strong)
4. First-person voice ("I", "we") → 26% open rate vs 22% (+18%, n=18 strong)
5. Sent Tue 09:00-11:00 → 28% open rate vs 22% (+27%, n=12 weak)

🔴 DRAGS
1. ALL CAPS words → 14% open rate vs 23% (-39%, n=8 weak)
2. Emoji at start → 18% open rate vs 24% (-25%, n=11 strong)
3. Urgency words ("last chance", "today only") → 16% open rate vs 22% (-27%, n=9 weak)

Your best historical subject: "<actual subject>" at <open rate>% (<date>)
Your worst: "<actual subject>" at <open rate>% (<date>)

6. Generate tuned candidates

If a campaign topic was provided, generate 20 subject line candidates that follow the winning patterns and avoid the drags. If no topic was provided, generate 20 generic promo/newsletter candidates using your patterns.

Format:

Copy code
Candidates for: "<topic>"
(tuned to: ends-in-?, has-number, 30-50 chars, first-person voice)

Preheader recommendation: 85-100 chars, don't duplicate subject, extend the hook.

1. "What <specific number> of our customers asked last week?" (52 chars)
   Preheader: "We pulled the top 5 questions and answered them in one place — you'll recognize #2."
2. ...
20. ...

──
A/B test plan:
- Primary variant: candidate #<N> (leans hardest into top pattern)
- Challenger: candidate #<M> (different angle to avoid pattern overfitting)
- Hold-out: your house-style baseline subject
Segment: X% / Y% / Z%

7. Persist patterns as context

Save the winning-pattern summary to Cogny's context tree for future sessions:

jsonCopy code
{
  "path": "insights/email/subject-line-patterns",
  "body": "<the top 5 winners + top 3 drags, with numbers and date range>"
}

8. Record the finding

If the analysis surfaces a clear actionable insight (e.g., "50% of recent sends use emoji-at-start which is actively hurting opens"), record it: on Cloud call create_ticket first (fields mapped per section 2 of the routing guide); on Solo call create_finding:

jsonCopy code
{
  "title": "Emoji-at-start subjects are tanking opens (-25% vs baseline)",
  "body": "11 of last 100 sends started with emoji. Avg open 18% vs 24% non-emoji baseline. Recommend: move emoji to mid-subject or drop entirely on next 10 campaigns and re-measure.",
  "action_type": "subject_line_optimization",
  "expected_outcome": "Lift average open rate by 2-4 pp",
  "estimated_impact_usd": 0,
  "priority": "medium"
}

Notes

• Lift numbers are correlational, not causal — always surface sample size and flag weak signals.
• If the user's list is <5,000 subscribers, opens are noisy. Recommend 2-week lookback minimum and warn.
• Apple MPP inflates open rate for iOS Mail users. If open rate looks uniformly high (>40%+ across everything), the signal is degraded — note this and lean on CTR as a secondary metric.
• Name the ESP on every number ("Brevo: 31% open rate"). Klaviyo, Mailchimp and Brevo count opens differently around MPP; SFMC and Symplify report their own computed rates. Compare patterns across ESPs, never raw rates.

Recording findings

Use create_ticket when the workspace has it (Cloud) and create_finding otherwise (Solo). The fields differ; see section 2 of ../marketing-engineer/references/mcp-routing.md.