AI-Output Verifier
AI is fluent, confident, and sometimes completely wrong — inventing facts, citations, and details in the same authoritative tone as the correct ones. That confidence is exactly what makes unverified trust dangerous. This checks a specific output: which claims are most likely wrong or fabricated, what genuinely needs independent verification, how to verify it, and the tells of hallucination — so you use AI's speed without inheriting its errors.
What This Skill Produces
- A risk read of the output — which specific claims are most likely to be wrong, outdated, or made up (facts, numbers, citations, names, recent events, specifics)
- Verify vs. low-risk split — what genuinely needs independent checking vs. what's low-stakes or self-evident, so you spend effort where it counts
- How to verify each — the concrete way to check the high-risk claims (a primary source, a second tool, a domain expert, testing it)
- The hallucination tells — the signs AI is likely fabricating (oddly specific citations, confident claims about recent/niche facts, plausible-but-unverifiable details)
- A verification habit — how to build appropriate checking into your AI use by default, scaled to the stakes (trust more for low-stakes, verify hard for high-stakes)
Required Inputs
Ask for these if not provided:
- The output — the AI response to check (paste it)
- What it's for — the stakes (a casual question vs. something you'll publish, decide on, or act on)
- The domain — factual/technical/legal/medical/current-events (some are far higher-risk for AI)
- What you'd do with it — trust it, act on it, share it, build on it
Framework: Risk-Rate The Claims, Verify What Matters
- Scan for the high-risk claim types. Specific facts, numbers, dates, names, citations, recent events, and niche/technical specifics are where AI most often invents — flag these.
- Split by risk and stakes. Separate the claims that genuinely need verification (high-risk × high-stakes) from the low-risk or low-stakes ones you can reasonably accept — don't verify everything equally.
- Verify against real sources. For the high-risk claims, check a primary source, a second independent tool, an expert, or by testing — not by asking the same AI "are you sure?" (it'll often just re-confirm).
- Watch the hallucination tells. Oddly precise citations, confident answers about very recent or obscure things, and unverifiable specifics are red flags — treat them as unverified until checked.
- Scale trust to stakes. For low-stakes uses, light verification is fine; for anything you'll publish, decide on, or that could harm if wrong, verify hard. Build this reflex in.
Output Format
Verifying: [the output] · for [use/stakes]
High-risk claims (verify these): [specific facts/numbers/citations/recent/niche → most likely wrong]. Low-risk (reasonable to accept): [self-evident / low-stakes parts]. How to verify each: [primary source / second tool / expert / test — not re-asking the same AI]. Hallucination tells present: [odd-specific citations · confident on recent/niche · unverifiable specifics]. Trust level for your use: [light check for low-stakes / verify hard because it's high-stakes].
Quality Checks
- Flags the specific high-risk claim types in the output
- Splits what needs verification from what's low-risk, by stakes
- Gives concrete verification methods (not "ask the AI again")
- Names the hallucination/overconfidence tells present
- Scales the recommended trust to the actual stakes
Anti-Patterns
- "Verify everything" equally, ignoring stakes.
- Re-asking the same AI "are you sure?" as verification.
- Trusting confident tone as a signal of correctness.
- Missing the high-risk claim types (citations, recent facts, numbers).
- No stakes-based scaling of how hard to check.
Example Trigger Phrases
- "Can I trust this answer the AI gave me?"
- "How do I verify what ChatGPT told me before I use it?"
- "Fact-check this AI output — I'm about to publish it."
- "Is this AI response reliable enough to act on?"
- "What in this AI answer should I double-check?"
