Model-Card Skill
Purpose
This skill produces the documentation an engineer-built medical-imaging model must carry: a
Model Card (intended use, out-of-scope use, training data, per-subgroup performance, caveats), a
Datasheet for its dataset (provenance, composition, collection, labelling, consent), and a
METRIC-informed data-quality pass. It fills the templates from facts the user supplies — it
never invents a number, a provenance detail, a consent status, or a licence — and ships a deterministic
gate that no required section is missing or left as an unfilled [NEEDS INPUT] placeholder.
It is the reporting seam of the model-engineering lane: after /model-assessment audits the design
and produces the numbers, this skill records them in a portable, auditable card that
/write-paper and /check-reporting consume. It mirrors /version-dataset structurally (generate +
deterministic verify).
When to use
- A trained model needs a Model Card / Datasheet for a repo, Hugging Face card, or manuscript supplement.
When NOT to use
- Auditing the validation design / metrics →
/model-assessment. - Versioning the dataset bytes →
/version-dataset; tabular variable docs →/generate-codebook. - Item-by-item reporting-guideline compliance of the manuscript →
/check-reporting. - Building / training the model →
/model-scaffold.
Workflow
Phase 1 — Collect the facts
Gather, from the user / the model's developers: task + architecture + provenance + licence; intended use
and out-of-scope use; training and evaluation cohorts; the reference standard and inter-reader agreement;
overall and per-subgroup performance; data collection, consent, and de-identification. Anything not
supplied stays [NEEDS INPUT] — never guess.
Phase 2 — Fill the Model Card
Copy ${CLAUDE_SKILL_DIR}/references/model_card_template.md to MODEL_CARD.md and fill each section
from the facts. Keep the headings. Numbers come only from /model-assessment / executed results.
Phase 3 — Fill the Datasheet
Copy ${CLAUDE_SKILL_DIR}/references/datasheet_template.md to DATASHEET.md and fill the seven
question groups (Motivation, Composition, Collection, Preprocessing/Labeling, Uses, Distribution,
Maintenance).
Phase 4 — METRIC data-quality pass
Walk ${CLAUDE_SKILL_DIR}/references/metric_dimensions.md (completeness, correctness, consistency,
representativeness, timeliness, provenance, label provenance, fairness/coverage, leakage safety) and
record each finding in the Datasheet. Anything that affects the headline metric's validity is also a
/model-assessment finding — cross-check there.
Phase 5 — Verify completeness (deterministic gate)
bashpython3 ${CLAUDE_SKILL_DIR}/scripts/check_model_card_complete.py \ --card MODEL_CARD.md --datasheet DATASHEET.md --strict
MISSING_SECTION / EMPTY_REQUIRED_SECTION / UNFILLED_FIELD must be zero before the card ships.
UNFILLED_FIELD names each field of a required section still left as [NEEDS INPUT] / [VERIFY]
(e.g. License or subgroup performance), even when a sibling field is filled. An explicit N/A /
None counts as an answer only as a field's whole value, and never in Intended Use, Training Data,
Evaluation Data, Metrics or Quantitative Analyses. Only the template tokens as written
([NEEDS INPUT ...], [VERIFY], [VERIFY: ...], upper case) count; markdown link text such as
[verify the protocol](https://...) or [Verify][ref], and anything inside an HTML comment or a
fenced code block, is not a placeholder.
Known limits: a field left as a hand-written TODO / TBD / <...> / XXXX is caught only when
the whole section is unfilled; keep the template's bracketed [NEEDS INPUT] markers for open fields.
Phase 6 — Hand off
Carry the card into /write-paper (the Methods / supplement reference it), /check-reporting
(CLAIM 2024 / TRIPOD+AI item audit of the manuscript), and /self-review.
Anti-Hallucination
- Never invent evaluation numbers, subgroup results, or dataset provenance. Every figure comes from
/model-assessmentor the user's executed results; every provenance / consent / licence statement is user-confirmed. Unknown →[NEEDS INPUT], which the gate flags. - Never mark a section complete without user-supplied content, and never auto-fill a placeholder to pass the gate.
- Never assert a licence or consent status the user did not confirm.
- The gate checks presence, not truth — a complete card can still contain a wrong number; validity
is
/model-assessmentand the human's responsibility.
Deterministic gate
scripts/check_model_card_complete.py — verifies every required Model Card / Datasheet section is
present and non-empty (stdlib, network-free). Reproducible challenge:
bash ${CLAUDE_SKILL_DIR}/scripts/check_model_card_complete_challenge/verify.sh.
Note on classification
Model Cards (Mitchell et al. 2019) and Datasheets (Gebru et al. 2021) are documentation standards,
not clinical reporting guidelines, so they live here as references/ templates (uncounted), not in
/check-reporting's counted checklist set — the same way appraisal_tools/METRICS.md is kept separate.
/check-reporting still owns the manuscript-level CLAIM 2024 / TRIPOD+AI item audit.
Boundaries
model-assessment (audit design + metrics) └─ model-card (this skill: Model Card + Datasheet + METRIC pass, completeness-gated) └─ write-paper + check-reporting (manuscript) ; version-dataset (dataset bytes)
