Eda Sports

EDA for Sports Data

Outcome

Understand the data well enough that the next decision is explicit. Sports EDA is not one call to describe(): it is a structured review of grain, keys, time coverage, entities, missingness, targets, schedule structure, outliers, structural breaks, and fields unavailable at prediction time T.

Finish with a written go, repair, or stop decision. A clean EDA report does not certify that engineered features are time-safe.

When to Use This Skill

Use when:

• a sports table is new, freshly loaded, or not yet trusted;
• grain, keys, coverage, missingness, or base rates need an honest audit;
• before feature engineering or first model fit;
• a weird model result might just be bad data structure.

Do not use this skill as a substitute for:

• formal leakage verdicts → leakage-audit;
• feature construction → feature-rules / time-series-sports;
• source selection → data-sources.

Required context

Before calculating summaries, record the question, target, prediction time T, claimed row grain and key, source/retrieval time, requested period, completion status, and populations included. If grain or T is ambiguous, resolve it first.

Use the EDA checklist as the full run sheet. Read the grain guide when row meaning, keys, or aggregation are in doubt. Use the red-flags guide when a count, base rate, or model result is odd.

Workflow

1. Define one row: game, team-game, player-game, play, possession, pitch, or event.
2. Count rows/columns; test natural-key uniqueness and rows per contest.
3. List seasons, weeks/rounds, dates, gaps, partial periods, and active seasons.
4. Count teams/players by season and inspect ID churn and failed joins.
5. Measure missingness overall and by season, source, role, and completion state.
6. Inspect targets, base rates, impossible values, ties, overtime, and zero inflation.
7. Slice meaningful predeclared groups: home/away, phase, role, venue, and era.
8. Find rule, schedule, provider, tracking, definition, and population breaks.
9. Flag fields unavailable at T; reserve the formal verdict for leakage-audit.
10. Write findings, repairs, limitations, and the decision.

If grain is wrong or keys are unexplained duplicates, stop before modeling.

Grain and key diagnostics

pythonCopy code
key = ["game_id", "team"]
dupes = panel.duplicated(key, keep=False)
print("duplicate key rows:", int(dupes.sum()))
print(panel.loc[dupes].sort_values(key).head(20))
print(panel.groupby("game_id", dropna=False).size().value_counts().sort_index())

Do not call drop_duplicates() until the cause is known. Repeated pulls, provider revisions, multiple competitions, and mixed grains need different repairs. A normal team-game panel usually has two rows per game.

Coverage and entity balance

pythonCopy code
coverage = (
    panel.groupby(["season", "week"], dropna=False)
    .agg(rows=("game_id", "size"), games=("game_id", "nunique"),
         teams=("team", "nunique"))
    .reset_index()
)
print(coverage.to_string(index=False))
print(panel.groupby(["season", "team"]).size().unstack(0).fillna(0))

Investigate empty periods, low game counts, abrupt entity-count changes, and the tail of an active season. Imbalance can be real—byes, playoffs, injuries, promotion/relegation—but can also reveal missing loads. Use stable IDs and a documented crosswalk for relocations and renames.

The portable coverage helper expects season, week, game_id, and team. Map alternate schemas explicitly:

bashCopy code
python /path/to/eda-sports/scripts/coverage_table.py --input games.csv
python /path/to/eda-sports/scripts/coverage_table.py \
  --input games.parquet --season-col year --period-col round \
  --game-col event_id --team-col club_id

Panel base rates

On a complete two-row team-game panel, overall won.mean() is near 0.5 because each game contributes both perspectives. It is not home advantage.

pythonCopy code
home = panel.loc[panel["is_home"].eq(1)].copy()
print("overall team-row win rate:", panel["won"].mean())
print("home win rate:", home["won"].mean())
print(home.groupby("season")["won"].mean())

Ties, neutral sites, forfeits, incomplete games, and duplicate perspectives can alter this identity. Explain them rather than forcing the expected value.

Missingness

Overall rates can hide an era or provider boundary:

pythonCopy code
critical = [c for c in ["event_time", "points_for", "points_against", "won"] if c in panel]
print(panel[critical].isna().mean().sort_values(ascending=False))
print(panel.groupby("season")[critical].agg(lambda s: s.isna().mean()))

Classify each important pattern as structurally expected, temporarily unavailable, failed collection/join, not applicable, or unknown. Never silently fill early-history rolling nulls with zero; feature rules must define a prior, missing indicator, or minimum-history threshold.

Targets, distributions, and outliers

pythonCopy code
print(home["point_diff"].describe(percentiles=[.01, .05, .5, .95, .99]))
print(panel["won"].value_counts(dropna=False, normalize=True))

pythonCopy code
import matplotlib.pyplot as plt
fig, ax = plt.subplots(figsize=(8, 4.5))
ax.hist(home["point_diff"].dropna(), bins=40, edgecolor="black", alpha=.85)
ax.axvline(0, color="red", linestyle="--")
ax.set(title=f"Home point differential (n={len(home):,})", xlabel="point differential")
ax.grid(alpha=.25, axis="y")
plt.show()

Check scores, margins, counts, and rates against sport semantics. Determine whether extremes are authentic, unit errors, or duplicates. For count targets, report zeros and overdispersion; for players, slice by role and playing-time eligibility before comparing distributions.

Structural breaks

Before pooling seasons, check rule and overtime changes, schedule length, shortened seasons, labor disruptions, expansion/relocation, tracking start dates, provider/schema migrations, definition revisions, and playoff mixing. A real break may require a known-at-T era flag, separate analyses, a sliding window, or exclusion. Do not normalize it away without explanation.

Leakage scouts

For a pre-event task, current-event outcomes and summaries are legal labels but illegal predictors: current score, margin, result, yards, EPA, win probability, box-score totals, final-season aggregates, and post-event participation.

pythonCopy code
suspects = {"points_for", "points_against", "won", "point_diff", "final_score"}
print("present suspects:", sorted(suspects.intersection(panel.columns)))

Presence is not failure; use as a pre-event feature is. Keep candidate features separate and send them through feature-rules and leakage-audit.

Automated panel report

panel_report.py reads user-owned CSV, Parquet, JSON, JSONL, or NDJSON and expects season, game_id, team, is_home, and binary won, with mapping flags. Install pandas; Parquet also needs pyarrow or fastparquet.

bashCopy code
python /path/to/eda-sports/scripts/panel_report.py \
  --input games.parquet --out data/eda.json
python /path/to/eda-sports/scripts/panel_report.py \
  --input games.csv --out data/eda.json --season-col year \
  --game-col event_id --team-col club --home-col home_flag --outcome-col win

Do not guess missing columns. Stop or create an explicit reviewed mapping. The report uses only the documented EDA decisions: GO (status 0), REPAIR (status 1), or STOP (status 2). A game without exactly two distinct team rows and exactly one home row, a null natural key, or a duplicate (game, team) key is structurally incompatible with this helper and returns STOP. Missing outcomes return REPAIR; they may be legitimate future events, but must be separated from completed-event analysis. GO still requires the separate time and leakage reviews described above.

Red flags

EDA note template

textCopy code
Question / target / decision time T:
Dataset, source, retrieval/version:
Grain and natural key:
Population and period:
Rows / games / entities:
Key and rows-per-game findings:
Coverage gaps:
Missingness findings:
Target, distribution, and slice findings:
Structural breaks:
Leakage suspects:
Repairs performed / remaining limitations:
Artifacts and plots:
Decision: GO | REPAIR | STOP
Reasons and next action:

Worked example

For a two-season pre-game team win project: declare (game_id, team); run both helpers; confirm two rows per completed game and expected period coverage; measure home win rate on home rows; separate future scheduled games from failed result joins; mark current score/result fields target-only; record era/provider breaks; and issue GO only for the supported population.

Integrity and resource routing

1. State grain before metrics; never model through unexplained duplicate keys.
2. Never hide incomplete periods, missing entities, or repairs.
3. Compare base rates on the correct independent unit.
4. Preserve raw data and separate outcomes from feature candidates.
5. Use references/eda_checklist.md for completeness, references/grain_guide.md for grain/key questions, and references/red_flags.md for diagnosis.
6. Run scripts/coverage_table.py for coverage and scripts/panel_report.py for a machine-readable team-game summary.