Diagnosing Perf Regressions

Diagnosing Performance Regressions

Overview

There is a difference between "this game is slow" and "this game got slow." This skill is for the second one.

Core principle: A regression has a cause. The cause is a specific change. Bisect to find the change, then root-cause from there.

This skill is not general performance tuning. If the game has always run at 30fps and you want to push it to 60, that is tune-performance (the domain skill for budget-driven optimization). This skill is the diagnostic — what broke, where, when.

When To Use

• "It used to run at 60fps, now it's at 30."
• "Load time was 3s yesterday, now it's 15s."
• "Frame stutters started after I added X."
• "Build went from buttery to choppy and I don't know why."

When NOT To Use

• "The game has always been slow, help me speed it up." → tune-performance.
• "I want to know if my game is fast enough to ship." → tune-performance.
• "The game crashes." → debug.

The Iron Law

Copy code
NO PERF FIX WITHOUT A KNOWN-GOOD vs KNOWN-BAD MEASUREMENT

You cannot fix a regression you have not measured. Eyeballing "feels laggy" is not a measurement. Get a number for the good state and a number for the bad state before touching code.

The Loop

Copy code
  Measure now → Find last good → Bisect changes → Match to a cliff → A/B fix → Re-measure

1. Measure now

Measure in a RunVerification probe, not by eye. It spawns a hidden, disposable game instance with a real renderer, so Performance.get_monitor(...) returns the game's actual numbers and the user's editor session is untouched.

Copy code
summer_batch ops:[{"op":"RunVerification","probe_source":"<probe>","max_seconds":20}]

gdscriptCopy code
extends SummerProbeBase
func _ready() -> void:
    await super._ready()
    await get_tree().process_frame
    var samples: Array = []
    for i in 30:                                  # ~5 s at 60 fps, sampled over time
        await get_tree().create_timer(0.16).timeout
        samples.append(Performance.get_monitor(Performance.TIME_FPS))
    report("fps_samples", samples)                # spike pattern lives here
    report("draw_calls", Performance.get_monitor(Performance.RENDER_TOTAL_DRAW_CALLS_IN_FRAME))
    report("bodies_3d", Performance.get_monitor(Performance.PHYSICS_3D_ACTIVE_OBJECTS))
    report("objects", Performance.get_monitor(Performance.RENDER_TOTAL_OBJECTS_IN_FRAME))
    dump_tree()
    finish()

What to capture:

summer_get_diagnostics does not report performance. It returns counts of console errors, debugger errors, debugger warnings and script errors. Use it to confirm the slow run isn't also throwing, never as a source of FPS or draw-call numbers.

Discard every sample from the first second — TIME_FPS is a per-second average and reads exactly 1.0 until it has a second of history. A leading run of 1.0 in the array is warm-up, not a stall.

Write the bad-state number down. "Scene Level3 runs at 31fps avg, drops to 12fps when 5+ enemies are on screen."

2. Find the last known-good state

Ask the user: when did it last run well? Then go to the VCS:

bashCopy code
git log --oneline --since="<last good date>" --until="now"

If they can name a commit ("it was fine on Monday's build"), check that out and measure it the same way. You now have:

• Known good: commit X, scene Level3, 60fps avg.
• Known bad: HEAD, scene Level3, 31fps avg.

If they can't name a date: ask for the rough window, list the commits, ask which ones are likely candidates. Do not bisect blind across 200 commits.

3. Bisect the changes

bashCopy code
git log --oneline <last-good>..HEAD

Read the list. Group commits into suspects:

If a single commit is the obvious suspect (e.g. "Added 200 enemies to spawner"), test it first. Otherwise actually git bisect — check out the midpoint, measure FPS, repeat. Don't theorize through it; the bisect is fast and definitive.

4. Match to one of the four Summer Engine performance cliffs

Regressions cluster into four categories. Matching the symptom to one shortens diagnosis.

If the bad-state symptom matches one row, you have your hypothesis. Verify by reverting just the change you suspect and re-measuring.

5. A/B with one variable

This is the most violated step. Discipline:

• Revert one suspect commit (or stub the suspect feature behind a flag).
• Rebuild / reload the scene.
• Measure FPS the same way you did in step 1.
• Compare to known-bad. Did it recover?

If yes → root cause confirmed. Now decide: fix the change, or accept the perf cost. If no → that wasn't it. Restore. Try the next suspect.

Never revert multiple changes at once during diagnosis. You will not know which one was the cause, and the bug will return on the next merge.

6. Re-measure after the fix

The fix is not done until the same measurement that found the regression returns the good number. Same scene, same play duration, same warm-up.

Re-run the identical probe from step 1 — same scene, same sample count, same drive sequence. A number taken a different way is not a comparison.

State the result with the number: "Reverted the per-frame get_tree().get_nodes_in_group() call in enemy_manager.gd. Level3 back to 58fps avg." Not "should be faster now."

When To Escalate

Red Flags — STOP

Rationalization Prevention

When The Engine Isn't Running

If the MCP tools report the engine isn't running:

1. Get it running. summer run starts it and restores the probe route. That is one command, and it is almost always the right answer — do not degrade to interviewing the user because a tool returned "not running" once.

2. If you have shell access, you can run a probe without the editor at all. The same verify instance is a plain flag on the Summer binary:

   bashCopy code
   "<summer-binary>" --path <project-dir> \
     --summer-verify <abs/path/probe.gd> \
     --summer-verify-out <abs/out-dir> \
     --summer-verify-max 20
   

   Then read <out-dir>/results.json (reports, frames, errors_seen, duration_ms). Locate the binary rather than guessing a path — on macOS it is the executable inside the installed Summer.app bundle. The probe file must extend the probe base; when you invoke the binary directly, write the base alongside it and extends it by path.

3. Only if neither is possible: ask the user for an in-game perf overlay reading (FPS, frame time, draw call count) and walk the bisect with them — "run on commit X, tell me the FPS; run on commit Y, tell me the FPS."

4. Do not claim a fix is verified from code reading alone.

Related skills

• tune-performance — for general optimization once the regression is fixed or when no regression exists.
• investigating-bugs — when the slow path turns out to be a logic bug (errors in a tight loop).
• debug — when the perf issue is actually a crash or freeze.
• verification-before-completion — to claim the fix worked, with numbers.

From a shell

No Summer MCP tools in this session? Run each one as a command; when summer --version is missing or below 4.0, use npx -y summer-engine@latest in place of summer. Fields are flags in kebab-case (--scene-path res://main.tscn) or one --input '<json>', and summer ops describe <op> prints the exact input. The summer-cli skill has the rest.