Pie Testing

🧠 Brains complement: IF an unreal-engine-skills-manager tool (external MCP) exists in this session, call it with {action: "load", skill: "automation-and-testing"} for UE domain knowledge on this topic — correct APIs, architecture, best practices — and treat it as the rubric for any review / "best practices" question. If no such tool is available (e.g. running under Claude Code or Codex without that MCP), skip this line entirely and proceed with this skill alone — do NOT attempt the call.

Play-In-Editor (PIE) Testing Skill

PIE is the only way to validate runtime behavior — Blueprint logic, AI ticking, animation, input, widget interaction, gameplay events. Without starting PIE, your "fix" is unverified.

PIE control — engine EditorAppToolset

Generic PIE start/stop/status is owned by Unreal 5.8's native EditorToolset.EditorAppToolset, invoked through call_tool (run describe_toolset on it for exact action names/params):

Copy code
call_tool(toolset="EditorToolset.EditorAppToolset", tool="IsPIERunning")
call_tool(toolset="EditorToolset.EditorAppToolset", tool="StartPIE")
call_tool(toolset="EditorToolset.EditorAppToolset", tool="StopPIE")

PIE start/stop/query is not on WidgetService anymore for general testing — use the engine toolset above. WidgetService still owns the widget-in-PIE validation helpers below.

Standard Test Loop

For repeatable multi-step verification, prefer WorkflowService.run_scenario() over hand-composing the loop. It queues a state machine on editor ticks, waits for actual PIE readiness, scopes log assertions to the scenario start, uses VibeUE's focus-free input injection, captures evidence, and always tears PIE down on pass, failure, or cancellation. Poll get_scenario(id) until its status is passed, failed, cancelled, smoke_passed, or stale:

pythonCopy code
import json, unreal
spec = {
  "name": "secondary fire consumes ammo",
  "preflight": {"save_dirty_assets": True, "compile_blueprints": ["/Game/Weapons/BP_Rifle"]},
  "steps": [
    {"action":"start_pie"},
    {"action":"wait_for_pie", "timeout_seconds":30},
    {"action":"inject_action", "path":"/Game/Input/IA_Fire_Secondary"},
    {"action":"wait", "seconds":0.25},
    {"action":"assert_log", "contains":"SecondaryFire"},
    {"action":"capture_game", "name":"after-secondary-fire"}
  ],
  "teardown": {"stop_pie": True}
}
queued = json.loads(unreal.WorkflowService.run_scenario(json.dumps(spec)))
# Poll in separate tool calls; do not block the editor game thread with time.sleep.

Use the lower-level primitives below for interactive investigation, one-off probes, or actions not in the scenario schema. Never spin/sleep inside one editor Python call while waiting for a scenario.

Assertions and evidence freshness

A normal scenario must declare at least one assert_log, python_assert, or python_assert_number step. Completing input/wait/capture steps alone does not verify gameplay. For a deliberate boot-only check, set "smoke": true: an assertion-free run finishes with status="smoke_passed" and passed=false. Pollers must treat smoke_passed and stale as terminal alongside passed, failed, and cancelled.

python_assert compares the Python result with a required string expected. python_assert_number evaluates an expression and requires a finite numeric result:

jsonCopy code
{"action":"python_assert_number", "expression":"0.1 + 0.2",
 "operator":"eq", "expected":0.3, "tolerance":0.00001}

Operators are eq, lt, le, gt, and ge; nonnegative tolerance is supported only for eq and defaults to zero. For gameplay, use an expression reading a live actor/property instead of the arithmetic example. Expected/actual values are recorded per assertion; an expression error, nonnumeric result, or unmet comparison fails the scenario. Do not cache PIE objects in globals. If an assert_log supplies both contains and not_contains, both conditions must hold. Log assertions use the active log (including -abslog overrides); unavailable or truncated logs fail the assertion, including negative checks. Reports include assertionsDeclared and assertionsEvaluated, including failed assertions.

To detect stale evidence, add a top-level dependencies array of actual file paths relative to the project directory (or absolute paths), for example Content/Ships/BP_Ship.uasset and relevant source/config files. Dependencies must exist. VibeUE hashes their contents after preflight, records the scenario hash and engine version, and automatically tracks its plugin DLL when dependencies are provided. It does not persist the submitted Python source as provenance.

get_scenario() checks those hashes again for completed reports, including reports loaded from disk. A changed/missing file or changed engine version produces validity="stale"; a historical pass then returns status="stale", passed=false, and historicalPassed=true. verifiedCurrent=true requires passing assertions and unchanged tracked inputs. Without dependencies, validity="untracked" and verifiedCurrent=false, even if assertions passed. Freshness covers only explicitly listed files plus the plugin DLL: it does not infer transitive asset dependencies, detect unsaved edits, or prove a newly changed project binary is loaded. Save and compile first, list all relevant inputs, and rerun after changes. Hashes are freshness checks, not tamper-proof attestations. Large dependency sets increase polling cost.

These verification improvements were inspired by PageMastr/Gatekeeper, particularly its assertion coverage and source-bound verdicts. The VibeUE implementation is independently written for Unreal.

Copy code
# 1. Make sure you're starting from a clean state
call_tool(toolset="EditorToolset.EditorAppToolset", tool="StopPIE")   # no-op if not running

# 2. Start the session (uses the editor's current PIE settings — default map, viewport)
call_tool(toolset="EditorToolset.EditorAppToolset", tool="StartPIE")

# 3. Let the test run / inspect log output (LogsToolset) / interact via other services
# 4. Stop when done
call_tool(toolset="EditorToolset.EditorAppToolset", tool="StopPIE")

Net mode from Python — EngineSettingsService

StartPIE uses the editor's saved PIE settings, so set the net mode BEFORE starting the session. EngineSettingsService reads and writes ULevelEditorPlaySettings directly and persists the change, so a dedicated-server PIE gate no longer means quitting the editor to hand-edit EditorPerProjectUserSettings.ini and relaunching:

pythonCopy code
import unreal

# Read the current PIE multiplayer settings
info = unreal.EngineSettingsService.get_pie_settings()
print(info.net_mode, info.num_clients, info.run_under_one_process)

# Set dedicated-server PIE: 1 client, all windows in one process
ok = unreal.EngineSettingsService.set_pie_settings("Client", 1, True)   # returns True only on verified write
# Verify by readback, never by return value alone
assert unreal.EngineSettingsService.get_pie_settings().net_mode == "Client"

• net_mode maps EPlayNetMode: "Standalone" | "ListenServer" | "Client". "Client" is dedicated-server PIE — the editor's "Play As Client", where a windowless dedicated server is spawned behind the scenes and PIE instance 0 is that server ("DedicatedServer" is accepted as an alias for "Client").
• set_pie_settings refuses an unknown net mode or a num_clients outside 1-10 (returns False, logs a Warning), and returns False if any written value fails to read back.
• These are per-USER config, not project config — they live in Saved/Config/<Platform>/EditorPerProjectUserSettings.ini and follow the machine, not the repo. Do not expect a teammate or CI to inherit them; set them from the gate itself.

Validating widgets in PIE — WidgetService

VibeUE keeps a small set of widget-in-PIE helpers on unreal.WidgetService (run them via execute_python_code). These spawn and inspect live widget instances once PIE is running:

pythonCopy code
import unreal

# After StartPIE, spawn a widget instance into the running viewport
handle = unreal.WidgetService.spawn_widget_in_pie("/Game/UI/WBP_HUD", 0)

# Read a live property off the running instance
val = unreal.WidgetService.get_live_property(handle, "HealthText", "Text")

# Tear it down before stopping PIE
unreal.WidgetService.remove_widget_from_pie(handle)

unreal.WidgetService.is_pie_running() also still exists and is handy from inside Python; for tool-level control prefer the engine EditorAppToolset actions above.

Spawning test actors in PIE — PIEActorService

Why this exists: you cannot spawn an actor into a running PIE world from Python any other way. unreal.World / unreal.GameplayStatics expose no spawn (the deferred-spawn entry points are BlueprintInternalUseOnly), EditorActorSubsystem.spawn_actor_from_class targets the editor world, and the engine SceneTools toolset refuses with "Cannot create actors while PIE is active". PIEActorService resolves the live PIE world by role and spawns a transient actor into it — a blocker, a test dummy, a trigger volume — with no Blueprint and no editor-world contamination. Run it via execute_python_code.

Selector grammar (strict — anything else returns INVALID_SELECTOR):

Worlds come from the PIE world contexts only; the editor world and the ~100 stale /Memory/UEDPIE_* shells are never used. No PIE at all → PIE_NOT_RUNNING.

Authority rule: a spawn into a client world is cosmetic and never replicates, so it is refused with CLIENT_REQUIRES_OPT_IN unless you pass allow_client_local=True. Spawn on "server" for anything that must exist for every player.

pythonCopy code
import unreal, json

# Drop a static-mesh actor as a blocker in front of the host, then verify and clean up.
t = unreal.Transform(location=unreal.Vector(500, 0, 100))
res = json.loads(unreal.PIEActorService.spawn_actor(
    "server", "/Script/Engine.StaticMeshActor", t))
assert res["success"], res
handle = res["handle"]          # "<session_serial>:<guid>" — dies with this PIE session
print(res["actor_path"], res["net_mode"], res["is_authority"])

# ... run whatever movement / collision check needs the blocker ...

# Idempotent teardown before StopPIE.
print(unreal.PIEActorService.destroy_all())     # {"success": true, "destroyed": 1, "already_gone": 0}

resolve_world("server") returns {success, world_path, net_mode, pie_instance, is_authority, actor_count} and is the cheap way to confirm a session's role before spawning. destroy_actor(handle) is idempotent per handle: it destroys the actor and keeps the record, so calling it again returns success: true, already_gone: true (never UNKNOWN_HANDLE). A consumed handle stays in list_spawned() as alive: false until PIE ends — list_spawned() is the full spawn ledger for the session, not just the live actors — and destroy_all() counts an already-consumed actor under already_gone rather than destroyed.

Error codes: INVALID_SELECTOR, PIE_NOT_RUNNING, WORLD_NOT_FOUND, AMBIGUOUS_WORLD, CLASS_NOT_FOUND, NOT_AN_ACTOR_CLASS, CLASS_ABSTRACT, CLASS_DEPRECATED, CLASS_REINSTANCED, INVALID_TRANSFORM, INVALID_COLLISION_HANDLING, CLIENT_REQUIRES_OPT_IN, SPAWN_REJECTED (the placement collided under the chosen collision policy), UNKNOWN_HANDLE, STALE_HANDLE.

Gotcha — handles die with the PIE session. EndPIE bumps an internal session serial and clears the registry, so a handle kept across a stop/start reports STALE_HANDLE. And the same Python-globals rule as everywhere else applies with teeth here: release any global holding a spawned actor before StopPIE (del blocker / set to None, or vibeue.release_globals), or the editor asserts on PIE teardown ("Object from PIE level still referenced"). The returned JSON is a string, so keeping res/handle is safe — only a live actor reference is the hazard.

Driving gameplay input in PIE — InputService (issue #550)

Never remap game input assets or send OS keystrokes (SendKeys/AppActivate) to test a mechanic — inject input directly; no OS window focus needed:

pythonCopy code
import unreal

# Fire an Enhanced Input action once: QUEUED for the next input tick, released on the tick after.
# X/Y/Z map onto the action's value type; Boolean uses X != 0.
print(unreal.InputService.inject_action("/Game/Input/IA_Fire_Secondary"))

# Hold it for 1.5 s of real time, then release (returns at once; stop_injection releases early).
print(unreal.InputService.inject_action_for("/Game/Input/IA_Fire_Secondary", 1.5))

# Or send a raw key to the game viewport through Slate ("tap" = down+up; also "down"/"up",
# and "hold" with a duration: inject_key("RightMouseButton", "hold", 1.5)).
print(unreal.InputService.inject_key("SpaceBar"))

A Python call blocks the game thread, so PIE does not tick while it runs: inject in one call and read the game state in a LATER call, two or more frames on. While a hold is active the editor stays off its background frame rate, so an unfocused editor does not starve the hold. The action calls take pie_instance to pick a PIE world: -1, the default, is the first PIE world with a local player (the lowest instance number — the listen server's window, or client 1 under a dedicated server), and replies report the instance actually used. A hold is one hold per action and PIE world whatever the path spelling; holding a held key again extends it; StopPIE drops every hold, so nothing from an ended session is released into the editor or reported active in the next one.

All return JSON with success and an error_code naming the problem (PIE not running, NO_LOCAL_PLAYER / NO_PLAYER_CONTROLLER while PIE or a client's join is still starting — retry on a later call —, unknown key, ...). inject_key additionally rejects a Simulate In Editor session with SIMULATE_NOT_PLAY: Simulate has no player game viewport, so the Slate key events would be dropped. Use StartPIE (Play), not Simulate, when driving keys. Its handled_down / handled_up fields report whether a Slate widget consumed the event, not whether the key reached the player — a key the game polls with WasInputKeyJustPressed lands with handled_down: false. Verify from game state, not from those flags.

Seeing the game — capture_image (issues #544/#546)

The capture_image MCP tool with source="game" screenshots the PIE viewport including the Slate/UMG HUD and returns a real image block you can look at (plus a PNG under Saved/VibeUE/Captures). This replaces the old Shot showui + read-the-file workaround — and CaptureViewport can never see PIE or UI at all.

Full-rate unfocused verification — PerformanceService (issue #549)

An unfocused editor throttles to ~3 FPS, wrecking timed PIE runs. Disable throttling for the session instead of focus-hacking the window:

pythonCopy code
unreal.PerformanceService.set_background_throttling(False)   # before the run
unreal.PerformanceService.set_background_throttling(True)    # after

Gotchas

• Python blocks the game thread, so you cannot sample a value over time in one call. A loop with time.sleep() between reads returns the same number every iteration — the world never ticks while your script holds the thread. This makes a working animation look frozen and a stuck one look identical to a healthy one. Take each sample in a separate execute_python_code call and compare across them:

  pythonCopy code
  # WRONG - three identical readings, proves nothing
  for i in range(3):
      print(ai.get_editor_property("SailAngle")); time.sleep(0.4)
  
  # CORRECT - one reading per call; the elapsed wall-clock between calls is real tick time
  print(ai.get_editor_property("SailAngle"))
  

  Convert the difference into the engineering unit you expect and check it: 306° → 709° across ~8 s is 48°/s, which is exactly the 8 RPM that was configured — that is a pass, "the number changed" is not.

• PIE start is asynchronous. StartPIE returns immediately after RequestPlaySession is queued. The world isn't actually playing until the editor processes the request on its next tick. If you need to act inside the running world, give it a tick or poll IsPIERunning.

• Already-running is treated as success. StartPIE succeeds if a PIE session already exists — it does NOT restart. Stop first if you need a fresh session.

• StopPIE tears down the world via RequestEndPlayMap. Spawned PIE widget instances should be removed with WidgetService.remove_widget_from_pie(handle) before stopping.

• Save before starting. Dirty asset changes are NOT picked up by PIE unless saved/compiled. Always compile_blueprint(...) before launching PIE to test Blueprint changes.

• Don't leave PIE running between tasks. Subsequent edits (recompiles, asset moves, hot reload) can fail or behave oddly while a PIE world is alive. Call StopPIE before returning control to the user.

• Map-load delegates never fire on PIE start. FCoreUObjectDelegates::PreLoadMap / PostLoadMapWithWorld (loading screens, post-load hooks, subsystem map handlers) are skipped because the PIE world is duplicated from the editor world, not loaded. To exercise them, trigger a real in-game transition once PIE is up: unreal.GameplayStatics.open_level(pie_world, "MapName").

• Python globals holding PIE objects CRASH the editor on PIE end. Module-level variables from execute_python_code persist between calls and are GC roots (FPyReferenceCollector). If any still reference a PIE object (pawn, subsystem instance, quest/objective, widget) when PIE tears down, the engine asserts in PlayLevel.cpp ("Object from PIE level still referenced") and the editor dies. Before stopping PIE — or as the last statement of any call that inspected PIE objects — release them: del pawn, sub, quest or set them to None. Also holds for a world pointer cached across an in-game open <map> travel: it goes stale and using it raises "WorldContext requested with invalid context object"; re-fetch get_game_world() in the same call that uses it.

When to use PIE

• Verifying a Blueprint event fires (combine with log inspection — see LogsToolset)
• Validating gameplay logic (damage, scoring, state transitions)
• Testing widgets in their real runtime context (combine with umg-widgets skill's spawn_widget_in_pie)
• Reproducing user-reported runtime bugs

When NOT to use PIE

• Pure asset/editor validation (use compile_blueprint, find_assets, etc.)
• Static introspection (use get_nodes_in_graph, get_node_pins)
• Anything you can verify without a live world — PIE is slow, save it for genuine runtime checks.

Additional gotchas

• StopPIE then StartPIE in one Python call fails with "A play session is already running" — the editor does not tick between statements. Split them across calls and confirm with IsPIERunning.
• EditorAssetLibrary.load_asset returns None while PIE runs, and downstream calls then return 0/None instead of raising (a fake negative). Load assets before StartPIE, or find live objects with ObjectIterator.
• Match PIE worlds by their /Game/<path>/UEDPIE_0_<Map> (server/host) and UEDPIE_1_ (client) object paths, not by name — roughly 100 stale /Memory/UEDPIE_* shells match by name. vibeue.pie_worlds() returns them keyed by role, and UnrealEditorSubsystem.get_game_world() gives the local one.
• A two-client run is Standalone by default: both worlds read ROLE_AUTHORITY, so confirm get_editor_property("role") (or vibeue.role(actor)) before claiming a host/client result. Read and set the PIE net mode with get_pie_settings/set_pie_settings (dedicated server = PlayNetMode 2).
• A Server RPC invoked from Python runs LOCALLY and is silently dropped; never Start/StopFire a client world's weapon from Python (it crashes the editor). Move a client pawn through its server copy; control rotation is client-owned.
• slomo 0.1 stretches a 2 s timer to 20 s of wall time so a transient state survives between calls; restore slomo 1 after. PIE time restarts each run.
• Never leave an automation test run queued before starting PIE — it wakes on PIE activity and tears the session down.
• Globals in the execute_python_code script namespace persist across calls (it is a separate dict from sys.modules["__main__"], so import __main__ does not see them): use them to collect async tool results (vibeue.exec_tool_async/collect_tool_result), and release PIE object references (vibeue.release_globals) before StopPIE.