Uloop Simulate Mouse Input

Task

Simulate mouse input via Input System in Unity PlayMode, or dry-run a Game View coordinate against 3D physics without injecting input.

Workflow

1. When checking what a screenshot coordinate would hit in 3D physics before clicking, run uloop simulate-mouse-input --dry-run --x <x> --y <y> first (EditMode is fine; no Input System required)
2. Ensure Unity is in PlayMode (use uloop control-play-mode --action Play if not) before injecting real mouse input. Real input requires the Input System package and Active Input Handling set to 'Input System Package (New)' or 'Both'
3. For Click/LongPress: determine the target Game View input position from annotated SimX/SimY, raycast-grid InputX/InputY, or raw image pixels converted with ScreenshotToInputFormula
4. Execute the needed uloop simulate-mouse-input commands
5. Inspect the result with the lightest useful evidence: runtime state, logs, or a screenshot
6. When this input verifies a state transition, use Pause Point inspection from the section below as the standard frame proof
7. Report what happened and which evidence was used

Two rules while verifying:

• Do not touch the physical mouse or keyboard, and keep the OS pointer off the Unity window — real device input mixes into the same Mouse.current state this tool injects.
• Read the starting values you will assert against immediately before firing the input; a value measured earlier in the session may have changed.

Tool Reference

bashCopy code
uloop simulate-mouse-input --action <action> [options]
uloop simulate-mouse-input --dry-run --x <x> --y <y> [--layer-mask <mask>] [--max-distance <distance>]

Parameters

Actions

Pause Point Inspection (Standard for E2E)

For standard frame proof when this input drives a state transition, follow the uloop-pause-point skill — it covers line placement and interruption semantics. Tool-specific note: if InterruptedByPausePoint: true, Unity is paused; read PressDeliveredToGame before retrying. Clear inspection-only pause points (uloop clear-pause-point --all) before final validation.

When to use this vs simulate-mouse-ui

All rows below assume the New Input System is installed.

Dry-run (3D physics hit check)

Use --dry-run to check what a top-left Game View coordinate hits in 3D physics before clicking. This path does not require PlayMode or the Input System package.

• Requires an active Camera.main.
• Uses Unity Physics raycasts, not UI EventSystem raycasts.
• --x / --y use the same top-left Game View input coordinates as Click/LongPress.

bashCopy code
# Check what is under a screenshot coordinate
uloop simulate-mouse-input --dry-run --x 960 --y 540

# Check only specific layers
uloop simulate-mouse-input --dry-run --x 960 --y 540 --layer-mask 1

Examples

bashCopy code
# Left-click at a representative Game View point (--button Right for right-click)
uloop simulate-mouse-input --action Click --x 400 --y 300

# Hold left-click for 2 seconds
uloop simulate-mouse-input --action LongPress --x 400 --y 300 --duration 2.0

# Scroll up (negative --scroll-y scrolls down)
uloop simulate-mouse-input --action Scroll --scroll-y 120

# Smooth camera pan right over 0.5 seconds
uloop simulate-mouse-input --action SmoothDelta --delta-x 300 --delta-y 0 --duration 0.5

Coordinate System

--x/--y use top-left Game View coordinates: pass AnnotatedElements[].SimX/SimY directly, convert raw screenshot pixels with ScreenshotToInputFormula, and never flip Y in the caller (the tool converts internally). Conversion formula, Device Simulator notes, and how the injected position reads back inside Unity are in references/output-and-coordinates.md.

Prerequisites

• For real mouse injection: Unity must be in PlayMode, and the Input System package (com.unity.inputsystem) must be installed; game code must read input via Input System API (e.g. Mouse.current.leftButton.wasPressedThisFrame).
• For --dry-run: an active Camera.main is required; PlayMode and the Input System package are not required.

Output

The response reports Success, Message, Action, the target/injected coordinates, InterruptedByPausePoint/PausePointHits, and — for --dry-run — the resolved camera (CameraName/CameraPath; check these first when a No physics hit looks wrong) plus the physics hit details. Field-by-field semantics are in references/output-and-coordinates.md. Verify visual outcome with a follow-up screenshot.