Testdriver:Customizing Devices

TestDriver Options

Configure the TestDriver behavior with options that you give to the TestDriver() function:

javascriptCopy code
const testdriver = TestDriver(context, {
  // === Sandbox & Connection ===
  newSandbox: true,          // Force creation of a new sandbox (default: true)
  reconnect: false,          // Reconnect to last sandbox (default: false)
  keepAlive: 60000,          // Keep sandbox alive after disconnect in ms (default: 60000)
  os: "linux",               // 'linux' | 'windows' (default: 'linux')
  resolution: "1366x768",    // Sandbox resolution (e.g., '1920x1080')
  ip: "203.0.113.42",        // Direct IP for self-hosted sandbox
  sandboxAmi: "ami-1234",    // Custom AMI ID (AWS deployments)
  sandboxInstance: "i3.metal", // EC2 instance type (AWS deployments)

  // === Preview & Debugging ===
  preview: "browser",        // "browser" | "ide" | "none" (default: "browser")
  headless: false,           // @deprecated - use preview: "none" instead
  debugOnFailure: false,     // Keep sandbox alive on test failure for debugging

  // === Caching ===
  cache: true,               // Enable element caching (default: true)
  // Or use advanced caching config:
  // cache: {
  //   enabled: true,
  //   thresholds: {
  //     find: { screen: 0.05, element: 0.8 },
  //     assert: 0.05
  //   }
  // },
  cacheKey: "my-test",       // Cache key for element finding operations

  // === Recording & Screenshots ===
  dashcam: true,             // Enable/disable Dashcam video recording (default: true)
  autoScreenshots: true,     // Capture screenshots before/after each command (default: false)

  // === AI Configuration ===
  ai: {                      // Global AI sampling configuration
    temperature: 0,          // 0 = deterministic, higher = more creative
    top: {
      p: 0.9,               // Top-P nucleus sampling (0-1)
      k: 40,                // Top-K sampling (1 = most likely, 0 = disabled)
    },
  },

  // === Screen Change Detection ===
  redraw: true,              // Enable redraw detection (default: true)
  // Or use advanced redraw config:
  // redraw: {
  //   enabled: true,
  //   thresholds: {
  //     screen: 0.05,       // Pixel diff threshold (0-1), false to disable
  //     network: false,     // Monitor network activity (default: false)
  //   }
  // },

  // === Logging & Analytics ===
  logging: true,             // Enable console logging output (default: true)
  analytics: true,           // Enable analytics tracking (default: true)

  // === Advanced ===
  apiRoot: "https://...",    // API endpoint URL (for self-hosted deployments)
  environment: {},           // Additional environment variables for the sandbox
});

Preview Mode

Control how test execution is visualized. The preview option determines where the live debugger view opens:

javascriptCopy code
const testdriver = TestDriver(context, {
  preview: "browser",  // Opens in default browser (default)
});

IDE Preview

For the best development experience, use preview: "ide" with the TestDriver extension for VSCode or Cursor:

javascriptCopy code
const testdriver = TestDriver(context, {
  preview: "ide",  // Opens preview in IDE panel
});

Headless Mode

Run tests without any visual preview. Useful for CI/CD pipelines:

javascriptCopy code
const testdriver = TestDriver(context, {
  preview: "none",  // No visual preview (headless)
});

Debug on Failure

Keep the sandbox alive when a test fails so you can reconnect and debug interactively. The sandbox ID is printed to the console along with instructions for reconnecting via MCP.

javascriptCopy code
const testdriver = TestDriver(context, {
  debugOnFailure: true,
});

IP Target

If self-hosting TestDriver, use ip to specify the device IP. See Self-Hosting TestDriver for details.

javascriptCopy code
const testdriver = TestDriver(context, {
  ip: "203.0.113.42",  // Your allowlisted IP
});

Operating System

Set the os property to run tests on a specific operating system. Available options are linux (default) and windows.

javascriptCopy code
const testdriver = TestDriver(context, {
  os: "windows",  // Run on Windows sandbox
});

Using Environment Variables

You can make the operating system configurable via environment variables. This requires adding code to read from process.env in your test:

javascriptCopy code
const testdriver = TestDriver(context, {
  os: process.env.TD_OS || "linux",  // Read from env, default to Linux
});

Then pass the variable when running tests:

bashCopy code
# Run tests on Windows
TD_OS=windows vitest run

# Run tests on Linux (default)
TD_OS=linux vitest run

This pattern is useful for running the same test suite across multiple operating systems in CI/CD:

yamlCopy code
# Example GitHub Actions matrix
strategy:
  matrix:
    os: [linux, windows]
steps:
  - run: TD_OS=${{ matrix.os }} vitest run

Dashcam Recording

Dashcam video recording is enabled by default. Disable it to skip recording:

javascriptCopy code
const testdriver = TestDriver(context, {
  dashcam: false,
});

Automatic Screenshots

Screenshots are automatically captured before and after every command (click, type, find, assert, etc.) by default. Each screenshot filename includes the line number from your test file.

Disable automatic screenshots:

javascriptCopy code
const testdriver = TestDriver(context, {
  autoScreenshots: false,
});

Caching

Element caching speeds up repeated find() and assert() calls. Enabled by default.

javascriptCopy code
// Disable caching
const testdriver = TestDriver(context, {
  cache: false,
});

// Advanced: custom thresholds
const testdriver = TestDriver(context, {
  cache: {
    enabled: true,
    thresholds: {
      find: { screen: 0.05, element: 0.8 },
      assert: 0.05,
    },
  },
  cacheKey: "my-test",
});

Redraw Detection

Redraw detection waits for the screen to stabilize before taking actions. Enabled by default.

javascriptCopy code
// Disable redraw detection
const testdriver = TestDriver(context, {
  redraw: false,
});

// Advanced: custom thresholds with network monitoring
const testdriver = TestDriver(context, {
  redraw: {
    enabled: true,
    thresholds: {
      screen: 0.05,
      network: true,
    },
  },
});

AI Configuration

Control how the AI model generates responses for find() verification and assert() calls:

javascriptCopy code
const testdriver = TestDriver(context, {
  ai: {
    temperature: 0,       // 0 = deterministic
    top: { p: 0.9, k: 40 },
  },
});

Environment Variables

Pass additional environment variables to the sandbox:

javascriptCopy code
const testdriver = TestDriver(context, {
  environment: {
    MY_VAR: "value",
    DEBUG: "true",
  },
});

Keepalive

By default, sandboxes stay alive for 60 seconds after disconnect. Customize this with keepAlive:

javascriptCopy code
const testdriver = TestDriver(context, {
  keepAlive: 300000,  // Keep sandbox alive for 5 minutes after test
});

Set to 0 to terminate immediately:

javascriptCopy code
const testdriver = TestDriver(context, {
  keepAlive: 0,  // Terminate sandbox immediately on disconnect
});

Connecting to an Existing Sandbox

Speed up test development by connecting to an existing sandbox instead of starting fresh each time. This lets you iterate quickly on failing steps without re-running the entire test from the beginning.

Split your test into two files: one for known-good steps that set up the desired state, and another for work-in-progress steps you want to debug.

javascriptCopy code
const testdriver = TestDriver(context, {
  keepAlive: 60000,  // Keep sandbox alive for 60 seconds after test
});

javascriptCopy code
// Second test file: experiment.test.mjs (run within keepAlive window)
const testdriver = TestDriver(context, {
  keepAlive: 60000,
});

await testdriver.connect({ sandboxId: "sandbox-abc123" });

Then, you can run both tests in sequence:

bashCopy code
vitest run -t known-good.test.mjs -t work-in-progress.test.mjs

And as you make changes to work-in-progress.test.mjs, you can re-run just that file to quickly iterate on the failing steps.

bashCopy code
vitest run work-in-progress.test.mjs