Testdriver:Test Results Json

Overview

TestDriver writes a JSON result file for each test case after it finishes. These files have full metadata about the test run. This includes the SDK version, the runner version, infrastructure details, interaction statistics, and links to recordings.

TestDriver writes the result files to:

Copy code
.testdriver/results/<testFile>/<testName>.json

For example, a test file tests/login.test.mjs with a test named "should log in" produces:

Copy code
.testdriver/results/tests/login.test.mjs/should_log_in.json

Enabling

You do not need configuration. The TestDriver Vitest reporter plugin writes the JSON files automatically each time the tests run.

JSON Schema

Each result file is organized into logical groups:

versions

test

urls

replay

The replay object contains the recording replay URL and derived embed links. The gifUrl and embedUrl are generated automatically from the replay URL.

The replay.markdown field produces a clickable GIF badge you can paste directly into PR comments, README files, or issue descriptions:

markdownCopy code
[![Test Recording](https://api.testdriver.ai/replay/abc123/gif?shareKey=xyz)](https://console.testdriver.ai/replay/abc123?share=xyz)

date

team

infrastructure

realtime

interactions

Example Output

jsonCopy code
{
  "sdkVersion": "7.8.0",
  "vitestVersion": "4.0.0",
  "apiVersion": "1.45.0",
  "runnerVersionBefore": "2.1.0",
  "runnerVersionAfter": "2.1.1",
  "wasUpdated": true,
  "apiUrl": "https://api.testdriver.ai",
  "consoleUrl": "https://console.testdriver.ai",
  "testRunLink": "https://console.testdriver.ai/runs/abc123/def456",
  "dashcamUrl": "https://app.dashcam.io/replay/abc123",
  "vncUrl": "wss://sandbox-123.testdriver.ai/vnc",
  "date": "2025-01-15T14:30:00.000Z",
  "team": {
    "id": "team_abc123",
    "sessionId": "sess_xyz789"
  },
  "infrastructure": {
    "sandboxId": "sandbox-123",
    "instanceId": "i-abc123",
    "os": "linux",
    "amiId": "ami-0abc123",
    "e2bTemplateId": null,
    "imageVersion": "v2.1.0"
  },
  "realtime": {
    "channel": "sandbox:sandbox-123",
    "messageCount": 42
  },
  "interactions": {
    "total": 15,
    "cached": 3,
    "byType": {
      "find": 8,
      "click": 5,
      "assert": 2
    }
  }
}

Using Result Files in CI

Result files are useful for extracting test metadata in CI pipelines without parsing log output.

GitHub Actions Example

Use fromJSON to parse a result file into a GitHub Actions expression you can reference in subsequent steps:

yamlCopy code
- name: Run tests
  run: npx vitest run tests/login.test.mjs

- name: Parse result
  id: result
  run: |
    # Read the first JSON result file
    FILE=$(find .testdriver/results -name '*.json' | head -n 1)
    echo "json=$(cat "$FILE")" >> "$GITHUB_OUTPUT"

- name: Comment on PR
  if: fromJSON(steps.result.outputs.json).test.passed == false
  uses: actions/github-script@v7
  with:
    script: |
      const result = ${{ steps.result.outputs.json }};
      await github.rest.issues.createComment({
        owner: context.repo.owner,
        repo: context.repo.repo,
        issue_number: context.issue.number,
        body: [
          `❌ **${result.test.name}** failed`,
          ``,
          `Error: ${result.test.error}`,
          ``,
          result.replay.markdown,
          ``,
          `[View full recording](${result.urls.testRun})`
        ].join('\n')
      });

You can also load all results into a matrix or iterate over them:

yamlCopy code
- name: Run tests
  run: npx vitest run tests/*.test.mjs

- name: Collect results
  id: results
  run: |
    # Merge all result files into a JSON array
    echo "json=$(find .testdriver/results -name '*.json' -exec cat {} + | jq -s '.')" >> "$GITHUB_OUTPUT"

- name: Summary
  run: |
    echo '## Test Results' >> $GITHUB_STEP_SUMMARY
    RESULTS='${{ steps.results.outputs.json }}'
    echo "$RESULTS" | jq -r '.[] | "| \(.test.name) | \(if .test.passed then "✅" else "❌" end) | \(.urls.testRun) |"' >> $GITHUB_STEP_SUMMARY

Reading Results Programmatically

javascriptCopy code
import fs from "fs";
import path from "path";

const resultsDir = ".testdriver/results";

function readResults(dir) {
  const results = [];
  for (const testDir of fs.readdirSync(dir, { recursive: true })) {
    const fullPath = path.join(dir, testDir);
    if (fullPath.endsWith(".json") && fs.statSync(fullPath).isFile()) {
      results.push(JSON.parse(fs.readFileSync(fullPath, "utf-8")));
    }
  }
  return results;
}

const results = readResults(resultsDir);
const passed = results.filter(r => r.test.passed);
const failed = results.filter(r => !r.test.passed);

console.log(`${passed.length} passed, ${failed.length} failed`);
for (const r of failed) {
  console.log(`  FAIL: ${r.test.name} — ${r.test.error}`);
  console.log(`  Recording: ${r.urls.testRun}`);
  console.log(`  Embed: ${r.replay.markdown}`);
}