Arize Dataset

Arize Dataset Skill

SPACE — --space flags accept a space name (e.g., my-workspace) or a base64 space ID (e.g., U3BhY2U6...). Find yours with ax spaces list.

Concepts

• Dataset = a versioned collection of examples used for evaluation and experimentation
• Dataset Version = a snapshot of a dataset at a point in time; updates can be in-place or create a new version
• Example = a single record in a dataset with arbitrary user-defined fields (e.g., question, answer, context)
• Space = an organizational container; datasets belong to a space

System-managed fields on examples (id, created_at, updated_at) are auto-generated by the server -- never include them in create or append payloads.

Prerequisites

Proceed directly with the task — run the ax command you need. Do NOT check versions, env vars, or profiles upfront.

If an ax command fails, troubleshoot based on the error:

• command not found or version error → see references/ax-setup.md
• 401 Unauthorized / missing API key → run ax profiles show to inspect the current profile. If the profile is missing or the API key is wrong, follow references/ax-profiles.md to create/update it. If the user doesn't have their key, direct them to https://app.arize.com/admin > API Keys
• Space unknown → see Space
• Project unclear → ask the user, or run ax projects list -o json --limit 100 and present as selectable options
• Security: Never read .env files or search the filesystem for credentials. Exception: the non-secret ARIZE_SPACE_ID line (see Space). Use ax profiles for Arize credentials and ax ai-integrations for LLM provider keys. Never ask the user to paste secrets into chat. For missing credentials, see references/ax-profiles.md.

List Datasets: ax datasets list

Browse datasets in a space. Listing is for human browsing; when the dataset name or ID is known, call ax datasets get NAME_OR_ID --space SPACE -o json directly.

bashCopy code
ax datasets list
ax datasets list --space SPACE --limit 20
ax datasets list --cursor CURSOR_TOKEN
ax datasets list -o json

For all flags, run ax datasets list --help.

Get Dataset: ax datasets get

Quick metadata lookup -- returns dataset name, space, timestamps, and version list.

bashCopy code
ax datasets get NAME_OR_ID
ax datasets get NAME_OR_ID -o json
ax datasets get NAME_OR_ID --space SPACE   # required when using dataset name instead of ID

For all flags, run ax datasets get --help.

Response fields

Export Dataset: ax datasets export

Download examples to a file. By default, export returns one page of examples (--limit sets the page size, --cursor selects a later page). Pass --all to export the full dataset. Use --filter for server-side narrowing with a SQL-like expression over example columns, id, and annotation.<name>.*; --filter is not supported with --all.

bashCopy code
ax datasets export NAME_OR_ID
# -> dataset_abc123_20260305_141500/examples.json

ax datasets export NAME_OR_ID --all
ax datasets export NAME_OR_ID --filter "annotation.Correctness.label = 'Correct'"
ax datasets export NAME_OR_ID --version-id VERSION_ID
ax datasets export NAME_OR_ID --output-dir ./data
ax datasets export NAME_OR_ID --stdout
ax datasets export NAME_OR_ID --stdout | jq '.[0]'
ax datasets export NAME_OR_ID --space SPACE   # required when using dataset name instead of ID

For all flags, run ax datasets export --help.

Agent rule: Pass --all whenever the task needs every example; an export without --all stops after one page.

Export completeness verification: After exporting, confirm the row count matches what the server reports:

bashCopy code
# Get the server-reported count from dataset metadata
ax datasets get DATASET_NAME --space SPACE -o json | jq '.versions[-1] | {version: .id, examples: .example_count}'

# Compare to what was exported
jq 'length' dataset_*/examples.json

# If counts differ, re-export with --all

Output is a JSON array of example objects. Each example has system fields (id, created_at, updated_at) plus all user-defined fields:

jsonCopy code
[
  {
    "id": "ex_001",
    "created_at": "2026-01-15T10:00:00Z",
    "updated_at": "2026-01-15T10:00:00Z",
    "question": "What is 2+2?",
    "answer": "4",
    "topic": "math"
  }
]

Create Dataset: ax datasets create

Create a new dataset from a data file (--file) or inline JSON (--json).

bashCopy code
ax datasets create --name "My Dataset" --space SPACE --file data.csv
ax datasets create --name "My Dataset" --space SPACE --file data.json
ax datasets create --name "My Dataset" --space SPACE --file data.jsonl
ax datasets create --name "My Dataset" --space SPACE --file data.parquet
ax datasets create --name "My Dataset" --space SPACE --json '[{"question": "What is 2+2?", "answer": "4"}]'

For all flags, run ax datasets create --help.

Passing data via stdin

Use --file - to pipe data from another command:

bashCopy code
echo '[{"question": "What is 2+2?", "answer": "4"}]' | ax datasets create --name "my-dataset" --space SPACE --file -

# Or with a heredoc
ax datasets create --name "my-dataset" --space SPACE --file - << 'EOF'
[{"question": "What is 2+2?", "answer": "4"}]
EOF

Supported file formats

Format gotchas:

• CSV: Loses type information — dates become strings, null becomes empty string. Use JSON/Parquet to preserve types.
• JSONL: Each line is a separate JSON object. A JSON array ([{...}, {...}]) in a .jsonl file will fail — use .json extension instead.
• Parquet: Preserves column types. Requires pandas/pyarrow to read locally: pd.read_parquet("examples.parquet").

Append Examples: ax datasets append

Add examples to an existing dataset. Two input modes -- use whichever fits.

Inline JSON (agent-friendly)

Generate the payload directly -- no temp files needed:

bashCopy code
ax datasets append DATASET_NAME --space SPACE --json '[{"question": "What is 2+2?", "answer": "4"}]'

ax datasets append DATASET_NAME --space SPACE --json '[
  {"question": "What is gravity?", "answer": "A fundamental force..."},
  {"question": "What is light?", "answer": "Electromagnetic radiation..."}
]'

From a file

bashCopy code
ax datasets append DATASET_NAME --space SPACE --file new_examples.csv
ax datasets append DATASET_NAME --space SPACE --file additions.json

To a specific version

bashCopy code
ax datasets append DATASET_NAME --space SPACE --json '[{"q": "..."}]' --version-id VERSION_ID

For all flags, run ax datasets append --help.

Exactly one of --json or --file is required.

Validation

• Each example must be a JSON object with at least one user-defined field
• Maximum 100,000 examples per request

Schema validation before append: If the dataset already has examples, inspect its schema before appending to avoid silent field mismatches:

bashCopy code
# Check existing field names in the dataset
ax datasets export DATASET_NAME --space SPACE --stdout | jq '.[0] | keys'

# Verify your new data has matching field names
echo '[{"question": "..."}]' | jq '.[0] | keys'

# Both outputs should show the same user-defined fields

Fields are free-form: extra fields in new examples are added, and missing fields become null. However, typos in field names (e.g., queston vs question) create new columns silently -- verify spelling before appending.

Update Examples: ax datasets update-examples

ax datasets update-examples patches rows by example id; ax datasets delete-examples removes rows from one version and requires --version-id plus --example-ids as a JSON array of example IDs. Uncommon — run ax datasets update-examples --help or ax datasets delete-examples --help for flags and input formats.

Delete Dataset: ax datasets delete

bashCopy code
ax datasets delete NAME_OR_ID
ax datasets delete NAME_OR_ID --space SPACE   # required when using dataset name instead of ID
ax datasets delete NAME_OR_ID --force   # skip confirmation prompt

For all flags, run ax datasets delete --help.

Update Dataset: ax datasets update

Rename an existing dataset.

bashCopy code
ax datasets update NAME_OR_ID --name "new-dataset-name"
ax datasets update NAME_OR_ID --name "new-dataset-name" --space SPACE

For all flags, run ax datasets update --help.

Annotate Examples: ax datasets annotate-examples

Write annotations onto dataset examples in bulk from a file. Upsert semantics — existing annotations with the same key are updated, new ones are created. Up to 1000 examples per request. Each record needs a record_id (the dataset example ID) and values; unmatched record IDs are silently ignored, so confirm IDs against an export first.

bashCopy code
ax datasets annotate-examples NAME_OR_ID --file annotations.json
ax datasets annotate-examples NAME_OR_ID --file annotations.csv --space SPACE

For all flags, run ax datasets annotate-examples --help.

Workflows

Find a dataset by name

All dataset commands accept a name or ID directly (add --space SPACE when passing a name). To resolve a name to its ID, read it from get:

bashCopy code
ax datasets get "eval-set-v1" --space SPACE -o json | jq -r '.id'

Create a dataset from file for evaluation

1. Prepare a CSV/JSON/Parquet file with your evaluation columns (e.g., input, expected_output)
   • If generating data inline, pass it with --json or pipe it via --file - (see the Create Dataset section)
2. ax datasets create --name "eval-set-v1" --space SPACE --file eval_data.csv
3. Verify: ax datasets get DATASET_NAME --space SPACE
4. Use the dataset name to run experiments

Add examples to an existing dataset

bashCopy code
# Append inline or from a file using the dataset name (see Append Examples section for full syntax)
ax datasets append DATASET_NAME --space SPACE --json '[{"question": "...", "answer": "..."}]'
ax datasets append DATASET_NAME --space SPACE --file additional_examples.csv

Download dataset for offline analysis

1. ax datasets export DATASET_NAME --space SPACE --all -- download every example to file
2. Parse the JSON: jq '.[] | .question' dataset_*/examples.json

Export a specific version

bashCopy code
# List versions
ax datasets get DATASET_NAME --space SPACE -o json | jq '.versions'

# Export that version
ax datasets export DATASET_NAME --space SPACE --version-id VERSION_ID

Iterate on a dataset

1. Export current version: ax datasets export DATASET_NAME --space SPACE --all
2. Modify the examples locally
3. Append new rows: ax datasets append DATASET_NAME --space SPACE --file new_rows.csv
4. Or create a fresh version: ax datasets create --name "eval-set-v2" --space SPACE --file updated_data.json

Pipe export to other tools

bashCopy code
# Count examples
ax datasets export DATASET_NAME --space SPACE --all --stdout | jq 'length'

# Extract a single field
ax datasets export DATASET_NAME --space SPACE --stdout | jq '.[].question'

# Convert to CSV with jq
ax datasets export DATASET_NAME --space SPACE --stdout | jq -r '.[] | [.question, .answer] | @csv'

Dataset Example Schema

Examples are free-form JSON objects. There is no fixed schema -- columns are whatever fields you provide. System-managed fields are added by the server:

Related Skills

• arize-trace: Export production spans to understand what data to put in datasets → use arize-trace
• arize-experiment: Run evaluations against this dataset → next step is arize-experiment
• arize-prompt-optimization: Use dataset + experiment results to improve prompts → use arize-prompt-optimization

Troubleshooting

Save Credentials for Future Use

See references/ax-profiles.md § Save Credentials for Future Use.