Arize Ai Provider Integration

Arize AI Integration Skill

SPACE — --space flags accept a space name (e.g., my-workspace) or a base64 space ID (e.g., U3BhY2U6...). Resolve a known name with ax spaces get NAME; ax spaces list is for browsing when the name is unknown. Note: ai-integrations create does not accept --space — AI integrations are account-scoped. Use --space only with list, get, update, and delete.

Concepts

• AI Integration = stored LLM provider credentials registered in Arize; used by evaluators to call a judge model and by other Arize features that need to invoke an LLM on your behalf
• Provider = the LLM service backing the integration (e.g., OPEN_AI, ANTHROPIC, AWS_BEDROCK)
• Integration ID = a base64-encoded global identifier for an integration (e.g., TGxtSW50ZWdyYXRpb246MTI6YUJjRA==); required for evaluator creation and other downstream operations
• Scoping = visibility rules controlling which spaces or users can use an integration
• Auth type = how Arize authenticates with the provider: DEFAULT (provider API key), PROXY_WITH_HEADERS (proxy via custom headers), BEARER_TOKEN (bearer token auth), or OAUTH2_CLIENT_CREDENTIALS

Prerequisites

Proceed directly with the task — run the ax command you need. Do NOT check versions, env vars, or profiles upfront.

If an ax command fails, troubleshoot based on the error:

• command not found or version error → see references/ax-setup.md
• 401 Unauthorized / missing API key → run ax profiles show to inspect the current profile. If the profile is missing or the API key is wrong, follow references/ax-profiles.md to create/update it. If the user doesn't have their key, direct them to https://app.arize.com/admin > API Keys
• Space unknown → see Space
• LLM provider call fails (missing provider credentials) → run ax ai-integrations list --space SPACE to check for platform-managed credentials. If none exist:
  • Preferred: Give the user the exact ax ai-integrations create command from Supported Providers below (reference env var names like $OPENAI_API_KEY, never raw values). Ask them to export the provider key in their own terminal and run the command there — never paste the key into chat. Do not run the create command yourself unless the var is already exported in this terminal session.
  • Bedrock/Vertex: use --provider-metadata instead of --api-key (see Supported Providers below).
• Security: Never read .env files or search the filesystem for credentials. Exception: the non-secret ARIZE_SPACE_ID line (see Space). Use ax profiles for Arize credentials and ax ai-integrations for LLM provider keys. Never ask the user to paste secrets into chat. For missing credentials, see references/ax-profiles.md.

List AI Integrations

List all integrations accessible in a space:

bashCopy code
ax ai-integrations list --space SPACE

Filter by name (case-insensitive substring match):

bashCopy code
ax ai-integrations list --space SPACE --name "openai"

Paginate large result sets:

bashCopy code
# Get first page
ax ai-integrations list --space SPACE --limit 20 -o json

# Get next page using cursor from previous response
ax ai-integrations list --space SPACE --limit 20 --cursor CURSOR_TOKEN -o json

Run ax ai-integrations list --help for flags. When the integration name or ID is known, use ax ai-integrations get (below) instead of paging the list; list is for browsing.

Response fields:

Get a Specific Integration

bashCopy code
ax ai-integrations get NAME_OR_ID
ax ai-integrations get NAME_OR_ID -o json
ax ai-integrations get NAME_OR_ID --space SPACE   # required when using name instead of ID

Use this to inspect an integration's full configuration or to confirm its ID after creation.

Create an AI Integration

Before creating, always list integrations first — the user may already have a suitable one:

bashCopy code
ax ai-integrations list --space SPACE

If no suitable integration exists, create one. The required flags depend on the provider.

Every create command follows this shape; the provider-specific flags come from the table below:

bashCopy code
ax ai-integrations create \
  --name "My OpenAI Integration" \
  --provider OPEN_AI \
  --api-key $OPENAI_API_KEY \
  --enable-default-models

Every integration except LiteLLM, Fireworks AI and Together AI needs a model source: pass --enable-default-models (Arize's default model list for the provider) and/or one or more --model-name flags. Without either, the server rejects the create with 400: An integration must have at least one model available.

For a provider's full example and notes (Bedrock/Vertex metadata, base URLs, flags to avoid), open only that provider's file from the Details column below.

Supported Providers

Run ax ai-integrations create --help for optional flags. Pair --headers with --auth-type PROXY_WITH_HEADERS when the provider sits behind a proxy.

After creation

Capture the returned integration ID (e.g., TGxtSW50ZWdyYXRpb246MTI6YUJjRA==) — it is needed for evaluator creation and other downstream commands. If you missed it, retrieve it:

bashCopy code
ax ai-integrations get NAME_OR_ID --space SPACE -o json

Update an AI Integration

update is a partial update — only the flags you provide are changed. Omitted fields stay as-is.

bashCopy code
# Rename
ax ai-integrations update NAME_OR_ID --name "New Name"

# Rotate the API key
ax ai-integrations update NAME_OR_ID --api-key $OPENAI_API_KEY

# Change the model list (replaces all existing model names)
ax ai-integrations update NAME_OR_ID --model-name gpt-4o --model-name gpt-4o-mini

# Update base URL (for Azure, custom, or NIM)
ax ai-integrations update NAME_OR_ID --base-url "https://new-endpoint.example.com/v1"

# Restrict visibility to specific spaces (full replace — lists all spaces that should have access)
ax ai-integrations update NAME_OR_ID \
  --scopings '[{"space_id": "SPACE_GLOBAL_ID", "scoping_type": "include"}]'

Add --space SPACE when using a name instead of ID. Any flag accepted by create can be passed to update.

--scopings flag: Controls which spaces can use this integration. Accepts a JSON array of scoping rules. Replaces all existing scopings on update. Use ax spaces list -o json to find space global IDs.

Delete an AI Integration

Warning: Deletion is permanent. Evaluators that reference this integration will no longer be able to run.

bashCopy code
ax ai-integrations delete NAME_OR_ID --force
ax ai-integrations delete NAME_OR_ID --space SPACE --force   # required when using name instead of ID

Omit --force to get a confirmation prompt instead of deleting immediately.

ax integrations (LLM and agent integrations)

ax integrations manages LLM and agent integrations in one group (list, get, delete, create llm|agent, update llm|agent). Run ax integrations <sub> --help for flags.

• Choosing a create command: use ax ai-integrations create (the examples above) for AZURE_OPEN_AI or when --auth-type is needed; ax integrations create llm accepts neither. Use ax integrations create llm when scopings must be set at create time (--scopings).
• create llm provider flags differ from ax ai-integrations create: Bedrock and Vertex AI take dedicated flags instead of --provider-metadata. Run ax integrations create llm --help for the per-provider list.
• Agent endpoints: use ax integrations create agent to register an agent endpoint; run its --help for flags.
• Provider is immutable in ax integrations update llm; delete and recreate to change it.
• --type: list returns every type merged; filter with --type LLM or --type AGENT. get and delete by name need --type because names are unique only per type; IDs work without it.

Troubleshooting

Related Skills

• arize-evaluator: Create LLM-as-judge evaluators that use an AI integration → use arize-evaluator
• arize-experiment: Run experiments that use evaluators backed by an AI integration → use arize-experiment

Save Credentials for Future Use

See references/ax-profiles.md § Save Credentials for Future Use.