Genai Hub Orchestration

Rules

1. Orchestration always requires a running orchestration deployment as a prerequisite — invoke aicore-lifecycle-management first if one does not exist.
2. The orchestration endpoint is POST {DEPLOYMENT_URL}/v2/completion — not /v1/chat/completions.
3. Orchestration scenario_id and executable_id are always "orchestration" — these are fixed.
4. The Harmonized API normalizes all models to OpenAI message format — the same client code works for GPT, Gemini, and Bedrock.
5. If credentials are not configured, invoke aicore-admin-resources first.
6. generative-ai-hub-sdk is deprecated — always use sap-ai-sdk-gen. Import namespace gen_ai_hub is shared by both.
7. If the user's request does not explicitly specify an approach, use AskUserQuestion to ask whether they prefer the REST API (curl) or the Python SDK before writing or running any code.

Deployment Prerequisite

Before using the orchestration service, a deployment must be running:

bashCopy code
# Check for an existing orchestration deployment:
uv run skills/aicore-lifecycle-management/scripts/get_deployments.py --scenario orchestration

# If none exists, create a configuration first (reuse if "orchestration" config already exists):
uv run skills/aicore-lifecycle-management/scripts/get_configurations.py --scenario-id orchestration --json | jq '.[] | select(.name == "orchestration")'
uv run skills/aicore-lifecycle-management/scripts/create_configuration.py \
  --name orchestration \
  --scenario-id orchestration \
  --executable-id orchestration

# Then deploy (takes up to 10 minutes):
uv run skills/aicore-lifecycle-management/scripts/create_deployment.py --configuration-id <config-id> --wait --timeout 600

Workflow

Asset Map

Python SDK — Key Patterns

Basic pipeline

pythonCopy code
from gen_ai_hub.orchestration.service import OrchestrationService
from gen_ai_hub.orchestration.models.config import OrchestrationConfig
from gen_ai_hub.orchestration.models.llm import LLM
from gen_ai_hub.orchestration.models.template import Template, TemplateValue
from gen_ai_hub.orchestration.models.message import SystemMessage, UserMessage

config = OrchestrationConfig(
    template=Template(messages=[
        SystemMessage("You are a helpful assistant."),
        UserMessage("{{?user_query}}"),
    ]),
    llm=LLM(name="gpt-4o-mini", version="latest"),
)
svc = OrchestrationService(config=config)
result = svc.run(template_values=[TemplateValue("user_query", "Hello")])
print(result.orchestration_result.choices[0].message.content)

Add content filtering

pythonCopy code
from gen_ai_hub.orchestration.models.content_filtering import (
    ContentFiltering,
    InputFiltering,
    OutputFiltering,
)
from gen_ai_hub.orchestration.models.azure_content_filter import AzureContentFilter

filtering = ContentFiltering(
    input_filtering=InputFiltering(filters=[AzureContentFilter(hate=2, violence=2, sexual=2, self_harm=2)]),
    output_filtering=OutputFiltering(filters=[AzureContentFilter(hate=2, violence=2, sexual=2, self_harm=2)]),
)
config = OrchestrationConfig(template=..., llm=..., filtering=filtering)

REST API (curl)

bashCopy code
export TOKEN=$(uv run skills/aicore-admin-resources/scripts/get_token.py)
export DEPLOYMENT_URL=$(uv run skills/aicore-lifecycle-management/scripts/get_deployments.py \
  --scenario orchestration --json 2>/dev/null | jq -r '.[0].deployment_url')
export RESOURCE_GROUP=$(uv run skills/aicore-admin-resources/scripts/check_setup.py 2>/dev/null | jq -r '.resource_group')

curl --request POST "${DEPLOYMENT_URL}/v2/completion" \
  --header "Authorization: Bearer ${TOKEN}" \
  --header "AI-Resource-Group: ${RESOURCE_GROUP}" \
  --header "content-type: application/json" \
  --data-raw '{
    "config": {
      "modules": {
        "prompt_templating": {
          "prompt": {
            "template": [
              {"role": "system", "content": "You are a helpful assistant."},
              {"role": "user",   "content": "{{?user_query}}"}
            ]
          },
          "model": {
            "name": "gpt-4o-mini",
            "params": {"max_completion_tokens": 512}
          }
        }
      }
    },
    "placeholder_values": {"user_query": "What is SAP AI Core?"},
    "messages_history": []
  }'

Extract the answer with: jq '.final_result.choices[0].message.content'

Reasoning models (Gemini 2.5 Pro, o3): set max_completion_tokens to at least 1024 — reasoning tokens consume the budget before the visible response.

Handoffs

• No orchestration deployment running? → invoke aicore-lifecycle-management
• Credential/auth error? → invoke aicore-admin-resources
• Need to find a model name? → invoke genai-hub-foundation-models
• Want full Java SDK boilerplate (non-orchestration)? → invoke aicore-sdk-java
• Want full JS SDK boilerplate (non-orchestration)? → invoke aicore-sdk-js

When to Load Reference Files