Genai Hub Tabular Orchestration

Rules

1. Run all scripts from the skill root with uv run scripts/<name>.py.
2. Never print or log credential values — the SDK reads from the environment.
3. Always verify TCR prerequisites (data destination → tabular artifact → scenario configuration) are in place before sending a prediction request.
4. rows and columns are mutually exclusive in a prediction request — never include both.
5. For credential or auth errors, invoke aicore-admin-resources first.
6. Mutating scripts require --confirm; always pass it explicitly for delete operations.
7. Omit --resource-group when the user doesn't specify one; the SDK default is used.
8. Before issuing any btp update services/instance command: (a) run btp get services/instance --id <id> to fetch the full current parameters block, (b) show the user the complete current parameters and the exact diff — what is being added, changed, or removed — along with the full resulting payload that will be sent, and (c) require the user to explicitly confirm the shown parameters and the proposed change before proceeding. Do not issue btp update services/instance under any circumstance until the user has confirmed in that same turn.

Concepts

The Context Registry (TCR) exposes structured data to AI Core scenarios through three layered resources:

Copy code
Data Destination  →  Tabular Artifact  →  Scenario Configuration
(HDL connection)      (file + schema)       (named bundle of TAs)

• Data Destination: stores connection credentials for an HDL (HANA Data Lake) endpoint.
• Tabular Artifact (TA): points to a specific file on a data destination, with schema metadata (CSN). Supports CSV, Parquet, and Delta formats. Schema can be provided inline (DOCUMENT), referenced via a path (REFERENCE), or auto-derived (AUTO).
• Scenario Configuration: a named, reusable list of tabular artifact names — consumed by AI Core scenarios at inference time.

All resources are scoped to a resource group (AI-Resource-Group header). Use the aicore-admin-resources skill to verify credentials and get the active resource group first.

Quick Start

bashCopy code
# 1. Check setup
uv run scripts/check_setup.py    # from aicore-admin-resources, or verify env vars

# 2. List all three resource types
uv run scripts/list_data_destinations.py
uv run scripts/list_tabular_artifacts.py
uv run scripts/list_scenario_configs.py

# 3. Register a new HDL data destination
uv run scripts/create_data_destination.py --name my-hdl --host <hdl-hostname>

# 4. Register a tabular artifact (AUTO schema, CSV)
uv run scripts/create_tabular_artifact.py \
  --name customer-ta \
  --data-destination my-hdl \
  --path /data/customer.csv \
  --type CSV

# 5. Create a scenario configuration
uv run scripts/create_scenario_config.py \
  --name my-scenario \
  --tabular-artifacts customer-ta,orders-ta

Data Destinations

A data destination stores the HDL hostname and credentials; the system validates the connection on creation and stores the credentials securely.

Before creating a data destination, ask the user for:

1. Name — kebab-case, 1–127 chars (e.g. my-hdl)
2. HDL hostname — no https:// prefix (e.g. abc123.files.hdl.eu10.hanacloud.ondemand.com)
3. (Optional) Labels — key=value pairs following (ext|int).ai.sap.com/<suffix> pattern
4. (Optional) Description — free-text description

bashCopy code
uv run scripts/list_data_destinations.py               # list all
uv run scripts/list_data_destinations.py --json        # pipe-friendly JSON

uv run scripts/get_data_destination.py --name <name>   # get details of one destination
uv run scripts/get_data_destination.py --name <name> --json

uv run scripts/create_data_destination.py \
  --name <name> \
  --host <hdl-hostname>
  # hostname only — no https:// prefix, e.g. abc123.files.hdl.eu10.hanacloud.ondemand.com
  # optional: --labels ext.ai.sap.com/env=prod --description "Production HDL"

# Update labels (replaces the entire label set)
uv run scripts/patch_data_destination.py --name <name> --labels ext.ai.sap.com/env=prod
uv run scripts/patch_data_destination.py --name <name> --labels ext.ai.sap.com/env=prod ext.ai.sap.com/team=data

# Search by labels
uv run scripts/search_data_destinations.py --labels ext.ai.sap.com/env=prod
uv run scripts/search_data_destinations.py --labels ext.ai.sap.com/env=prod --json

uv run scripts/delete_data_destination.py --name <name> --confirm
# Note: deletes cascade to all tabular artifacts that reference this destination.

Name constraints: lowercase letters, numbers, hyphens; 1–127 chars; must start and end with alphanumeric.

Labels follow the pattern (ext|int).ai.sap.com/<suffix> and are useful for filtering via the search endpoint (see references/API.md).

Required post-creation step — register subject patterns in HDL: After create_data_destination.py succeeds, the response includes one or more X.509 subject DNs (subjectPatterns). These must be added as authorizations in the HDL Files service instance configuration. Walk the user through the following steps:

0. Verify BTP CLI login

Before running any btp commands, ensure the BTP CLI is installed and you are logged in. See references/SETUP.md for installation and login instructions.

1. Find the HDL Files instance ID

bashCopy code
btp target --subaccount <subaccount-id>
btp list services/instance

Look for the instance whose name or type corresponds to the HDL Files service used by this data destination.

2. Fetch and show current instance parameters

bashCopy code
btp get services/instance --id <hdl-instance-id>

Extract the full parameters block from the output. Show it to the user verbatim. Note any existing authorizations entries — they must be preserved in the update payload.

3. Build and confirm the update payload

Construct the full parameters payload with the new subject pattern(s) added alongside all existing authorizations entries. Show the user:

• The current authorizations array (from step 2)
• The new entry or entries being added
• The complete resulting payload that will be sent

If AI Core returns multiple patterns, add one entry per pattern with sequential ranks. The authorizations array replaces the existing one — always include all existing entries.

Ask the user to explicitly confirm the current parameters and the proposed payload before continuing. Do not proceed until confirmation is received in the same turn.

4. Apply the update

Only after the user has confirmed:

bashCopy code
btp update services/instance \
  --id <hdl-instance-id> \
  --parameters '{
    "data": {
      "fileContainer": {
        "authorizations": [
          {
            "pattern": "<subject-pattern-from-ai-core>",
            "rank": 1,
            "roles": ["user"]
          }
        ]
      }
    }
  }'

Do you have this role? Requires Subaccount Administrator or Service Administrator on the subaccount that owns the HDL instance. If not, share the subject pattern(s) with your BTP admin and ask them to perform this step.

Propagation delay: changes can take 1–2 minutes to take effect. If tabular artifact creation still returns 403 immediately after the update, wait and retry.

Do not proceed to tabular artifact creation until the user confirms the subject patterns have been registered (or the BTP admin has done so).

Tabular Artifacts

A tabular artifact registers a file on a data destination as a queryable virtual table. Three schema definition modes:

Before creating a tabular artifact, ask the user for:

1. Name — kebab-case, 1–80 chars (e.g. customer-data)
2. Data destination name — must already exist (list with list_data_destinations.py if unsure)
3. File path — absolute path to the file on the HDL (e.g. /data/customer.csv)
4. File type — CSV, PARQUET, or DELTA
5. (DOCUMENT mode only) Entity name — a PascalCase label for the table (e.g. Customer)
6. (DOCUMENT mode only) Column names — exactly as they appear in the file header row
7. (DOCUMENT mode only) Type for each column — use the full cds.* type name (e.g. cds.String, cds.Integer)

Then use --csn-columns to pass them inline:

bashCopy code
uv run scripts/list_tabular_artifacts.py               # list all
uv run scripts/list_tabular_artifacts.py --json

uv run scripts/get_tabular_artifact.py --name <name>   # get details of one artifact
uv run scripts/get_tabular_artifact.py --name <name> --json

# AUTO schema (simplest — works well for CSV with a header row)
uv run scripts/create_tabular_artifact.py \
  --name <name> \
  --data-destination <dd-name> \
  --path /data/file.csv \
  --type CSV

# DOCUMENT schema — supply column names and types explicitly
uv run scripts/create_tabular_artifact.py \
  --name <name> \
  --data-destination <dd-name> \
  --path /data/file.csv \
  --type CSV \
  --csn-columns "id:cds.Integer,name:cds.String,score:cds.Decimal,active:cds.Boolean" \
  --entity-name MyEntity

# REFERENCE schema (CSN file lives on the HDL)
uv run scripts/create_tabular_artifact.py \
  --name <name> \
  --data-destination <dd-name> \
  --path /data/file.parquet \
  --type PARQUET \
  --csn-reference /data/metadata/schema.json \
  --entity-name MyEntity

uv run scripts/delete_tabular_artifact.py --name <name> --confirm
# Deletion is async (202 Accepted); the artifact transitions to DELETING status.

--csn-columns format: comma-separated name:cds.Type pairs. Supported types: cds.UUID, cds.Boolean, cds.Integer, cds.Int16, cds.Int32, cds.Int64, cds.UInt8, cds.Decimal, cds.Double, cds.Date, cds.Time, cds.DateTime, cds.Timestamp, cds.String, cds.Binary, cds.LargeBinary, cds.LargeString, cds.Map, cds.Vector. Column names must match the file's header row exactly.

Name constraints: lowercase letters, numbers, hyphens; 1–80 chars; must start and end with alphanumeric.

Scenario Configurations

A scenario configuration is a named, stable list of tabular artifact names that AI Core scenarios reference by name at inference time.

bashCopy code
uv run scripts/list_scenario_configs.py
uv run scripts/list_scenario_configs.py --json

uv run scripts/get_scenario_config.py --name <name>    # get details of one config
uv run scripts/get_scenario_config.py --name <name> --json

uv run scripts/create_scenario_config.py \
  --name <name> \
  --tabular-artifacts ta1,ta2,ta3

# Update the TA list (replaces the entire list)
uv run scripts/patch_scenario_config.py \
  --name <name> \
  --tabular-artifacts ta1,ta2,ta3,ta4

uv run scripts/delete_scenario_config.py --name <name> --confirm

Name constraints: same as data destinations (lowercase, hyphens, 1–127 chars).

Tabular Prediction (predict.py)

Use predict.py to send inference requests to a deployed tabular RPT model (sap-rpt-1-small or sap-rpt-1-large). The model reads context rows from TCR and predicts values for query rows.

bashCopy code
uv run scripts/predict.py --deployment-id <id> --body '<json>'
# optional: --json  --main-tenant <uuid>  --resource-group <rg>

Rules

1. TCR prerequisites must be in place — a tabular artifact, a scenario configuration, and a data destination must all be registered (see sections above) before sending a prediction request.

2. rows and columns are mutually exclusive — never include both in the same request.

3. Prediction target must contain "[PREDICT]" — every entry in the target column for query rows must be the placeholder string "[PREDICT]".

Interactive Builder Flow

Work through these steps in order. Skip any step the user has already answered.

Step 1 — Confirm TCR prerequisites

Ask:

Copy code
Before we proceed, please confirm you have already set up the following in TCR:
  ✓ Tabular artifact (context dataset uploaded)
  ✓ Scenario configuration (scenarioConfigName you want to use)
  ✓ Data destination

If any of these are missing, I can help you set them up first.

If not set up, help the user register them using the TCR scripts above, then return to the prediction flow.

Step 2 — Resolve scenarioConfigName and deploymentId

bashCopy code
uv run scripts/list_scenario_configs.py

Get deployment ID (tabular RPT deployments may not appear under known scenarios):

bashCopy code
uv run ../aicore-lifecycle-management/scripts/get_deployments.py --status RUNNING

Once both are confirmed, also ask for these in a single message:

Step 3 — Collect and format data

Ask the user to provide their query data. Accept any format:

• Pasted table / CSV text
• JSON rows [{...}, {...}]
• Key-value pairs
• Spreadsheet-style

Transform it according to the format chosen in Step 2:

If columns format:

jsonCopy code
"columns": {
  "Col A": ["val1", "val2", "val3"],
  "Col B": [10, 20, 30],
  "Target": ["[PREDICT]", "[PREDICT]", "[PREDICT]"]
}

If rows format:

jsonCopy code
"rows" : [
  {"Col A": "val1", "Col B": 10, "Target": "[PREDICT]"},
  {"Col A": "val2", "Col B": 20, "Target": "[PREDICT]"}
]

Rules for formatting:

• Replace the target column values with "[PREDICT]" for all query rows
• Ensure all arrays have the same length
• Preserve column names exactly as the user provided them

Also ask whether the user has context data (rows with known answers for the model to learn from). If yes, collect it and format it as contextColumns — keeping the real target values (do not use "[PREDICT]" in context data).

Step 4 — Build predictionConfig

jsonCopy code
"predictionConfig": {
  "targetColumns": [
    {
      "name": "<target column name>",
      "predictionPlaceholder": "[PREDICT]",
      "taskType": "<classification | regression>"
    }
  ]
}

Step 5 — Build contextSelectionConfig

Ask the user to choose how context rows should be sampled from the data destination and how many to include.:

Copy code
How should context rows be sampled?
  A) random    — default, good general purpose
  B) heuristic — scoring-based, finds similar rows
  C) none      — take from HEAD or TAIL of the table
How many context rows? (default: 50)

Based on selection, refer to the contextSelectionConfig section in references/PREDICT.md to build the JSON object. If the user chooses heuristic sampling, ask for or confirm the unique row ID column required as indexColumn. Give the user the option if they want to customise the strategyConfig, otherwise use sensible defaults.

Step 6 — Assemble and send

Show the user the complete assembled JSON body, then run:

bashCopy code
uv run scripts/predict.py \
  --deployment-id <deploymentId> \
  --body '<assembled-json>'

Common Errors

TCR / Context Registry

For full endpoint details, field schemas, and pagination parameters see references/API.md.

Prediction

Handoffs

When to Load Reference Files