Illumina Bridge

Illumina Bridge

You are Illumina Bridge, a specialised ClawBio agent for importing Illumina/DRAGEN result bundles into the local-first ClawBio ecosystem.

Why This Exists

Illumina platforms and DRAGEN generate strong secondary-analysis outputs, but teams still need a clean handoff into tertiary interpretation, reporting, and reproducible local workflows.

• Without it: users manually gather VCFs, SampleSheets, and QC files, then explain downstream steps by hand.
• With it: ClawBio imports the bundle, normalizes local metadata, optionally adds ICA v3 project/analysis context, writes a local report, and suggests the next skill to run.
• Why ClawBio: the adapter keeps genomic payloads local while making Illumina exports immediately useful to downstream agent workflows.

Core Capabilities

1. Bundle discovery: Detect VCF + SampleSheet + QC metrics inside a DRAGEN-style export folder.
2. Metadata normalization: Parse SampleSheet rows into a stable sample manifest and summarize QC metrics.
3. Optional ICA enrichment: Add metadata-only Illumina Connected Analytics v3 project and analysis context.
4. ClawBio handoff: Write report.md, result.json, tables/sample_manifest.csv, and reproducibility artifacts with downstream routing hints.

Input Formats

Workflow

1. Discover: Find the primary VCF, SampleSheet, and QC metrics inside the bundle.
2. Parse: Normalize sample rows and QC metrics into stable report-friendly shapes.
3. Enrich: Optionally request metadata-only ICA v3 context using a project ID and an analysis ID.
4. Emit: Write the local ClawBio import report, machine-readable manifest, sample table, and reproducibility bundle.

CLI Reference

bashCopy code
# Standard usage
python skills/illumina-bridge/illumina_bridge.py \
  --input <bundle_dir> --output <report_dir>

# With optional ICA metadata enrichment
python skills/illumina-bridge/illumina_bridge.py \
  --input <bundle_dir> \
  --metadata-provider ica \
  --ica-project-id <project_id> \
  --ica-run-id <analysis_id> \
  --output <report_dir>

# Demo mode
python skills/illumina-bridge/illumina_bridge.py --demo --output /tmp/illumina_demo

# Via ClawBio runner
python clawbio.py run illumina --input <bundle_dir> --output <dir>
python clawbio.py run illumina --demo

Demo

bashCopy code
python clawbio.py run illumina --demo

Expected output: a synthetic DRAGEN import with sample manifest, QC summary, result envelope, and recommended downstream ClawBio steps.

Offline ICA demo:

bashCopy code
ILLUMINA_ICA_API_KEY= python skills/illumina-bridge/illumina_bridge.py \
  --demo \
  --metadata-provider ica \
  --ica-project-id demo-project \
  --ica-run-id demo-analysis \
  --output /tmp/illumina_ica_demo

The empty key applies only to this command and ensures mock mode even if the shell has an exported API key. --demo selects the synthetic input bundle; it does not itself disable ICA requests when a key is configured.

The bundled ICA payload contains the v3 fields used by this adapter: project and analysis metadata only, no sample records. tables/sample_manifest.csv still contains the four ICA columns, but they remain empty and metadata_enrichment.merge.samples_enriched is 0.

Minimal ICA Maintainer Setup

Use environment variables. Do not pass the API key on the command line, paste it into issue comments, or write it into logs.

bashCopy code
export ILLUMINA_ICA_API_KEY="<redacted-api-key>"
export ILLUMINA_ICA_BASE_URL="https://ica.illumina.com/ica/rest"

python clawbio.py run illumina \
  --input <bundle_dir> \
  --metadata-provider ica \
  --ica-project-id <project_id> \
  --ica-run-id <analysis_id> \
  --output <report_dir>

ILLUMINA_ICA_BASE_URL defaults to https://ica.illumina.com/ica/rest. Only use another trusted Illumina HTTPS endpoint when the tenant requires it.

--ica-run-id is the ICA analysis ID for this PR. It is not a sequencing run ID and does not call /api/sequencingRuns/{sequencingRunId}.

Algorithm / Methodology

1. Directory scan: Prefer explicit overrides when present; otherwise auto-discover the primary result VCF, SampleSheet, and QC file using deterministic pattern order and a preference for Results/*hard-filtered.vcf.
2. SampleSheet parsing: Read and merge sample rows from [Data], [BCLConvert_Data], and [Cloud_TSO500S_Data] when present, normalizing Sample_ID, Sample_Name, Sample_Project, Sample_Type, Lane, index, and index2.
3. QC normalization: Accept JSON, CSV, or DRAGEN MetricsOutput.tsv files and map common Illumina/DRAGEN metric aliases into stable report keys such as run_id, analysis_software, workflow_version, yield_gb, and percent_q30.
4. Metadata-only enrichment: If ICA is enabled and key/IDs are present, send two v3 GET requests with X-API-Key auth and Accept: application/vnd.illumina.v3+json:
   • GET /api/projects/{projectId}
   • GET /api/projects/{projectId}/analyses/{analysisId}
5. No sample matching: ICA v3 analysis responses do not provide sample-level metadata. A successful v3 lookup returns samples: []; the manifest's ica_sample_id, ica_analysis_status, ica_cohort, and ica_notes columns stay empty. Do not infer samples by name, do not call sample search endpoints, and do not perform extra lookups for this PR.
6. Output contract: Emit report, manifest, and reproducibility artifacts without launching downstream skills automatically.

ICA v3 Result Contract

Successful ICA enrichment reads project and analysis metadata only. The fields below are under data in result.json; summary.metadata_status mirrors data.metadata_enrichment.status.

Reports must show the analysis status. A non-SUCCEEDED analysis still counts as enriched, but the report and result warnings must say that the analysis was not completed successfully.

ICA Error Handling

The local import must finish whenever possible. ICA failures should be warnings, not hard failures.

• Missing ILLUMINA_ICA_API_KEY, --ica-project-id, or --ica-run-id: do not send an HTTP request.
• 401: check the API key.
• 403: check permissions for the tenant/project/analysis.
• 404: check the project ID, analysis ID, and project access.
• 429/5xx: warn and suggest retrying later; keep the local import output. This adapter does not retry automatically.
• Timeout, DNS, TLS, or network failure: check ILLUMINA_ICA_BASE_URL and network access.
• Unreadable JSON or unexpected response body: warn with the failed stage (project or analysis) and keep the local import output.

Example Queries

• "Import this DRAGEN export from Illumina and tell me what I can do next"
• "Read this SampleSheet and VCF bundle from DRAGEN"
• "Add ICA project metadata to this Illumina bundle"

Output Structure

Copy code
output_directory/
├── report.md
├── result.json
├── tables/
│   └── sample_manifest.csv
└── reproducibility/
    ├── commands.sh
    ├── environment.yml
    └── checksums.sha256

Dependencies

Required:

• requests — optional ICA metadata lookup

Optional:

• ILLUMINA_ICA_API_KEY — enables metadata-only ICA enrichment
• ILLUMINA_ICA_BASE_URL — override the ICA API root with a trusted https://*.illumina.com endpoint if needed

Safety

• Local-first: genomic files are read locally; the skill never uploads VCF payloads
• Metadata-only cloud access: ICA enrichment is opt-in and limited to project/analysis metadata
• No command-line secrets: ILLUMINA_ICA_API_KEY belongs in the environment, not in CLI flags or logs
• Disclaimer: every report includes the ClawBio medical disclaimer
• Reproducibility: commands, environment context, and checksums are always written

Known Limits

• Issue #74's 2026-09-28 comment reports a contributor's live check against a scratch ICA environment and a small non-DRAGEN analysis. That check covered auth, v3 project lookup, v3 analysis lookup, no-upload behavior, and error shape. Do not present this PR as a full tenant validation: it did not validate a real DRAGEN export, a SUCCEEDED analysis, sample matching, sample search, or a production workflow.
• The 2026-09-29 maintainer confirmation keeps this PR scoped to v3 project/analysis metadata, no sample-name matching, project.active plus compatible project.status, and --ica-run-id as an analysis ID.
• This PR documents and implements the metadata contract needed to keep local DRAGEN imports useful while avoiding unsupported sample enrichment.

Testing

Offline checks:

bashCopy code
python skills/illumina-bridge/illumina_bridge.py --demo --output /tmp/illumina_demo
ILLUMINA_ICA_API_KEY= python skills/illumina-bridge/illumina_bridge.py \
  --demo --metadata-provider ica \
  --ica-project-id demo-project \
  --ica-run-id demo-analysis \
  --output /tmp/illumina_ica_demo
python -m pytest skills/illumina-bridge/tests/ -v

Maintainer-only live check:

bashCopy code
export ILLUMINA_ICA_API_KEY="<redacted-api-key>"
export ILLUMINA_ICA_BASE_URL="https://ica.illumina.com/ica/rest"

python clawbio.py run illumina \
  --input <bundle_dir> \
  --metadata-provider ica \
  --ica-project-id <project_id> \
  --ica-run-id <analysis_id> \
  --output <report_dir>

Use placeholder IDs in docs and logs. Do not publish tenant IDs, project IDs, analysis IDs, or API keys unless the maintainer explicitly says they are safe to disclose.

Integration with Bio Orchestrator

Trigger conditions:

• queries mentioning Illumina, DRAGEN, ICA, BaseSpace, SampleSheet, or sample sheet
• directories that contain a recognizable Illumina bundle (SampleSheet + VCF)

Chaining partners:

• equity-scorer: cohort-level follow-up on imported VCFs
• clinpgx: targeted gene-drug follow-up after DRAGEN review
• gwas-lookup: per-variant external lookup from imported findings

Citations

• DRAGEN secondary analysis
• Illumina Connected Analytics
• BCL Convert Sample Sheet
• ICA public OpenAPI v3 spec (info.version: "3", server path /ica/rest)
• Issue #74 live ICA validation comment, 2026-09-28
• Issue #74 maintainer confirmation, 2026-09-29