Evo2 Nim

Evo 2 NIM

Use Evo 2 for DNA generation and, locally, layer-output extraction. Load supplemental files only when needed:

• references/api.md: exact schemas, layer names, Docker flags, hardware notes.
• references/science.md: genomic use cases, limits, and interpretation.
• references/parameters.md: generation/forward parameter effects.
• references/validation.md: DNA, probability, timing, and tensor checks.
• references/examples.md: compact hosted/local request patterns.

Instructions

For generation, use scripts/generate.py to execute the request, validate the response, and save its artifacts. Resolve the script path relative to this skill's directory and choose an output directory in the user's workspace. Use the user's sequence and requested parameters; the example below is only a smoke test.

1. Select the requested mode. For hosted generation, go directly to the generation example; Docker setup and local forward passes are separate tasks.
2. When the user asks to run generation, execute the client and inspect its exit status and result. Writing a script alone does not complete that request.
3. Report the generated DNA (or its file for long sequences), actual elapsed_ms, sampled-probability summary, seed, and artifact paths from the successful run. Read the saved response or metrics if any result is unclear.

If the request or validation fails, report the actual failure and any diagnostic files. Do not replace an unavailable API response with example values. For a code-only request, provide the command without making an inference call.

Choose Mode

Honor NIM_API_MODE when it is set. Accepted values are hosted and local. If it is unset, treat an explicit EVO2_NIM_URL as local; otherwise ask when the requested mode is unclear:

Hosted NVIDIA API or local Docker Evo 2 NIM?

• Hosted generation: https://health.api.nvidia.com/v1/biology/arc/evo2-40b/generate
• Local base URL: $EVO2_NIM_URL, falling back to http://localhost:8000
• Local generation: $EVO2_NIM_URL/biology/arc/evo2/generate
• Local forward/layer outputs: $EVO2_NIM_URL/biology/arc/evo2/forward

Always resolve local health and inference routes from EVO2_NIM_URL when it is present. localhost works only when the caller and NIM share a network namespace; a caller in a separate container usually needs a service URL such as http://evo2-nim:8000. Do not silently switch modes when the selected endpoint is unavailable. Report the failed endpoint and fix its configuration.

The hosted docs expose generation. /forward is documented for local Docker; do not invent a hosted /forward endpoint. Hosted requests use Authorization: Bearer $NGC_API_KEY. Supported local Docker startup uses NGC_API_KEY (or NVIDIA_API_KEY via the preflight) for registry login, entitlement checks, and first-run model downloads; pass it into the container with -e NGC_API_KEY. Local inference requests use no auth header after readiness. Warm-cache key-free startup varies by image/version and should not be assumed.

Examples

Normalize prompts before sending. Use A/C/G/T unless ambiguous bases are a deliberate modeling choice and clearly reported.

For a hosted generation request, run the bundled client with the user's inputs (the script path below is relative to the skill directory):

bashCopy code
python scripts/generate.py \
  --mode hosted \
  --sequence ACTGACTGACTGACTG \
  --num-tokens 64 --seed 1 \
  --temperature 0.7 --top-k 3 --top-p 0.0 \
  --output-dir /path/to/workspace/evo2-output

For an already-ready local NIM, use --mode local; the client resolves EVO2_NIM_URL and sends no Authorization header. It never switches endpoints after a failed request. Set --timeout for a longer read if the user requests a larger generation; failed requests are not automatically resubmitted.

The client saves request.json, the actual response.json, generated.fasta, and metrics.json in the chosen output directory. It also saves the exact response body in response.raw before checking HTTP status or parsing JSON, so diagnostics survive malformed JSON and non-finite probability/timing values. It validates the requested number of generated bases, A/C/G/T alphabet, finite sampled probabilities in [0, 1], and nonnegative timing before printing a successful summary. Existing directories are never reused, even if empty. Choose an output directory that does not exist; the client creates it atomically so concurrent runs cannot overwrite each other's artifacts. The FASTA contains generated bases only, not the input prompt prepended again.

sampled_probs is requested by the client and summarized with count/min/max/mean; the full values stay in the saved response. A missing or malformed probability array is a validation failure, not permission to invent confidence values. Only request enable_logits in a custom request when needed; logits can make responses large. See references/api.md for custom payloads. random_seed supports development reproducibility, not biological certainty.

Local Docker Requirements

Evo 2 local deployment requires FP8-capable GPUs. Do not present A100 as compatible; A100 can pull the image but fails warmup because FP8 requires compute capability 8.9 or higher.

• Default 40B: 2x H100 80 GB or 1x H200 141 GB. Use NIM_TEST_GPUS=0,1 for 2x H100, or NIM_TEST_GPUS=0 for one H200.
• 7B fallback: set NIM_VARIANT=7b; supported GPUs include H100, H200, RTX 6000 Ada, and L40S.
• Approximate disk: 110 GB for 40B, 50 GB for 7B.

Use shell env first; source repo-root .env only if present. Do not invent a cache default or drop the NVIDIA_API_KEY fallback.

bashCopy code
set -a
[ -f .env ] && . ./.env
set +a

if [ -z "${NGC_API_KEY:-}" ] && [ -n "${NVIDIA_API_KEY:-}" ]; then
  export NGC_API_KEY="$NVIDIA_API_KEY"
fi
: "${NGC_API_KEY:?Set NGC_API_KEY or NVIDIA_API_KEY}"
: "${LOCAL_NIM_CACHE:?Set LOCAL_NIM_CACHE}"

echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

# 40B default: 0,1 for 2x H100; set 0 for a single H200.
export NIM_TEST_GPUS="${NIM_TEST_GPUS:-0,1}"
mkdir -p "${LOCAL_NIM_CACHE}"
chmod 700 "${LOCAL_NIM_CACHE}"   # owner-only; if the NIM runs as a different UID, add -u "$(id -u)" to docker run

# For 7B: export NIM_VARIANT=7b; export NIM_TEST_GPUS="${NIM_TEST_GPUS:-0}"
docker run --rm -it --name evo2-nim \
  --runtime=nvidia \
  --gpus "\"device=${NIM_TEST_GPUS}\"" \
  -e NGC_API_KEY \
  -e NIM_VARIANT \
  -v "${LOCAL_NIM_CACHE}:/opt/nim/.cache" \
  -p 8000:8000 \
  nvcr.io/nim/arc/evo2:2

Readiness:

bashCopy code
evo2_nim_url="${EVO2_NIM_URL:-http://localhost:8000}"
until curl -sf "${evo2_nim_url%/}/v1/health/ready"; do sleep 10; done

If RTX PRO 6000 Blackwell Workstation fails with no Transformer Engine attention backend, treat it as outside the current validated matrix and rerun on a documented GPU/runtime.

Local Forward Pass

Forward returns base64-encoded NPZ tensors.

pythonCopy code
import base64
import io
import os
import numpy as np
import requests

mode = os.getenv("NIM_API_MODE", "local")
if mode != "local":
    raise RuntimeError("Evo 2 /forward is available only in local mode")
nim_url = os.getenv("EVO2_NIM_URL", "http://localhost:8000").rstrip("/")
sequence = "ACTGACTGACTG"  # Replace with the user's DNA sequence.
sequence = "".join(sequence.upper().split())
if not sequence or set(sequence) - set("ACGT"):
    raise ValueError("Expected nonempty A/C/G/T DNA")
payload = {
    "sequence": sequence,
    "output_layers": ["output_layer", "decoder.layers.3.self_attention"],
}
response = requests.post(
    f"{nim_url}/biology/arc/evo2/forward",
    headers={"Content-Type": "application/json"},
    json=payload,
    timeout=300,
)
response.raise_for_status()
npz_bytes = base64.b64decode(response.json()["data"])
with open("evo2_forward_outputs.npz", "wb") as handle:
    handle.write(npz_bytes)
arrays = np.load(io.BytesIO(npz_bytes), allow_pickle=False)
for name in arrays.files:
    arr = arrays[name]
    print(name, arr.shape, arr.dtype, bool(np.isfinite(arr).all()), float(arr.mean()))

Validate And Report

Save request/response JSON, generated FASTA, and a metrics JSON with sequence length, GC fraction, ambiguous-base fraction, homopolymer length, sampled-prob checks, and elapsed timing. Treat invalid schema or alphabet as hard failures; treat extreme GC, low complexity, duplicates, and missing motifs as warnings. For deeper checks, read references/validation.md.

Key fields: sequence, num_tokens, temperature, top_k (0-6), top_p (0-1), random_seed, enable_sampled_probs, enable_elapsed_ms_per_token, and optional enable_logits.

Troubleshooting

• 401/403: hosted key missing/expired or not sent as Bearer token.
• 422: wrong field names such as max_tokens instead of num_tokens.
• Local endpoint confusion: print NIM_API_MODE and EVO2_NIM_URL; do not replace a configured service URL with localhost.
• Local auth confusion: do not send Authorization to local inference.
• Local startup: first run downloads model assets; poll $EVO2_NIM_URL/v1/health/ready before inference.
• FP8 failure: use hosted, 7B on a supported FP8 GPU, or documented 40B GPUs.