Otel Instrumentation

OpenTelemetry Instrumentation Guide

Expert guidance for implementing high-quality, cost-efficient OpenTelemetry telemetry.

Rules & Quick Reference

Official documentation

• OpenTelemetry Documentation
• Semantic Conventions
• Dash0 Integration Hub

Getting started

Follow these steps when instrumenting an application from scratch:

1. Pick your SDK rule — choose the language-specific rule from the table above (e.g., nodejs, python).
2. Set up resource attributes — define service identity and environment per resources.
3. Add spans, metrics, and logs — instrument your code following spans, metrics, and logs.
4. Guard sensitive data — scrub PII before export per sensitive-data.
5. Validate — confirm telemetry reaches the backend using the checklist in validation.

The snippet below shows a complete span with attributes and status for Node.js — see nodejs for full setup including SDK initialisation, exporter configuration, and auto-instrumentation:

jsCopy code
import { trace, SpanStatusCode } from '@opentelemetry/api';
const tracer = trace.getTracer('my-service', '1.0.0');

tracer.startActiveSpan('operation-name', async (span) => {
  try {
    span.setAttribute('user.id', userId);
    span.setAttribute('order.id', orderId);

    const result = await processOrder(orderId);

    span.setAttribute('order.status', result.status);
    span.setStatus({ code: SpanStatusCode.OK });
    return result;
  } catch (err) {
    // Record the exception as a structured log record, not span.recordException — see rules/spans.md
    span.setStatus({ code: SpanStatusCode.ERROR, message: `${err.name}: ${err.message}` });
    const spanContext = span.spanContext();
    logger.error('operation-name.failed', {
      'trace_id': spanContext.traceId,
      'span_id': spanContext.spanId,
      'exception.type': err.name,
      'exception.message': err.message,
      'exception.stacktrace': err.stack,
    });
    throw err;
  } finally {
    span.end();
  }
});

Key principles

Signal density over volume

Every telemetry item should serve one of three purposes:

• Detect - Help identify that something is wrong
• Localize - Help pinpoint where the problem is
• Explain - Help understand why it happened

If it doesn't serve one of these purposes, don't emit it.

Sample in the pipeline, not the SDK

Set the AlwaysOn sampler explicitly in every SDK (OTEL_TRACES_SAMPLER=always_on). This overrides the OpenTelemetry default, ParentBased(root=AlwaysOn), under which callers outside the trust boundary can silently switch off a service's tracing through the traceparent header. Do not configure ratio-based SDK samplers either — they make irreversible decisions before the outcome of a request is known. Defer all sampling to the Collector, where policies can be changed centrally without redeploying applications. See SDK sampling for the full rationale and the conditions under which ParentBased is acceptable.

Copy code
SDK (AlwaysOn, set explicitly)  →  Collector (sampling)  →  Backend (retention)
             ↓                           ↓                       ↓
       All spans                   Head or tail            Storage policies
       exported                    sampling applied