# Propagating Trace Context Through Tile Pipelines

A tile request that hands off to an asynchronous generation worker through a queue loses its parent trace unless the W3C traceparent is carried explicitly across the boundary, because message brokers do not forward HTTP headers and the SDK’s automatic context ends at the process edge. This operation injects and extracts traceparent across both an HTTP hop and a Kafka queue hop, and carries crs and bbox as OpenTelemetry baggage, so a tile request stays traceable end to end even when its work runs minutes later on a different node. It is the propagation procedure the instrumentation in Distributed Tracing for Spatial Request Flows assumes, within Spatial Pipeline Orchestration & Observability, and it keeps the async tile path consistent with the idempotent DAGs of Orchestrating Spatial Pipelines in Python.

# Prerequisites

Figure — Trace context is injected into the HTTP request, re-injected into the Kafka record headers at the async boundary, then extracted by the worker so the span tree stays connected.

# Step-by-Step Implementation

Each step carries the context one hop further and verifies the link before the next.

# 1. Configure the composite propagator

Trace context and baggage travel in separate headers, so the process must run both propagators. Set them globally at startup so every inject/extract uses the same format.

# propagation_setup.py — enable W3C tracecontext + baggage globally.
from opentelemetry.propagate import set_global_textmap
from opentelemetry.propagators.composite import CompositePropagator
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
from opentelemetry.baggage.propagation import W3CBaggagePropagator

set_global_textmap(CompositePropagator([
    TraceContextTextMapPropagator(),   # traceparent header
    W3CBaggagePropagator(),            # baggage header (crs, bbox)
]))

Verify: confirm both propagator fields are advertised.

python -c "from opentelemetry.propagate import get_global_textmap; print(get_global_textmap().fields)"
# expect: {'traceparent', 'tracestate', 'baggage'}

# 2. Extract the inbound context and set spatial baggage

At the API service, extract the parent context from the incoming HTTP headers so the API span is a child of the caller, then attach crs and bbox as baggage that will ride downstream.

# api_ingest.py — continue the inbound trace and set spatial baggage.
from opentelemetry import trace, baggage, context
from opentelemetry.propagate import extract

def handle_request(headers: dict, crs: str, bbox: str):
    ctx = extract(headers)                        # parent from inbound traceparent
    ctx = baggage.set_baggage("crs", crs, context=ctx)
    ctx = baggage.set_baggage("bbox", bbox, context=ctx)
    tracer = trace.get_tracer("api-service")
    with tracer.start_as_current_span("tiles.enqueue", context=ctx):
        return ctx                                # carried into the producer

Verify: log the active context’s trace id to confirm it matches the caller’s.

curl -s -H 'traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01' \
  http://localhost:8000/tiles?bbox=... | jq '.trace_id'   # must echo 4bf92f35...

# 3. Inject context into the Kafka record headers

The broker will not forward the HTTP header, so inject traceparent and baggage into the record’s message headers as bytes. This is the boundary where propagation is most often lost.

# producer.py — carry trace context across the async boundary.
from opentelemetry.propagate import inject

def enqueue_tile_job(producer, topic: str, payload: bytes, ctx):
    carrier: dict[str, str] = {}
    inject(carrier, context=ctx)                  # writes traceparent + baggage
    headers = [(k, v.encode()) for k, v in carrier.items()]
    producer.produce(topic, value=payload, headers=headers)  # ctx rides the record
    producer.flush()

Verify: read the record headers back off the topic and confirm traceparent is present.

kcat -b broker:9092 -t tile-jobs -C -c1 -f '%h\n'   # prints headers; expect traceparent=...

# 4. Extract context in the worker and continue the trace

The consumer rebuilds the carrier from the record headers, extracts the context, and opens the tile-generation span as a child — reconnecting the tree across the queue.

# worker.py — resume the trace in the async tile generator.
from opentelemetry import trace, baggage
from opentelemetry.propagate import extract

def process_record(record):
    carrier = {k: v.decode() for k, v in record.headers()}
    ctx = extract(carrier)                         # parent from the record headers
    crs = baggage.get_baggage("crs", context=ctx)  # spatial context survives the hop
    bbox = baggage.get_baggage("bbox", context=ctx)
    tracer = trace.get_tracer("tile-worker")
    with tracer.start_as_current_span("tile.generate", context=ctx) as span:
        span.set_attribute("crs", crs)
        span.set_attribute("bbox", bbox)
        return render(record.value(), crs, bbox)

Verify: query the backend for the trace id and confirm the worker span is a descendant, not a new root.

curl -s "http://otel-query.mesh.internal/api/traces/4bf92f35...b34da6" \
  | jq '[.spans[].operationName]'   # includes tile.generate under the same trace

# Configuration Reference

# Common Failure Modes & Fixes

Worker spans appear as separate root traces. Root cause: start_as_current_span was called without the extracted context=ctx, so the SDK opens a fresh root. Fix: pass the extracted context explicitly into the span start in the consumer.

traceparent missing from Kafka records. Root cause: inject wrote to a carrier that was never attached to the record, or headers were dropped by a serializer. Fix: encode the carrier into the record headers as bytes and confirm with kcat -f '%h'.

Baggage is empty in the worker. Root cause: only the tracecontext propagator is configured, so the baggage header is neither written nor read. Fix: register the composite propagator with both TraceContextTextMapPropagator and W3CBaggagePropagator.

Trace links but crs/bbox are wrong downstream. Root cause: baggage was set on a context that was not the one injected, so a stale value rode the record. Fix: thread the same ctx through set-baggage, span start, and inject without reassigning it in between.

Context leaks across unrelated jobs in the worker. Root cause: the extracted context was attached globally and never detached, so the next record inherits the previous trace. Fix: scope the context to the with block per record and never call a bare context.attach without a matching detach.

# FAQ

# Related

• Up to the parent: Distributed Tracing for Spatial Request Flows
• Orchestrating Spatial Pipelines in Python — the async pipelines whose hops carry this context
• SLA Monitoring for Spatial Data Products — the SLOs a whole trace helps diagnose
• Spatial Pipeline Orchestration & Observability — up to the section overview