Last reviewed: 2026-08-26

A failover should change the provider attempt, not the identity of the user operation. Give the operation one gateway trace, then represent each primary, retry, and CometAPI attempt as a child span. Forward a validated trace context at service boundaries and store each provider request identifier as an attribute of the relevant attempt. This keeps a timeout, a route switch, and the eventual answer in one causal view without pretending that a provider request ID is itself a distributed trace.

Direct answer

Use a three-layer contract: a gateway operation, an attempt span for every provider call, and provider correlation fields attached to that attempt.

  1. Extract at ingress. Read the W3C traceparent and optional tracestate headers from the incoming request. Validate their shape and apply your trust policy before accepting them. If the context is missing or invalid, start a new local root rather than copying arbitrary input into telemetry.
  2. Create one logical operation span. Name it something stable such as llm.request. Put the user-action identifier, tenant classification, and route plan in controlled attributes. Do not put prompts, completions, authorization material, or raw user text in span attributes.
  3. Create one child per attempt. A primary request, a bounded retry, and a CometAPI fallback are separate llm.attempt spans under the same operation. Record the provider, model alias, endpoint class, region policy, attempt number, start and end times, and outcome. A new attempt span makes the causal order visible while preserving one trace ID.
  4. Propagate context on internal hops. OpenTelemetry describes extracting remote context and injecting it into downstream calls. Use that mechanism for your gateway, queue worker, policy service, and CometAPI adapter. If a third-party provider contract does not explicitly accept tracing headers, keep the provider call as a child span locally and do not disclose internal context merely to make a vendor dashboard line up.
  5. Attach provider IDs without replacing trace IDs. OpenAI documents x-request-id as a unique request identifier in response headers. Anthropic documents a unique request-id header and the same identifier in an error body’s request_id field. Store those values on the attempt span and in the incident record. They are lookup handles for a provider, not substitutes for traceparent.
  6. Close the graph deliberately. On success, end the attempt span and mark the operation as successful, with fallback_used set to false or true as appropriate. On a terminal error, end every open attempt span, record the final route, and return a user-safe response. Never leave a retry as an unclosed parent while starting a new unrelated root.

The W3C Trace Context specification defines traceparent as the portable position in a trace and tracestate as optional vendor-specific state. OpenTelemetry context propagation explains how those values connect spans and correlated signals across process and network boundaries. Together they support a simple rule: preserve the operation context, create a new span for each attempt, and keep provider-specific IDs in a separate namespace.

A small internal record can make the rule explicit without retaining content:

logical_request_id: [LOCAL_REQUEST_ID]
traceparent: [TRACEPARENT]
attempts:
  - attempt_id: attempt_01
    provider: primary
    span_role: llm.attempt
    provider_request_id: [PROVIDER_REQUEST_ID]
    outcome: timeout
  - attempt_id: attempt_02
    provider: cometapi
    span_role: llm.attempt
    provider_request_id: [PROVIDER_REQUEST_ID]
    outcome: success

Who this is for

This is for platform engineers, SREs, and on-call operators who own a gateway that can retry or change providers for one user action. It is especially useful when the primary vendor, a CometAPI adapter, a queue worker, and an observability backend are maintained by different teams. Application developers can use the same contract when a streaming or asynchronous request outlives the original HTTP handler.

It is not a guide to selecting a tracing vendor or to exporting customer content. The focus is the boundary between routing and evidence: which span owns a decision, which identifier can be handed to a provider, and which fields are safe to retain during an incident.

Key takeaways

  • Keep one trace for one user operation; represent provider changes as child attempts.
  • Validate incoming traceparent; treat tracestate as optional, bounded, and untrusted until policy allows it.
  • Preserve OpenAI x-request-id and Anthropic request IDs beside, never instead of, your own trace and span IDs.
  • Make retry, failover, and final outcome explicit span events so an operator can reconstruct the route without reading prompts.
  • Carry context across queues and workers explicitly; an asynchronous hop that drops context creates a misleading trace with a missing middle.
  • Review the retry and backoff evidence checklist alongside this design, and compare the resulting events with the fallback decision log guidance .

Sources checked

The following public sources were checked for the protocol and provider facts used here:

  • The W3C Trace Context Recommendation specifies the HTTP header format, processing model, forwarding behavior, and privacy considerations for traceparent and tracestate.
  • OpenTelemetry Context Propagation describes extracting context from a remote request, injecting it into a downstream request, and correlating traces, logs, and other signals.
  • The OpenAI API Reference overview lists x-request-id as a unique request identifier used for troubleshooting, alongside rate-limit response headers. It also warns that oversized headers may prevent an ID from being returned.
  • Claude API errors documents HTTP error classes, retry guidance for transient failures, mid-stream errors after a 200 response, and the request ID in both the response header and error body.

These sources define transport and provider evidence fields. They do not prescribe your retention period, sampling policy, fallback threshold, or CometAPI routing rule; those are local contracts that must be tested and documented.

Contract details to verify

Ingress validation. Decide what happens when traceparent is absent, malformed, sampled off, or supplied by an untrusted caller. A safe default is to create a new root for malformed input and to accept remote context only at a trusted edge. Preserve only the headers and members your tracing library understands. Do not place account names, prompts, or free-form error messages in tracestate.

Span hierarchy. Define a stable operation span name and a deterministic attempt attribute. A retry should be a sibling of the failed attempt, not a child of a child that makes duration and ownership ambiguous. If a queue or worker performs the fallback, serialize the trace context through the queue envelope and start a worker span when it resumes.

Provider correlation. Confirm how each adapter reads a response request ID, how it represents a transport failure with no response, and how long the ID remains useful to support. OpenAI’s reference calls the header x-request-id; Anthropic’s error contract exposes request_id in the body and a request-id header. Normalize both into a field such as provider_request_id while retaining the provider name and original field name.

Outcome vocabulary. Keep success, timeout, rate_limited, server_error, client_error, canceled, and unknown distinct. Add retryable and fallback_selected as booleans decided by policy. Do not infer success from HTTP 200 alone: Anthropic notes that an SSE stream can report an error after a 200 response. The attempt should stay open until the stream’s terminal event or a local cancellation is recorded.

Header budget and privacy. OpenAI’s reference says request headers have a total size budget and that an oversized request may not receive an x-request-id. Keep propagation headers lean, reject unexpectedly large tracestate, and redact user-controlled values before export. Sampled traces still need a local correlation strategy in logs; unsampled does not mean “log the payload.”

A concrete operator workflow looks like this:

on ingress:
  context = extract_and_validate_w3c(request.headers)
  operation = start_span("llm.request", context)

for route in [primary, bounded_retry, cometapi_fallback]:
  attempt = start_child(operation, "llm.attempt", route=route.name)
  inject_context(attempt, outbound_headers)
  result = call_provider(route)
  capture_provider_request_id(result.headers, result.error_body)
  record_outcome(attempt, result.class, result.http_status)
  if result.success:
    end(attempt)
    end(operation, status="ok", fallback_used=(route.name == "cometapi_fallback"))
    return result.safe_output
  end(attempt)
  if not policy_allows_next_attempt(result.class):
    break

end(operation, status="error")
return safe_degraded_response()

On the happy path, an operator sees one operation span, one primary attempt, a provider request ID, and a terminal success. On the error path, the first attempt records its timeout or provider error, the fallback appears as the next sibling span with the same trace ID, and the final result states whether fallback succeeded. If the network fails before any response, provider_request_id is null and attempt_id remains the local evidence handle; do not invent a provider ID.

Failure modes

  1. A new root is created for every retry. The trace viewer shows unrelated requests, so the operator cannot see why fallback happened. Create siblings under the original operation and add an explicit fallback.selected event.
  2. A provider request ID replaces the trace ID. Provider IDs have provider-specific scope and format. Keep them as attributes and index fields; retain W3C trace and span identifiers as the causal keys.
  3. Malformed or attacker-supplied context is trusted. This can join unrelated work or leak vendor state. Validate at the edge, apply a trust boundary, and start a fresh root when validation fails.
  4. A queue drops context. The fallback worker looks like a separate service incident. Put a bounded, serialized trace context in the queue envelope and create a worker span on receipt; never put message content in that envelope merely for correlation.
  5. A 200 streaming response is treated as complete. Anthropic documents errors that arrive inside an SSE stream after the HTTP response begins. Keep the span open through terminal handling and mark a mid-stream error distinctly.
  6. No request ID is available after a transport failure. There may be no response headers to inspect. Preserve the local attempt ID, route, timing, and error class, and tell support that the provider ID was unavailable rather than fabricating one.
  7. Header growth prevents correlation. Long tracestate values, baggage, or copied metadata can exceed an upstream header budget. Enforce a size limit and allow-list propagated keys.
  8. Sensitive fields enter telemetry. Prompts, completions, account identifiers, and authentication material can escape through attributes or error strings. Use fixed enums, redaction, and field-level export policies; inspect sampled logs before enabling a new adapter.

FAQ

Should a provider switch start a new trace? Usually no when it is still the same user operation. Keep the original operation trace and create a new child attempt. If a trust boundary requires a separate trace, retain a non-sensitive local correlation reference and document the break so operators do not mistake it for a missing span.

Should we send traceparent to every model provider? Only when the provider contract, privacy policy, and intermediary behavior permit it. You can preserve complete local causality without forwarding internal headers to an external service. Always propagate context between components you control where the policy allows it.

What do we do when the provider has no request ID? Store provider_request_id: null, keep the local attempt ID, and record whether a response was received. A missing provider ID is a fact about the failure path, not a reason to reuse a different request’s identifier.

How much should the log contain? Enough to join the operation, attempt, route, provider, model alias, outcome, status, duration, retry decision, and provider request ID. It should contain no prompt, completion, raw headers, credential material, or unbounded vendor message.

How do we prove the contract works? Exercise a primary success, a retryable timeout followed by CometAPI success, a transport failure with no provider ID, and a mid-stream error. For each case, verify one trace, sibling attempt spans, bounded fields, and a terminal operation status. Keep the resulting timeline with the incident timeline evidence checklist .

Reader next step

Add the contract to the gateway middleware before changing routing thresholds. Start with one operation span and one attempt span, then instrument the CometAPI adapter as a sibling attempt. Run the four fault cases in a staging environment, inspect that traceparent remains continuous, and confirm that OpenAI or Anthropic request IDs are searchable without exposing content. Finally, hand the sanitized field list and the trace screenshots to the on-call owner, and link the fallback decision record to the same operation ID. This makes the next provider incident answerable with evidence rather than guesswork.