Normalize Provider Stop Reasons Before CometAPI Fallback

Last reviewed: 2026-08-03

Direct answer

Normalize each provider’s stop or finish reason before making a CometAPI fallback decision. An HTTP success, a nonempty response, or the end of a stream does not by itself tell you whether the user received a complete answer. The terminal metadata might instead describe a token limit, a tool handoff, a content restriction, a malformed function call, or a state that requires continuation.

A reliable adapter should preserve the native reason and map it into a small application-owned state set: complete, continuation_required, incomplete_limit, policy_terminal, contract_failure, admission_failure, or unknown. The router then combines that normalized state with output validation, bytes already delivered, the user-action scope, and the remaining attempt budget.

That separation is the core of LLM API stop reason normalization. It prevents three dangerous shortcuts: treating every HTTP 200 response as complete, treating every non-natural stop as retryable, and letting an unfamiliar value inherit an unsafe default. Fallback should be an explicit action allowed by the normalized state, not a side effect of a provider-specific string comparison.

Who this is for

This design is for platform engineers, SREs, and backend developers responsible for multi-model applications, especially teams that place routing and fallback logic around CometAPI-backed requests. It is also useful to application owners who need stable behavior while underlying model families expose different completion metadata.

The stop-reason adapter sits between transport parsing and product behavior. It complements, but does not replace, output-schema validation, tool execution controls, safety policy, or attempt limits. Teams still defining how partial output affects retries should review the partial-success classification guide alongside this contract.

Key takeaways

• Preserve both the native provider reason and the normalized application state. The raw value is evidence for debugging and future mapping changes.
• Treat natural completion, tool continuation, output limits, content restrictions, and malformed output as different states with different actions.
• Do not use fallback to route around a safety refusal or another terminal content restriction.
• Do not interpret a tool request as a provider failure. It belongs to the same user action and usually requires controlled continuation.
• Do not splice a fallback response onto text already delivered from another route.
• Make unknown values non-retryable until an owner reviews and maps them.
• Apply output and schema validation after normalization. A natural stop can still produce an unusable result.

Sources checked

The OpenAI Chat Completions API reference documents finish_reason states that distinguish natural stops, length limits, content filtering, and tool calls. It also distinguishes a completed response object from streamed completion chunks, which matters because a router must not decide from an intermediate chunk.

The Anthropic Messages API reference defines stop_reason values including end_turn, max_tokens, stop_sequence, tool_use, pause_turn, refusal, and model_context_window_exceeded. Those values show why a single generic success-or-failure flag cannot represent every terminal condition.

The Google Gemini GenerateContent reference defines finishReason values including STOP, MAX_TOKENS, SAFETY, RECITATION, MALFORMED_FUNCTION_CALL, UNEXPECTED_TOOL_CALL, and TOO_MANY_TOOL_CALLS. That range supports a normalization layer that separates content restrictions, capacity limits, and tool-contract failures.

Contract details to verify

Define an application-owned state table

Start with a mapping table that names the default action but still permits application-specific validation:

The mapping is a starting policy, not proof that the response is usable. For example, an expected stop sequence may be complete for one workflow but premature for another. Likewise, a natural stop does not prove that required JSON fields, citations, or tool arguments are present.

Preserve sanitized decision evidence

Log enough data to reconstruct the decision without storing prompts, generated text, tool arguments, request headers, or direct user identifiers. A sanitized record can look like this:

{
  "request_id": "req-042",
  "action_scope": "action-17",
  "route_id": "chat-primary",
  "provider_family": "provider-a",
  "model_alias": "general-chat",
  "http_status": 200,
  "stream_started": true,
  "bytes_emitted": 0,
  "native_reason": "max_tokens",
  "normalized_state": "incomplete_limit",
  "schema_valid": false,
  "selected_action": "repair",
  "fallback_attempt": 0,
  "latency_ms": 1840,
  "input_units": 730,
  "output_units": 612,
  "policy_class": "none"
}

At minimum, retain the request and user-action correlation IDs, route and model aliases, HTTP status, stream state, emitted-byte count, raw reason, normalized state, validation result, chosen action, attempt number, latency, usage counters, and a non-sensitive policy classification. The fallback decision logging guide provides a broader decision-record pattern.

Happy-path operator workflow

1. Assign the request to a user-action scope and record its deadline and attempt budget.
2. Verify that the selected route supports the required response format, tools, and context size.
3. Dispatch the request and collect transport status, stream progress, and provider-native terminal metadata.
4. Wait for a terminal marker. Do not classify an intermediate streaming value as success or failure.
5. Map the native reason to the application-owned state while preserving the original value.
6. Run the required content, schema, and tool-contract validators.
7. When the state is complete, validation passes, and no conflicting partial response has been delivered, return the response and close the action with selected_action=accept.
8. Emit the sanitized decision record and update metrics for native reasons, normalized states, and selected actions.

A concrete happy path is an HTTP 200 response with a native natural-stop reason, no prior partial delivery conflict, and a valid application response. The adapter maps it to complete; the validator passes; the gateway returns it without spending a fallback attempt.

Error-path operator workflow

1. Stop delivery while classifying a limit, malformed output, restriction, continuation request, or unknown reason.
2. If bytes have already reached the client, mark the response as exposed partial output. Do not append a second provider’s answer to it.
3. For policy_terminal, return the approved restricted response. Do not provider-shop for a different answer.
4. For continuation_required, validate the requested tool or continuation and keep it bound to the same user action. It is not an ordinary failover.
5. For incomplete_limit, inspect whether the output is repairable, whether any output was exposed, and whether the remaining deadline and spend permit another action. Prefer a bounded continuation or local repair when the contract allows it.
6. For contract_failure, use fallback only if the alternate route supports the same output and tool contract. Otherwise return a controlled degradation.
7. For admission_failure, change the request before retrying. Sending the same oversized input to another route is not a repair.
8. For unknown, fail closed, preserve the raw value, alert the route owner, and add a test before enabling any new mapping.
9. Record the final action and increment the user-action attempt counter, not merely a per-request counter.

Failure modes

Treating HTTP success as product success

A route can return HTTP 200 while reporting an output limit or content restriction. If the gateway ignores terminal metadata, it may send truncated prose, incomplete JSON, or a missing tool result downstream as though the task finished.

Retrying every non-natural stop

A blanket retry turns tool handoffs, pauses, and restrictions into traffic amplification. It can also duplicate side effects when the application treats a tool request as a failed model call. Keep tool continuation semantics stable using the tool-call fallback contract .

Routing around content restrictions

A policy or content-restriction outcome is not an availability incident. Automatically asking another provider for the same blocked content can defeat the application’s intended controls and produce inconsistent user behavior.

Splicing two streamed answers

Once text from the first route reaches the client, a second route cannot safely pretend to be a continuation unless the application has a deliberate, tested reconciliation protocol. The likely result is duplicated sentences, contradictory answers, invalid structured output, or mismatched tool state.

Losing the native reason

Logging only contract_failure or incomplete_limit hides whether the provider reported a new enum, a known limit, or a malformed function call. Without the raw value, operators cannot distinguish mapping drift from a real workload change.

Defaulting unknown values to complete

Providers can add terminal states. A permissive default silently accepts behavior the application has never tested. The safer default is unknown, no automatic fallback, and an alert tied to the route and model alias.

Applying a mapping without output validation

A natural stop can still yield invalid JSON, absent required fields, or tool arguments that violate the application schema. Stop-reason normalization selects the next decision branch; it does not replace response validation.

Letting attempts escape the user action

If every route maintains its own retry counter, one user action can accumulate multiple repairs and fallbacks. Enforce the shared boundary described in the fallback attempt limit guide .

FAQ

Is the HTTP status enough to decide whether to fall back?

No. HTTP status describes the transport-level result. The application also needs the provider-native terminal reason, normalized state, validation result, stream exposure state, and remaining user-action budget.

Should every token-limit response go to another provider?

No. First determine whether the response is already sufficient, can be continued safely, or can be repaired locally. A fallback may be reasonable when no conflicting output has been delivered, the alternate route supports the same contract, and the action still has time and spend available.

Can a safety or refusal result trigger fallback?

Not as a way to obtain content another route restricted. Map it to a terminal content-policy state and follow the application’s approved response behavior. Availability fallback and policy enforcement need separate decision branches.

How should tool calls be normalized?

Expected tool calls should become continuation_required, not failed. Validate the tool name, arguments, authorization within the application, and side-effect controls before continuing. Unexpected or malformed tool calls belong in contract_failure.

What should happen when a provider adds a new reason?

Map it to unknown, preserve the raw value, stop automatic fallback, and alert the route owner. Review the provider contract, add fixtures for happy and error paths, then deploy the explicit mapping in shadow mode before it controls traffic.

Does streaming need a different state model?

The state names can remain the same, but classification must wait for terminal metadata. Intermediate chunks can have no final reason. Track whether bytes were emitted separately because client-visible partial output changes which repairs are safe.

Reader next step

Build a fixture matrix for every raw reason your active routes can return. For each fixture, specify the normalized state, validator outcome, expected action, whether bytes were already delivered, and whether another attempt is permitted. Include at least one unknown-value fixture and one stream that ends after partial output.

Run the adapter in shadow mode first. Compare its proposed action with current production behavior, review mismatches, and alert on unmapped values. Once the mapping is stable, make it the only path that can authorize repair or fallback.

For a unified route to evaluate with this contract, Start with CometAPI . Apply the state table, validation gates, and user-action attempt budget before enabling automatic fallback.