Chat completions fallback runbook for CometAPI

Last reviewed: 2026-05-09

Who this is for: on-call engineers, platform owners, and application operators who run user-facing chat completion workloads through CometAPI and need a practical fallback decision process.

This runbook treats fallback as an operator-owned reliability control. It does not assume that every error should be routed away. Some failures should trigger a retry, some should page the owning team, and some should be stopped because fallback would hide a bad request, expired credential, or unsupported contract assumption.

For broader reliability patterns, keep this page connected to the site index at /sites/llm-api-reliability/ and related operational drafts under /sites/llm-api-reliability/posts/.

Key takeaways

• Use fallback only after classifying the failure signal: transport, provider response, rate-limit/quota, output contract, latency, or client configuration.
• Do not fallback on deterministic client errors such as malformed requests, invalid model identifiers, missing authentication, or schema bugs.
• Keep the CometAPI endpoint path, auth format, request fields, response fields, and error shape tied to the current public documentation, especially the CometAPI API docs and chat completions reference.
• Tune thresholds to your own SLO, traffic shape, and cost controls. The numeric examples below are starting points for drills, not universal rules.
• Validate fallback with injected failures before relying on it during an incident.

Concise definition

A CometAPI chat completions fallback is an application-side policy that decides what to do when a primary chat completion request path is unhealthy.

In this runbook, fallback can mean:

• retrying the same request after a short backoff;
• switching to a secondary configured route;
• returning a degraded but explicit response;
• queueing the request for later processing;
• stopping the request and paging an operator.

It does not mean blindly replaying every failed request through another model or provider.

Evidence basis

The public CometAPI documentation landing page is the first source to check for current API navigation and contract references: https://apidoc.cometapi.com/. The chat completions endpoint reference should be treated as the source of truth for the request and response contract: https://apidoc.cometapi.com/api-13851472. For account, support, and operational questions that are not defined in the endpoint reference, check the CometAPI help center: https://apidoc.cometapi.com/help-center.

Contract details to verify

Before enabling automated fallback, record the exact contract your production client depends on. This avoids a common failure mode: the fallback layer works, but it is protecting an integration that was never aligned with the documented API.

Monitoring signals checklist

Use this checklist to classify the signal before choosing a fallback action.

Decision rules for on-call use

Use these rules during an incident:

1. Classify the failure first. Determine whether the dominant signal is transport, 5xx, 429, 4xx validation, auth, latency, output contract, or cost amplification.
2. Retry only when the request is safe to replay. If the request can trigger external side effects in your application, deduplicate before retrying.
3. Fallback only when the alternate route has been pre-approved. It must have compatible request fields, response parsing, safety behavior, data handling, and budget controls.
4. Stop on auth and malformed request failures. Fallback on 401, 403, or repeated deterministic 400 errors usually hides the real issue.
5. Log the decision, not just the error. Each event should say whether the system retried, fell back, degraded, queued, or failed closed.

Minimal sanitized request example

Use this as a shape check, not as a contract guarantee. Confirm the path, required fields, and supported parameters in the CometAPI chat completions reference before production.

curl -X POST "https://<COMETAPI_BASE_URL>/v1/chat/completions" \
  -H "Authorization: Bearer ${COMETAPI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id-approved-for-this-workload>",
    "messages": [
      {
        "role": "system",
        "content": "Answer with concise operational guidance."
      },
      {
        "role": "user",
        "content": "Summarize the current incident state from these sanitized symptoms: elevated 5xx and p95 latency."
      }
    ],
    "temperature": 0.2,
    "max_tokens": 300,
    "stream": false
  }'

For production, attach your own request correlation identifier in application logs. Do not assume an extra API header is accepted unless the endpoint documentation explicitly supports it.

Practical validation steps

1. Build a contract fixture

Create a fixture with one approved request per workload class:

• normal short chat;
• long-context request near your expected token budget;
• structured-output request if your application requires JSON;
• streaming request if you use streaming;
• blocked or rejected request if your application has policy gates.

For each fixture, store:

• request body shape;
• expected required response fields;
• parser expectations;
• timeout budget;
• retry eligibility;
• fallback eligibility.

2. Run failure injection in staging

Inject one failure mode at a time:

• DNS or connection failure;
• upstream timeout;
• HTTP 500;
• HTTP 429;
• HTTP 400 malformed request;
• HTTP 401 invalid credential;
• malformed successful response;
• delayed streaming chunks.

Expected outcome:

• retryable transport and 5xx failures may retry and then fallback;
• 400, 401, and 403 failures should not fallback automatically;
• malformed success responses should be counted as contract failures;
• cost and retry amplification should be visible in dashboards.

3. Validate observability fields

Every chat completion attempt should emit structured telemetry. At minimum, capture:

• route name, not secret URL;
• endpoint path;
• workload class;
• model identifier used by your configuration;
• HTTP status code;
• timeout phase if available;
• retry attempt number;
• fallback decision;
• latency and time to first token if streaming;
• response parser result;
• token usage if provided by the response and documented for the endpoint;
• sanitized error category.

Avoid logging prompts, secrets, full user data, or raw authorization headers.

4. Drill the on-call decision

Run a 30-minute drill with the application owner and on-call engineer:

1. Force 5xx for a small staging traffic slice.
2. Confirm retry volume stays inside the configured cap.
3. Confirm fallback begins only after the configured trigger.
4. Confirm fallback responses pass the same parser.
5. Confirm dashboards separate primary, retry, and fallback traffic.
6. Confirm the runbook tells the operator when to disable fallback.

Record the result in your internal incident-readiness notes. If this site later publishes a validation template, link it from /sites/llm-api-reliability/editorial/.

Suggested alert routing

FAQ

Should every failed chat completion request fallback automatically?

No. Automatic fallback is safest for transient transport failures, selected 5xx responses, and latency events where the alternate route is already validated. It is usually unsafe for malformed requests, auth failures, unsupported parameters, or schema bugs.

Is a retry the same as fallback?

No. A retry repeats the request against the same route, usually after a short delay. Fallback changes behavior: a secondary route, degraded response, queue, or fail-closed path. Track them separately.

What is the most important metric?

There is no single metric. Combine status code rate, timeout rate, latency, parser failure rate, fallback volume, and user-visible failure rate. A low HTTP error rate can still hide malformed successful responses.

Should 429 always trigger fallback?

Not always. A 429-style signal may indicate rate limiting, quota, or account-level constraints. Fallback can be appropriate only if the secondary route has approved capacity, compatible behavior, and cost controls.

Can fallback hide product bugs?

Yes. If the client sends an invalid request or parses the response incorrectly, fallback may make the incident harder to diagnose. That is why this runbook separates deterministic 4xx and parser failures from retryable transport failures.

How often should this runbook be reviewed?

Review it whenever the CometAPI endpoint documentation changes, when you add a new model route, after any production incident, and at least once per quarter for high-traffic workloads.

Sources checked