Fallback Decision Logs for CometAPI Gateway Calls

Last reviewed: 2026-06-05

Direct answer

When a CometAPI gateway call fails or degrades, your application makes a routing decision: retry the same endpoint, fall back to a secondary path, or surface an error. A fallback decision log is the structured record of that choice — capturing enough context to replay the decision, diagnose recurring failure patterns, and tune thresholds over time.

The core structure of a useful fallback decision log is:

1. Trigger — which HTTP status code, timeout signal, or error class caused the decision.
2. Decision — what the client chose to do (retry, fallback, abort, degrade gracefully).
3. Outcome — whether the follow-up call succeeded or compounded the failure.
4. Context — request metadata (endpoint path, elapsed ms, attempt number) that scopes the record without including sensitive content.

CometAPI exposes two primary text-generation endpoints — the chat completions path (/api/text/chat) and the responses path (/api/text/responses) — each with its own request and response shape. Verify the exact field names, error codes, and retry-relevant response properties directly in the CometAPI API reference before hard-coding field mappings in your logger.

Who this is for

This article is for backend engineers and platform teams who:

• Route traffic through CometAPI as a gateway layer for LLM calls.
• Need structured, queryable logs to diagnose fallback storms, cascading retries, or silent degradation.
• Are building or auditing retry/fallback logic and want a reference log schema to start from.
• Have already read the Timeout-budget fallback checks for chat completions and want to add durable observability on top.

It assumes you can make HTTP calls to CometAPI, emit structured logs from your application layer, and interpret HTTP status codes and timing signals.

Key takeaways

• A fallback decision log captures trigger, decision, outcome, and context — nothing more is needed for useful pattern analysis.
• The CometAPI chat completions and responses endpoints have distinct request shapes; log the endpoint path used, not just a generic “LLM call” label, so you can correlate failures per endpoint.
• Retry-explosion risk is real: log the attempt number and enforce a cap. The Google SRE book’s overload chapter is the canonical reference for why unbounded retries worsen outages.
• HTTP 429, 503, and timeout signals are the primary triggers for a fallback decision. Log the exact status code or error class — do not coerce them into a single “error” bucket.
• Exponential backoff with jitter reduces retry collisions. Log the computed wait duration alongside the attempt number so you can audit whether your backoff logic is working as intended. The AWS retry-backoff prescriptive guidance is a useful external reference for implementing this pattern.
• Keep log records small: placeholder IDs, elapsed milliseconds, attempt counts, and decision codes. Do not log prompt content, completion text, or authentication material.
• Verify exact error codes, request fields, and response field names in the CometAPI docs before finalising your field mappings — the body of this article uses safe descriptive names, not hard-coded API identifiers.
• The OpenTelemetry HTTP semantic conventions define standard span attributes that complement your fallback log without duplicating low-level HTTP trace data.

Designing the log schema

A practical fallback decision log record needs to answer four questions:

What triggered the decision?

• The HTTP status code returned (or the timeout/connection error class if no status was received).
• Whether the response body contained a machine-readable error indicator — verify the exact field name in the CometAPI chat reference.
• The elapsed time in milliseconds at the point the decision was made.

What did the client decide? A small controlled vocabulary keeps logs queryable. Suggested decision codes:

Choose codes that match your architecture. The important constraint is that every possible outcome of your fallback logic has exactly one code.

What happened next?

• The HTTP status of the follow-up call (or null for abort_* decisions).
• Whether the ultimate outcome was success, continued failure, or a degraded response.

What is the safe context?

• A correlation ID or request ID (opaque string, not content).
• The endpoint path: /api/text/chat or /api/text/responses — verify current paths in the CometAPI docs.
• Attempt number (1-indexed).
• Whether streaming was in use — streaming failures surface differently and are worth segmenting.

What to leave out

Do not include in fallback logs:

• Prompt text or completion content.
• Authentication tokens or API keys.
• Full request or response bodies.
• Any field whose value could change per-user in a way that creates high cardinality without diagnostic value.

High-cardinality fields that are safe to include: elapsed_ms (numeric), attempt_count (integer), http_status_code (integer or null).

Sample log record template

The following template uses placeholder values only. Replace field names and values with those confirmed in your CometAPI integration and the official docs.

{
  "schema": "fallback-decision-log/v1",
  "ts": "YYYY-MM-DDTHH:MM:SSZ",
  "correlation_id": "<opaque-request-id>",
  "endpoint_path": "/api/text/ENDPOINT",
  "attempt_number": 1,
  "trigger": {
    "type": "http_status | timeout | connection_error",
    "http_status": null,
    "elapsed_ms": 0
  },
  "decision": "retry_same | fallback_secondary | degrade_cached | abort_error | abort_timeout_budget",
  "backoff_wait_ms": 0,
  "followup_http_status": null,
  "outcome": "success | continued_failure | degraded",
  "streaming": false
}

This template deliberately omits model identifiers, pricing fields, and quota counters. Those are account-specific commercial details; log them separately in a billing or quota context if needed, not in your per-call fallback decision log.

Smoke-test workflow

This workflow lets you verify that your fallback decision logger is emitting correct records before you rely on it in production.

Setup assumptions:

• You have a valid CometAPI API key.
• Your application can reach the CometAPI chat completions endpoint (or the responses endpoint).
• Your logger writes structured JSON to stdout or a log sink you can query.
• You have a way to simulate a non-200 response — either a test environment that returns a forced error, or a request with a deliberately malformed payload that triggers a 4xx.

Happy-path check:

1. Send a minimal well-formed request to the chat completions endpoint.
2. Assert the response HTTP status is 2xx.
3. Assert your fallback logger emits no fallback decision record (a successful call should not generate a fallback log entry).
4. Assert the correlation ID in your application request context matches the correlation ID in any debug-level log emitted during the call.

Error-path check:

1. Send a request designed to trigger a non-2xx response (for example: a 429 by exceeding your test rate allowance, or a 4xx via a bad request shape).
2. Assert that your fallback decision logger emits exactly one record.
3. Assert the record contains: trigger.http_status matching the actual status code received, attempt_number equal to 1 for the first attempt, decision set to one of your controlled vocabulary codes, and ts set to a recent timestamp.
4. If your logic retries, assert that the second attempt increments attempt_number to 2 and that backoff_wait_ms is greater than 0.

Minimum assertions:

• One log record per fallback decision event (not per call).
• decision is always one of the controlled vocabulary codes.
• No prompt content or authentication material appears in any field.
• endpoint_path matches the path actually called — verify current paths in the CometAPI chat reference.

What the smoke test must not assert:

• Specific model availability or model identifiers (these change and are not within the scope of fallback decision logging).
• Specific uptime or latency targets.
• Specific pricing or quota values.
• The exact wording of error messages in response bodies — these may change without notice.

Pass/fail log fields to record after the smoke test:

smoke_test_run_id: <your-run-id>
date: YYYY-MM-DD
endpoint_tested: /api/text/ENDPOINT
happy_path_no_log_emitted: true | false
error_path_log_emitted: true | false
decision_code_valid: true | false
attempt_number_increments: true | false | not_tested
prompt_content_in_log: false  # must always be false
overall: pass | fail
notes: ""

Failure modes

These are the operational failure modes specific to fallback decision logging for CometAPI gateway calls. Each represents a real way the logging layer itself can fail or produce misleading output.

• Silent non-emission on 5xx: Your fallback handler catches the exception but the branch that writes the log record is behind a condition that evaluates false on certain 5xx codes (e.g., 502 from an upstream proxy rather than 503 from CometAPI directly). The result is a fallback that executes but leaves no log record — the decision is invisible to your diagnostic tooling. Verify that your trigger condition covers connection-level errors and gateway errors, not just the status codes you expect from CometAPI itself.

• Attempt-number reset on retry loop restart: When a retry loop is restarted by a higher-level supervisor (e.g., a queue worker that re-enqueues a failed job), attempt_number resets to 1 in the new execution context. This makes the log look like a first attempt when it is actually the third or fourth total try. Record a parent_correlation_id or a requeue_count field if your architecture uses job queues around the CometAPI call.

• Decision code coercion masking the real trigger: If your fallback logic maps multiple distinct triggers (429 rate-limit, 503 overload, connection timeout) to a single abort_error code without preserving the trigger type, you lose the ability to distinguish retry-safe failures from terminal ones. The Google SRE book’s overload chapter notes that treating all errors as equivalent under load is a primary cause of retry storms — the log schema must preserve the distinction.

• Backoff-wait field always zero: The backoff_wait_ms field is populated after the wait is computed but before it is actually applied. If the wait is skipped due to a race condition or a misconfigured flag, the field reads 0 but the client retried immediately. Cross-check backoff_wait_ms > 0 against ts delta between consecutive records for the same correlation_id.

• Streaming error not captured: Streaming responses from /api/text/chat may fail mid-stream after an HTTP 200 is received. Because the status code is 200, a trigger condition checking only http_status != 200 will not fire. Streaming failures require a separate trigger path that watches for stream interruption signals. Verify the error surface for streaming in the CometAPI chat reference and add a trigger.type: stream_interrupted code to your vocabulary.

• High-cardinality prompt hash leaking into correlation_id: Some implementations hash prompt content and use it as part of the correlation ID to enable deduplication. This is a PII/content-leakage risk and inflates log cardinality. Use an opaque random ID generated at request start; do not derive it from request content.

Sources checked

• CometAPI documentation - accessed 2026-06-05; purpose: verify current CometAPI documentation navigation.
• CometAPI chat completions reference - accessed 2026-06-05; purpose: verify chat completion contract areas.
• CometAPI responses reference - accessed 2026-06-05; purpose: verify responses endpoint contract areas.
• CometAPI models overview - accessed 2026-06-05; purpose: verify model catalog discovery guidance.
• CometAPI help center - accessed 2026-06-05; purpose: verify support and escalation documentation areas.

Contract details to verify

The following areas require direct verification in the linked CometAPI docs before you finalise your fallback decision log field mappings. Safe candidate wording is provided for use until you have confirmed the exact values.

FAQ

Should I log every API call or only calls that trigger a fallback decision? Log only the calls that trigger a fallback decision. Logging every successful call in a fallback-decision schema creates noise and high write volume with no diagnostic benefit. Use a separate request-level trace or metric for overall call volume.

What is the difference between the chat completions endpoint and the responses endpoint for fallback purposes? They have different request shapes and may have different timeout profiles. Your fallback decision log should record which endpoint was attempted so you can analyse whether failures cluster on one path. Verify the exact differences in the CometAPI API reference.

How many retries is safe before I should abort? The Google SRE book recommends keeping retry budgets small and using exponential backoff with jitter to avoid retry-explosion under load. A common starting point is 2-3 attempts with doubling wait times, but your specific timeout budget governs the practical ceiling. The AWS retry-backoff guidance is a useful external reference.

Can I use the same log schema for both the chat and responses endpoints? Yes, if you include endpoint_path as a field. The schema in this article is designed to be endpoint-agnostic; the endpoint_path field is what differentiates records across the two paths.

Should the fallback decision log include the model identifier used? It can if your fallback logic selects different models across attempts. Include it only as a reference value (e.g. model_id_attempt_1, model_id_attempt_2). Always verify current model identifiers against the CometAPI model catalog — model IDs change and stale identifiers cause silent fallback failures.

What OpenTelemetry fields map to fallback log fields? The OpenTelemetry HTTP semantic conventions define standard span attributes for http.response.status_code, http.request.method, and URL-related fields. Your fallback decision log can complement an OTel HTTP span rather than replace it — the span carries the low-level HTTP trace; the fallback log carries the application-level routing decision.

Where can I get started with CometAPI? See the CometAPI documentation home for the full API reference. To start using the gateway, visit CometAPI.

Reader next step

Run the next implementation or review pass against Timeout-budget fallback checks for chat completions, then keep Home nearby for the surrounding editorial and source boundary.

After the source checks, request assumptions, and review owner are clear, use CometAPI as the reference gateway only for the request paths, model routes, or cost checks the team has actually verified.

Use Timeout-budget fallback checks for chat completions as the next comparison point. Keep Home nearby for setup and permission checks.

After the source checks, request assumptions, and review owner are clear, use CometAPI as the reference gateway only for the request paths, model routes, or cost checks the team has actually verified.