Last reviewed: 2026-06-24
Direct answer
A useful LLM API fallback checklist starts with the public request contract, then adds retry safety and HTTP telemetry checks. For CometAPI chat calls, verify the documented chat completion route, authentication setup, message shape, streaming choice, and expected response fields in the official reference before the fallback path is trusted. Then test retry behavior separately with bounded backoff, and record HTTP status, error type, retry count, and fallback decision fields in a low-cardinality log record.
Use this with related evidence from Retry and Backoff Evidence for CometAPI Gateway Calls when you need a narrower retry review.
Smoke-test workflow:
- Setup assumptions: the caller has a non-production CometAPI credential stored outside source control as
<API_KEY_PLACEHOLDER>, a chosen model approved by the operator’s own account, and a test prompt that does not include sensitive data. - Happy-path request plan: send one minimal chat completion request to the documented chat completion endpoint using the documented base URL and message array shape. Record whether the response has a completion object, at least one choice, a finish reason, and usage metadata when present.
- Error-path check: run a controlled request with an intentionally invalid credential placeholder in a non-production environment and confirm the client records the HTTP status and stops before fallback promotion.
- Minimum assertions: the primary request returns a handled result or handled error, retry count stays within the local runbook limit, the fallback decision is logged, and the incident record links to the exact docs reviewed.
- What not to assert: do not treat one successful smoke test as evidence of uptime, latency, account quota, model availability, billing behavior, or provider-specific parameter support.
Sanitized log-record template:
{
"checked_at": "2026-06-24T00:00:00Z",
"route_family": "chat_completions",
"http_status": "placeholder_status",
"error_type": "placeholder_error_or_none",
"retry_count": "placeholder_integer",
"fallback_decision": "primary_used_or_fallback_considered",
"docs_reviewed": ["https://apidoc.cometapi.com/api/text/chat"],
"operator_notes": "placeholder_summary"
}
Who this is for
This checklist is for engineers who own LLM gateway reliability, on-call runbooks, and fallback promotion decisions. It is most useful when an application already has a primary LLM API path and needs evidence before adding or changing fallback behavior.
Key takeaways
- Verify the CometAPI chat request and response contract from the official chat completion reference before writing assertions.
- Keep retry behavior bounded and use backoff only for failures your application treats as transient.
- Instrument HTTP client activity with stable attributes so reliability reviews can compare primary and fallback paths.
- Escalation evidence should include clean request metadata, response status, and the public documentation links reviewed.
- Avoid claims about price, quota, uptime, latency, or model availability unless a current source directly supports that exact claim.
Failure modes
- Evidence gap: the agent cannot inspect the failing log, source page, pull request, or local command output. The safe action is to stop and record the missing evidence instead of guessing.
- Scope drift: the agent edits files that are not connected to the observed failure. Keep the repair tied to the failing signal and leave unrelated cleanup for a separate task.
- Environment mismatch: the local check uses different versions, credentials, feature flags, or runtime settings than the hosted path. Record the mismatch before treating the result as proof.
- Unreviewed fallback: the agent changes models, endpoints, permissions, or retry behavior to make a run pass without preserving the review boundary. Treat access and provider failures as operational blockers, not topic failures.
- Weak handoff: the final note says the issue is fixed but omits the command, result, changed files, and remaining uncertainty. That makes the next operator repeat the investigation.
Sources checked
- CometAPI chat completions reference - accessed 2026-06-24; purpose: verify chat completion contract areas.
- CometAPI help center - accessed 2026-06-24; purpose: verify support and escalation documentation areas.
- CometAPI documentation - accessed 2026-06-24; purpose: verify current CometAPI documentation navigation.
- AWS retry with backoff pattern - accessed 2026-06-24; purpose: verify retry and backoff guidance.
- OpenTelemetry HTTP semantic conventions - accessed 2026-06-24; purpose: verify HTTP telemetry field context.
Contract details to verify
| Area | What to verify | Source URL | Accessed | Safe candidate wording |
|---|---|---|---|---|
| Chat completion route | Confirm the documented route, base URL, message array shape, and response areas before coding assertions. | https://apidoc.cometapi.com/api/text/chat | 2026-06-24 | “Verify the current chat completion request and response contract in the official CometAPI reference.” |
| Streaming behavior | Confirm whether the request under test uses streaming and what final event or usage handling the client expects. | https://apidoc.cometapi.com/api/text/chat | 2026-06-24 | “Streaming checks should follow the official event format for the selected request path.” |
| Error handling | Confirm the documented error examples and record handled failure status without assuming account-specific limits. | https://apidoc.cometapi.com/api/text/chat | 2026-06-24 | “Record the observed status and error type; do not infer broader service behavior from one failure.” |
| Support escalation | Confirm the current support path and attach clean, sanitized evidence when escalating. | https://apidoc.cometapi.com/support/help-center | 2026-06-24 | “Use the current help center for support and account-specific questions.” |
| Retry safety | Confirm retry conditions are transient, bounded, and observable before enabling fallback promotion. | https://docs.aws.amazon.com/prescriptive-guidance/latest/cloud-design-patterns/retry-backoff.html | 2026-06-24 | “Use bounded backoff for transient failures, and avoid retry behavior that can amplify overload.” |
| HTTP telemetry | Confirm HTTP client spans and metrics use stable, reviewable fields. | https://opentelemetry.io/docs/specs/semconv/http/ | 2026-06-24 | “Record HTTP method, route family, status, error type, and retry count with low-cardinality values.” |
Reader next step
Compare the workflow against Start with CometAPI .
FAQ
Can this checklist prove an LLM provider is highly available?
No. It can show that one request path, fallback decision, and logging plan were checked against public references. Availability claims need separate operational evidence.
Should the fallback smoke test assert exact model behavior?
No. The safer assertion is that the client handled the response or error according to the documented request contract and the local runbook. Provider-specific behavior should be verified from the provider documentation for the selected model.
What should be logged after a failed primary request?
Log the request family, HTTP status, error type, retry count, fallback decision, and source links reviewed. Do not log credentials, sensitive prompts, or full generated responses.
When should retries stop?
Retries should stop according to the local retry budget and failure classification. The public retry source supports bounded backoff for transient faults, but the exact retry count is an application decision.