Build a CometAPI Fallback Evidence Checklist

Last reviewed: 2026-07-15

Direct answer

A fallback runbook needs a promotion decision for each contract layer: endpoint, authorization, request validation, response structure, usable content, and retry behavior. A bounded CometAPI validation used eight contract requests plus one authenticated model-catalog preflight. The Responses and non-streaming Chat paths produced usable 200 responses. Invalid authorization produced a structured 401. Missing required fields produced structured 400 errors. Two Chat streaming attempts completed their SSE envelopes but returned no content, so streaming remains a hold.

Do not collapse these observations into one green status. The non-streaming routes have evidence suitable for a controlled canary. The streaming route has a repeatable negative sample and should not be promoted until another bounded validation returns usable content.

Who this is for

This checklist is for on-call engineers, gateway owners, and reviewers preparing a fallback change. It is intentionally narrow: it records contract evidence without storing secrets, customer prompts, full generated responses, request IDs, or commercial data.

Key takeaways

• Responses with gpt-5-nano-2025-08-07 returned HTTP 200, object response, status completed, and passed the content assertion.
• Non-streaming Chat Completions with gpt-4.1-nano returned HTTP 200, object chat.completion, finish reason stop, and passed the content assertion.
• Invalid authorization returned HTTP 401 with error fields code, message, and type.
• Omitting messages or input returned HTTP 400 with code, message, param, and type; the observed code was invalid_request.
• Two Chat SSE attempts returned HTTP 200, three chunks, and [DONE], but zero content and finish_reason=length at a 64-token cap.
• A complete transport envelope is not a usable fallback result.

The sanitized eight-case contract evidence artifact records every result used below without credentials, raw request IDs, full prompts, or full responses.

Contract details to verify

Runbook setup

Before sending a request, record the intended endpoint family, selected model, maximum request count, secret source, owner, and stop condition. This validation executed serially with eight contract requests plus one authenticated catalog preflight. That made nine HTTP calls and exceeded the intended eight-request ceiling by one; the stop condition after discovery was no further CometAPI traffic. The evidence record retained only status classes, structural fields, assertion results, and model identifiers.

Run a minimal Responses request:

curl --fail-with-body --silent --show-error \
  https://api.cometapi.com/v1/responses \
  -H "Authorization: Bearer ${COMETAPI_KEY}" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-5-nano-2025-08-07",
    "input": "Reply exactly: OK",
    "max_output_tokens": 64
  }'

Run the separate Chat contract:

curl --fail-with-body --silent --show-error \
  https://api.cometapi.com/v1/chat/completions \
  -H "Authorization: Bearer ${COMETAPI_KEY}" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-4.1-nano",
    "messages": [{"role": "user", "content": "Reply exactly: OK"}],
    "max_completion_tokens": 64
  }'

Save a sanitized evidence packet

Keep the packet compact enough to review during an incident:

{
  "request_budget": {"total_calls": 8, "concurrency": 1},
  "responses_success": {
    "http_status": 200,
    "object": "response",
    "status": "completed",
    "content_assertion": "passed"
  },
  "chat_success": {
    "http_status": 200,
    "object": "chat.completion",
    "finish_reason": "stop",
    "content_assertion": "passed"
  },
  "streaming": {
    "attempts": 2,
    "http_status": 200,
    "chunks_per_attempt": 3,
    "done_seen": true,
    "content_characters": 0,
    "finish_reason": "length",
    "verdict": "hold"
  }
}

Do not add raw headers, request IDs, full prompts, full output, credentials, balances, or prices to this packet.

Verify safe failures

An invalid credential should be classified without activating fallback promotion:

curl --silent --show-error \
  https://api.cometapi.com/v1/chat/completions \
  -H 'Authorization: Bearer intentionally-invalid' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-5-nano-2025-08-07",
    "messages": [{"role": "user", "content": "Reply exactly: OK"}],
    "max_completion_tokens": 8
  }'

The observed 401 error contained code, message, and type. With the valid secret restored, omitting messages from Chat or input from Responses returned 400 with code=invalid_request and fields code, message, param, and type. These are non-retryable client corrections in this runbook; changing models or increasing retry count does not repair them.

Promotion decision table

Retry and rollback rules

• Never retry a 400 missing-field error; repair the request contract.
• Never route around a 401; repair the secret or account boundary.
• Do not retry the streaming negative sample automatically. It completed transport twice and still failed the content assertion.
• Permit a canary only for the exact non-streaming endpoint and model whose contract passed.
• Roll back if object type, completion status, finish reason, or required content differs from the asserted shape.
• Escalate rather than expanding traffic when the error is ambiguous or evidence is incomplete.

Retries can increase pressure during overload. Keep one owner and one shared retry budget so independent clients do not multiply attempts at the same time.

Failure modes and boundaries

• Eight bounded contract cases are evidence, not an uptime or performance sample; the separate catalog preflight brought HTTP traffic to nine requests.
• No result here establishes price, billing, quota, latency, throughput, or general provider quality.
• A different model, endpoint, token cap, client library, or parser requires its own check.
• Do not treat [DONE] as proof that streaming content was delivered.
• Keep credentials, request IDs, full prompts, and full responses outside the shared runbook.

Use How to Use Response Contract Evidence to Harden LLM API Failover for field-level assertions. Use Check CometAPI Authorization Before Fallback Routing when a route fails before content generation.

Sources checked

• CometAPI documentation — accessed 2026-07-15 for the API base and navigation.
• Chat Completions reference — accessed 2026-07-15 for Chat request and response fields.
• Responses reference — accessed 2026-07-15 for Responses fields and completion state.
• CometAPI model catalog — accessed 2026-07-15 before selecting the bounded test models.
• Google SRE overload guidance — accessed 2026-07-15 for retry-pressure boundaries.
• Sanitized eight-case contract evidence artifact — accessed 2026-07-15 for bounded promotion assertions, structured errors, usage, and streaming-negative evidence.

Reader next step

Compare CometAPI models , then create or review the API key before running the bounded checklist.

FAQ

Can the non-streaming route be promoted while streaming is held?

Yes, but only as separate route configurations with separate assertions. Do not let the non-streaming pass mask the streaming failure.

Does [DONE] mean the stream succeeded?

No. It means the SSE envelope terminated. Both observed streams reached [DONE] without producing content, so their content contracts failed.