Last reviewed: 2026-06-14
Direct answer
When a CometAPI provider silently swaps a model or retires a model version, your reliability checks can pass on stale assumptions. Model change evidence is the set of observable signals — model identifiers returned in API responses, version entries in the models catalog, and behavior deltas between calls — that let you confirm the model your code expected is still the model your code is using.
The core workflow: compare the model identifier field in each /api/text/chat or /api/text/responses response against the current entry in the CometAPI models catalog, then assert the match inside your smoke test before promoting traffic. Exact field names must be verified in the linked documentation before coding assertions; do not assume field paths from prior experience or prior API versions.
For broader release checks, see CometAPI chat reliability contract review.
Who this is for
Backend engineers and SRE teams who route production traffic through CometAPI and need reliable checks that survive upstream model changes — whether during a planned rollout, a provider-side swap, or an incident investigation. This guide is also relevant to anyone writing automated integration tests against the CometAPI chat completions or responses endpoints.
Key takeaways
The CometAPI chat completions endpoint and responses endpoint both return a model identifier in each response. Verify the exact JSON field path in the current docs before asserting it in code. See Contract details to verify.
The CometAPI models catalog lists available models and their identifiers. Checking this catalog before and after a deployment gives you a diff-able baseline for model availability claims.
A response-level model identifier assertion is a cheap, reliable canary: if the returned model differs from the one you requested or expected, your check should log the discrepancy before deciding whether to fail or alert.
A successful 2xx response does not confirm the expected model was used. The response shape may be valid while the model has silently changed. This is the core reliability gap that model-change evidence addresses.
Treat model-change detection as a distinct check from endpoint health. An endpoint can return healthy 2xx responses while the underlying model has changed.
See also: How to Use Response Contract Evidence to Harden LLM API Failover for the complementary pattern of asserting full response shape beyond the model identifier field.
Smoke-test workflow
Setup assumptions
- A valid CometAPI API key is available in your test environment via an environment variable. Do not hardcode credentials.
- You have confirmed the model identifier you intend to request is currently listed in the CometAPI models catalog.
- Your test environment can reach the CometAPI API. Verify the exact endpoint path from the current chat completions reference or responses endpoint reference before coding assertions.
Happy-path request plan
- Send a minimal request to the chat completions or responses endpoint, specifying the model identifier you expect. Exact request body field names must be confirmed in the endpoint documentation before coding.
- Assert the HTTP status is 200.
- Parse the response and read the model identifier field. Confirm the exact field path from the current endpoint documentation — do not assume a field path from prior experience.
- Assert that the returned model identifier matches the model identifier you specified in the request.
- Record a pass entry using the log template below.
Error-path check
- Send a request specifying a model identifier you know to be invalid for the current catalog (confirm an invalid example from the docs or use a clearly fictitious string).
- Assert the API returns a non-2xx status or contains an error field in the response body.
- Log the HTTP status and any error code or message fields returned. Exact error field names must be confirmed from the endpoint documentation.
Minimum assertions
- HTTP status 200 for the happy path.
- Returned model identifier matches requested model identifier.
- No error field present in the happy-path response body.
What this smoke test must not assert
- Specific latency targets, token counts, billing fields, rate limits, or quota values. These are not model-change evidence and can cause false failures on valid responses.
- The exact text content of the model reply. Response text varies by input and cannot be used as a reliability assertion.
Sanitized log record template
Record the following fields after each smoke-test run. Use actual values at test time; the template contains placeholders only.
smoke_test_run: run_id: site_id: llm-api-reliability check_type: model_change_evidence endpoint_path: requested_model: returned_model: model_match: true | false http_status: 200 error_field_present: false pass: true | false notes: ’' checked_at:
Do not log API keys, full request bodies, full response bodies, token counts, prices, or rate-limit header values.
Failure modes
- Evidence gap: the agent cannot inspect the failing log, source page, pull request, or local command output. The safe action is to stop and record the missing evidence instead of guessing.
- Scope drift: the agent edits files that are not connected to the observed failure. Keep the repair tied to the failing signal and leave unrelated cleanup for a separate task.
- Environment mismatch: the local check uses different versions, credentials, feature flags, or runtime settings than the hosted path. Record the mismatch before treating the result as proof.
- Unreviewed fallback: the agent changes models, endpoints, permissions, or retry behavior to make a run pass without preserving the review boundary. Treat access and provider failures as operational blockers, not topic failures.
- Weak handoff: the final note says the issue is fixed but omits the command, result, changed files, and remaining uncertainty. That makes the next operator repeat the investigation.
Sources checked
- CometAPI documentation - accessed 2026-06-14; purpose: verify current CometAPI documentation navigation.
- CometAPI models overview - accessed 2026-06-14; purpose: verify model catalog discovery guidance.
- CometAPI chat completions reference - accessed 2026-06-14; purpose: verify chat completion contract areas.
- CometAPI responses reference - accessed 2026-06-14; purpose: verify responses endpoint contract areas.
- CometAPI help center - accessed 2026-06-14; purpose: verify support and escalation documentation areas.
Contract details to verify
| Area | What to verify | Source URL | Accessed | Safe candidate wording |
|---|---|---|---|---|
| Error response shape | Field names and codes returned when a model identifier is invalid | https://apidoc.cometapi.com/api/text/chat | 2026-06-14 | “the error field returned by the API” |
| Help center reliability caveats | Any documented caveats about model availability or fallback behavior | https://apidoc.cometapi.com/support/help-center | 2026-06-14 | “as described in the CometAPI help center” |
Reader next step
Compare the workflow against Start with CometAPI.
Use CometAPI chat reliability contract review as the next comparison point. Keep Timeout-budget fallback checks for chat completions nearby for setup and permission checks.
FAQ
Q: Why can a 2xx HTTP response be insufficient to confirm model reliability?
A 2xx response confirms the endpoint accepted the request and returned a result, but it does not confirm which model processed the request. A model change can be completely silent at the HTTP layer. Asserting the model identifier from the response body against the catalog is the additional step that closes this gap.
Q: Where do I find currently valid model identifiers for CometAPI?
The CometAPI models catalog is the authoritative source. Check it directly before each deployment or reliability review cycle. Do not rely on a cached list that may not reflect recent provider changes.
Q: Should I assert model identity on every production request?
For high-volume traffic, sampling is practical. Assert model identity in your smoke test before each deployment and periodically in a low-traffic canary. For guidance on structuring observability fields around these checks, see HTTP Telemetry Fields for CometAPI Reliability Reviews.
Q: What should I do if the returned model differs from the requested model?
Log the discrepancy using the template in this guide, compare the returned identifier against the current models catalog, and escalate to CometAPI support if the expected model is no longer listed or the behavior has changed materially. The CometAPI help center documents the escalation path.
Q: Does the model-change evidence approach apply to the responses endpoint as well?
Yes. The same principle applies — read the model field from the response and compare to your expected value. The exact field path may differ between endpoints; verify it in the responses endpoint reference before writing assertions.
Ready to build more reliable LLM API integrations? Start with CometAPI