Timeouts
Timeout budgets, retries, and safe fallback boundaries for LLM API calls.
Reliability Runbook
LLM API Reliability Notes
Practical guides for LLM API reliability and fallback engineering.
Route health
monitored
- DetectTimeout or 5xx spike
- RouteFallback policy check
- VerifyResponse contract smoke test
- EscalateHuman review when evidence is thin
124
Published and archived Reliability notes
SLO
Primary lens
Daily
Review cadence
Failure Map
Reliability notes organized by failure mode
Use these notes to decide what to retry, route, stop, or escalate before production traffic is exposed.
Rate limits
How to identify quota behavior without hiding customer-impacting failures.
Fallback routes
Decision rules for switching providers, models, or degraded modes.
Incident checks
Smoke tests and evidence requirements before calling a route production-ready.
Latest Runbook Notes
Recent reliability guides
Historical archive entries remain available to readers while staying out of RSS, sitemap, and llms.txt.
Rotate LLM API Keys Without Stranding Stale Workers
Rotate LLM API credentials through explicit versions, canary checks, worker convergence, and provider revocation so old secret copies do not strand running consumers.
Block Unexpected Redirects in CometAPI Requests
Stop authenticated CometAPI calls at unexpected redirects, preserve POST semantics, and review endpoint changes before resending credentials or prompts.
Keep TLS Trust Intact During LLM API Failover
Preserve certificate verification, hostname identity, and deliberate trust-store selection when an LLM gateway changes routes during failover.
Stop Realtime LLM WebSocket Reconnects From Becoming Retry Storms
Control realtime LLM WebSocket reconnects with one reconnect owner, jittered backoff, session epochs, admission limits, and explicit replay rules.
Design end-to-end deadline propagation and per-attempt timeout enforcement for LLM API fallback chains.
A practical pattern for carrying one caller deadline through LLM retries and fallback providers while enforcing per-attempt limits and cancellation.
Reconcile Token Usage When CometAPI Streams End Early
Build a trustworthy usage ledger for LLM streams that terminate before provider-reported token totals arrive.