Last reviewed: 2026-10-15

Direct answer

Stop realtime LLM WebSocket reconnects from becoming a retry storm by giving each logical session exactly one reconnect owner, one bounded attempt budget, and a randomized backoff schedule. Put a separate admission limit in front of new handshakes so thousands of disconnected clients cannot all reconnect merely because their local timers say they may. A new socket must receive the provider’s documented readiness signal before it carries user traffic.

Treat a transport connection and an application session as different objects. Increment a connection_epoch whenever a logical session opens a replacement socket. Discard events from older epochs, and do not replay audio, prompts, tool approvals, or tool results unless the provider documents a resume or acknowledgement contract that makes the replay safe. A successful WebSocket handshake proves that a transport exists; it does not prove that an earlier model session was resumed or that unacknowledged work was not processed.

This separation follows the protocol evidence. RFC 6455 says clients recovering from abnormal closure should begin with a random delay and use increasingly longer delays after repeated failures so a service recovery is not overwhelmed. Its Ping and Pong frames can check whether a peer is responsive, but they do not define an LLM conversation-resume protocol. The OpenAI WebSockets guide shows a current LLM API exchanging client and server events over a long-lived WebSocket. It is provider-specific evidence for the transport pattern, not a promise that reconnecting resumes the same session.

Who this is for

This guide is for SREs, platform engineers, voice-agent teams, and client-library owners who operate server-to-server realtime LLM connections. It applies when a network interruption, edge restart, credential refresh, regional incident, or client deployment can disconnect many sessions at once.

The same controls can inform browser or mobile designs, but those clients may use a different transport or expose different liveness APIs. Verify the exact provider and client contract before applying a server-side Ping/Pong mechanism. The control objective is stable across runtimes: reconnect deliberately, admit within capacity, and never convert transport uncertainty into silent duplicate work.

Key takeaways

  • Elect one reconnect owner per logical session. Competing timers, SDK callbacks, and supervisors must not each open a replacement socket.
  • Use capped exponential backoff with fresh jitter, but charge every attempt and wait against one session-level retry budget.
  • Limit new handshakes globally and per tenant. A per-client delay alone does not protect the fleet from a correlated disconnect.
  • Increment a connection epoch and reject late events from superseded sockets.
  • Wait for the documented session-ready event before sending user traffic; TCP or WebSocket open is too early.
  • Separate liveness, readiness, and continuity. Ping/Pong can support liveness, while only the provider contract can establish session resume or event replay.
  • Never replay old audio or side-effecting tool work from inference. Resume only from an acknowledged boundary.

Sources checked

  • RFC 6455: The WebSocket Protocol , accessed 2026-10-15, defines Close, Ping, and Pong control-frame behavior and recommends randomized, progressively longer reconnect delays after abnormal closure. It is the stable wire-protocol standard; its reconnect example is guidance, not a universal production interval.
  • OpenAI: WebSockets , accessed 2026-10-15, documents current server-side WebSocket connections and client/server event exchange for voice and realtime APIs. It supports the concrete transport and readiness distinction used here. It does not establish a generic reconnect, replay, or session-resumption contract.
  • AWS Builder Center: Timeouts, retries, and backoff with jitter , accessed 2026-10-15, explains why retries add load, why capped backoff needs jitter, why retrying at one layer is safer than multiplying attempts, and how a token bucket can bound retry traffic. This is general distributed-systems guidance, not an LLM provider guarantee.
  • Google SRE: Handling Overload , accessed 2026-10-15, supports explicit capacity limits, client-side throttling, and rejecting excess work when a service cannot safely accept it. Its implementation examples are not default limits for a realtime LLM service.

All four sources were reachable public primary references when checked. The protocol and overload references remain current because they support stable transport and control principles; the OpenAI page was checked for a present-day example of an LLM WebSocket contract.

Contract details to verify

Define three identities

Use a stable logical_session_id for the user-visible interaction, a monotonically increasing connection_epoch for each socket attached to it, and a provider connection or session identifier only in the scope where that provider documents it. Keep the public session ID opaque. Do not place a user ID, phone number, prompt, access token, or raw provider credential in metrics or logs.

Only the reconnect owner may move a session from disconnected to connecting. Implement that rule with a single task, actor, lease, or compare-and-set transition. A transport callback may report failure, but it should signal the owner rather than starting its own loop. Cancel pending timers when the session closes normally, the user leaves, the deadline expires, or another epoch becomes ready.

Classify the disconnect before retrying

Create a small decision table for normal application close, local cancellation, credential failure, protocol error, provider overload, network loss, and unknown abnormal closure. A normal user exit and a revoked credential should stop. A transient network loss may enter backoff. An unknown state should not be treated as permission to replay work.

RFC 6455 reserves status code 1006 for a local indication that a connection ended abnormally; it is not a close code sent in a Close frame. Preserve the locally observed code, whether a peer Close frame was received, the last acknowledged application event, and the connection epoch. Do not let one numeric code decide session continuity on its own.

Put delay and admission in separate controls

A reasonable application policy can calculate the next delay as:

delay_cap = min(max_delay, base_delay * 2^attempt_index)
sleep_for = random_between(0, delay_cap)

This is a recommended full-jitter shape, not a value prescribed by the provider. Cap the exponent, maximum delay, elapsed reconnect window, and total attempt count. Reset the attempt counter only after the connection has remained ready for a documented stability interval, not immediately after the handshake.

Then require an admission token before starting DNS, TLS, authentication, or the WebSocket handshake. Maintain at least a fleet-wide ceiling and a per-tenant ceiling. Reserve some capacity for probes and operator tests. When no token is available, remain in backoff or return a controlled unavailable state; do not create an unbounded queue of already-expired sessions. This complements the site’s guidance on adaptive concurrency limits .

Separate open, live, ready, and resumed

Record these milestones independently:

  1. Transport open: the WebSocket handshake completed.
  2. Transport live: recent traffic or a supported heartbeat shows that the peer responds.
  3. Application ready: the provider emitted the documented event that permits session traffic.
  4. Continuity established: the provider explicitly accepted a resume token or acknowledged the replay boundary.

Do not synthesize step four from steps one through three. If the provider has no documented resume mechanism, create a new application session and tell the product layer that continuity was lost. Ping/Pong can reveal a dead path, but an alive path can still be unauthorized, overloaded, or not ready for model events.

Make replay an explicit policy

Track the last application event that the provider acknowledged under its documented semantics. If there is no acknowledgement, retain last_acknowledged_event as unknown. After reconnect, send only configuration and new user input required by the fresh session. Do not automatically resend an audio buffer, a completed prompt, or a tool approval because the client did not see the answer.

Side effects need a separate idempotency boundary owned by the application. A connection_epoch filters late transport events; it does not make a payment, message send, database write, or tool call safe to repeat. If the provider returns an old-epoch event after the new socket is ready, record and discard it before any side-effecting consumer.

Keep the original deadline

A reconnect is another phase of the same user action, not a fresh latency budget. Charge detection time, backoff, admission wait, handshake time, and readiness wait against the original deadline. Stop when the remaining time cannot cover a useful attempt. The site’s deadline propagation guide provides the matching calculation for downstream calls.

Record sanitized evidence

One reconnect decision record can look like this:

{
  "logical_session_ref": "sess_7d2a",
  "tenant_ref": "tenant_18",
  "connection_epoch": 4,
  "attempt_index": 2,
  "disconnect_class": "network_abnormal",
  "peer_close_received": false,
  "local_close_code": 1006,
  "last_acknowledged_event": null,
  "backoff_cap_ms": 4000,
  "selected_delay_ms": 1730,
  "admission_result": "granted",
  "remaining_deadline_ms": 11800,
  "outcome": "ready_new_session",
  "observed_at": "2026-10-15T07:10:07Z"
}

Use opaque references and bounded enums. Do not log audio, transcripts, prompt text, tool arguments, authorization headers, resume secrets, or raw close reasons supplied by an untrusted peer. Put high-cardinality identifiers in trace records rather than metric labels.

Happy-path operator workflow

  1. Detect that the active epoch lost transport and atomically assign its reconnect owner.
  2. Classify the disconnect, preserve the last documented acknowledgement, and check the original session deadline.
  3. Consume one attempt from the session budget, calculate fresh jitter, and cancel the wait if the user or deadline ends first.
  4. Acquire tenant and fleet handshake admission. If admission is unavailable, wait only within the remaining deadline.
  5. Open the socket with current credentials and release the handshake token according to the limiter’s documented boundary.
  6. Increment the epoch, attach handlers that reject older epochs, and wait for the provider’s documented ready event.
  7. Resume only if the provider explicitly confirms continuity. Otherwise start a new session and send only new, approved work.
  8. Mark the reconnect stable only after the configured observation interval, then clear the failure counter and retain the sanitized timeline.

Test this path with a controlled set of synthetic sessions before a fleet exercise. Increase the disconnect cohort gradually and observe handshake rate, admission denials, time to ready, sessions that exhaust their deadline, and old-epoch events. Do not claim recovery capacity from one successful connection.

Error-path operator workflow

If authentication fails, stop automatic reconnects for that credential version and route the result to the credential owner. If the provider reports overload, return the admission token, extend the bounded backoff, and reduce new-connection concurrency; opening more sockets is not a recovery action. If readiness never arrives, close the new socket, record ready_timeout, and charge the failed attempt to the same budget.

If continuity is uncertain, do not replay. Surface a controlled session-interrupted state and require the user or application workflow to decide whether to start again. If two reconnect owners appear, close the losing epoch and investigate the ownership invariant. If the client has disconnected, cancel the timer and active handshake so abandoned sessions do not consume recovery capacity. Pair that check with stopping canceled requests from triggering fallback .

Failure modes

Every layer reconnects. The SDK, application, supervisor, and load balancer each create attempts. Elect one owner and disable or account for hidden retries in other layers.

All clients share the same delay. Deterministic waits merely move the spike. Select fresh jitter per attempt and still enforce fleet admission.

Success resets backoff at socket open. A connection that fails before readiness can loop rapidly. Reset only after a stable ready interval.

Ping/Pong is treated as session health. A responsive transport may still lack authentication, model readiness, or conversation continuity. Measure each state separately.

Old events cross epochs. A late message from a replaced socket reaches a tool executor. Bind every handler to its epoch and reject stale events before business logic.

Audio or tools are replayed from guesswork. The provider may already have processed them. Require a documented acknowledgement or idempotency contract; otherwise report interruption.

The limiter protects requests but not handshakes. DNS, TLS, authentication, and session initialization consume resources before the first model event. Put admission in front of connection establishment.

A reconnect gets a new deadline. Repeated attempts outlive the user action and keep capacity busy. Carry the original absolute deadline through every wait and handshake.

Normal shutdown looks like failure. Deployments and user exits trigger the same callback as network loss. Record local intent before closing and suppress reconnect after an expected terminal transition.

FAQ

What is the difference between reconnect and resume?

Reconnect creates another transport connection. Resume preserves or restores application-session state under a provider-defined contract. Never infer resume from a successful handshake, a reused local ID, or a copied conversation transcript.

Should every abnormal close be retried?

No. Retry only a classified transient condition while attempts, elapsed time, deadline, and admission capacity remain. Credential, policy, protocol, user-cancellation, and deterministic request failures normally require stopping or repair.

Is exponential backoff enough to stop a reconnect storm?

No. Backoff reduces each client’s attempt rate, but a correlated fleet can still create a large handshake burst. Add randomized delay, a global and per-tenant admission limiter, and load shedding for sessions that cannot finish within their deadline.

Can close code 1006 be sent by the server?

No. RFC 6455 defines it as a reserved value for a local abnormal-closure result, not a value placed in a Close control frame. Record whether a peer Close frame actually arrived.

Can Ping/Pong prove that the model session is usable?

No. It can support transport-liveness detection when the runtime exposes it. Readiness, authorization, model availability, and continuity require application-level evidence from the provider contract.

When should the retry counter reset?

After the replacement connection has reached application readiness and remained stable for a deliberate observation interval. Resetting at handshake completion lets a failing session reconnect indefinitely without accumulating failures.

Reader next step

Choose one realtime route and draw its state machine from ready through disconnect, backoff, admission, handshake, readiness, and terminal failure. Name the single reconnect owner, original deadline, attempt ceiling, global and per-tenant handshake limits, stable-reset interval, and exact provider event that establishes readiness. Mark resume support as absent until a current provider page explicitly documents it.

Then run a bounded fault exercise with synthetic sessions. Disconnect a small cohort simultaneously, verify that connection attempts stay below the admission ceiling, cancel half the clients during backoff, and inject a late event from an old epoch. The pass condition is not simply that sessions reconnect: new handshakes remain bounded, canceled sessions stop, stale events are discarded, uncertain work is not replayed, and every terminal outcome has a sanitized reason.