Last reviewed: 2026-08-11

Direct answer

An LLM fallback must never make a side-effecting tool callable twice by accident. Treat every user-intended mutation as one durable action before sending a model request. Give that action an application-owned action ID, record the allowed tool and a canonical argument digest, and atomically reserve execution. A provider call or fallback model may attach its call_id or tool_use_id to that record, but it must not create a second execution.

When a route times out after the tool request may have reached a downstream system, the action ledger is the decision point. A terminal result is returned from storage, an in-progress action is joined, an unknown action is reconciled, and only a confirmed not-started action can run. This separates retrying language-model traffic from retrying a refund, provisioning request, ticket closure, or message send.

The Amazon Builders’ Library guidance on idempotent APIs describes an idempotent operation as one that can be retried without additional side effects and warns that matching parameters alone does not always express caller intent. Stripe’s idempotent request guidance illustrates the complementary result-record pattern: the same idempotency key returns the saved first status and response. Apply those ideas at the tool boundary, not only to the model API request.

Who this is for

This pattern is for platform, application, and reliability engineers whose model can request a tool that changes an external system. Examples include issuing a refund, creating infrastructure, sending a customer notification, changing a record, or opening and closing a support case.

It is also useful when the application currently has one provider. Provider outage handling, SDK retries, worker restarts, duplicate queue delivery, and client reconnects can all produce the same ambiguity: the caller did not receive a result, but the effect may already have happened. Adding a second model route makes that ambiguity easier to trigger because a fallback can generate a fresh tool request for the same user action.

Key takeaways

  • Create one action ID for one intended business effect. Do not make a new ID for each model attempt, provider route, or worker retry.
  • Store provider call identifiers as evidence attached to the action. They help correlate protocol messages, but they are not the application definition of a duplicate effect.
  • Validate that a reused action ID carries the same permitted tool, argument digest, and policy context. A mismatch should stop execution rather than overwrite an earlier intention.
  • Make the claim to execute atomic. Two workers that receive the same action must not both pass the tool boundary.
  • Treat an outcome as unknown when a timeout or crash leaves execution uncertain. Reconcile that state before any automatic retry or fallback.
  • Pair the ledger with a bounded retry policy. Cap CometAPI Fallback Attempts per User Action explains why retry scope belongs to the user action rather than to an individual network request.

Sources checked

The design uses the following public sources for the factual contract details behind the pattern.

  • Making retries safe with idempotent APIs explains that retrying a request after a timeout can produce an unknown result, and that caller-provided identifiers are safer than assuming equal parameters mean equal intent.
  • Idempotent requests in the Stripe API reference documents retaining the first status and response for an idempotency key, returning that result for later requests with the same key, and rejecting parameter mismatches.
  • Function calling in the OpenAI API shows tool outputs associated with a particular model tool call through call_id.
  • Tool use with Claude shows the client-tool loop in which an application executes a returned tool_use block and sends a tool_result associated with tool_use_id.

The provider identifiers in the latter two documents are valuable correlation fields. The durable action ID in this article is an application-level control that survives a new model attempt, a provider change, and a worker restart.

Contract details to verify

Define the contract before enabling automatic fallback for a mutating tool. The action record needs enough information to distinguish a repeat of the same intended effect from a genuinely new request with similar arguments.

action_id: act-7f3a
tool_name: issue_refund
args_digest: sha256:4f8c
policy_version: tool-policy-v3
state: reserved
provider_call_id: call-2b
result_ref: result-88

The caller creates action_id once, at the boundary where the product decides a user has requested an effect. It should not be derived from a model response. The tool name, normalized argument digest, authorization decision, and target scope are then bound to that action. The argument digest is useful for validation and safe logging, but it is not sufficient as the identity: two identical-looking requests can be two intended actions.

A reused action ID with changed tool semantics, changed target scope, or a changed digest is a conflict. Return a controlled error and keep the original record intact. That follows the same safety principle described in the Stripe documentation, where the idempotency layer compares later parameters with the original request instead of silently treating a mismatch as a retry.

The execution claim must be durable and exclusive. In a single transactional store, write the action record and claim it in one transaction. When the downstream tool supports idempotency, pass the same action ID as its idempotency value. When it does not, the system needs a lookup or reconciliation method before it can safely retry an unknown result. Do not let a provider fallback bypass that check.

A concrete happy-path operator workflow is:

  1. The gateway receives a user action and writes a prepared record after policy validation.
  2. The model returns a tool call. Record the provider route and provider call identifier against the existing action.
  3. A worker changes the action to running only if it owns the exclusive claim.
  4. The worker executes the tool, persists a sanitized result reference and a terminal succeeded or rejected state, then returns that stored outcome to the model.
  5. A duplicate delivery, retry, or fallback reads the terminal record and returns the stored outcome instead of executing the tool again.

The error-path operator workflow is different:

  1. A tool request times out after it may have crossed the downstream boundary. Set the action to unknown; do not immediately re-run it.
  2. Query the downstream system using the action ID or a recorded result reference. If the effect is found, persist the recovered terminal outcome.
  3. If the downstream system confirms no effect occurred, return the action to an executable state and retry with the same action ID.
  4. If the status remains unavailable, hold the action for an operator or a defined recovery queue. A model fallback may continue a conversational response, but it must not independently execute the uncertain tool action.

Log enough to reconstruct that decision without retaining raw prompts, raw arguments, user content, or tool output. A sanitized event can look like this:

timestamp: 2026-08-11T00:00:00Z
action_id: act-7f3a
tool_name: issue_refund
state: unknown
route: fallback-a
provider_call_id: call-2b
args_digest: sha256:4f8c
error_class: timeout
result_ref: none

For a provider transition, capture the prior route, proposed route, action state, and reason for the transition. This makes it possible to show that failover changed model routing while the tool effect remained bound to the same action. For response-shape work, pair this ledger with Keep Tool Calls Stable Across CometAPI Chat and Responses Fallback .

Failure modes

Timeout after send. The gateway sends a tool request and receives no response. Retrying on a new provider route is unsafe until reconciliation determines whether the downstream system performed the effect. Marking the action unknown preserves that distinction.

Provider call ID used as the dedupe key. A fallback model attempt can produce a different provider call identifier for the same user action. If the call ID is the only key, the application sees a new request and may execute the effect again. Store provider identifiers as many-to-one evidence under the action ID.

Parameter hash used as the business identity. A user may legitimately submit the same arguments twice. Conversely, a request can represent the same intended action while an incidental argument changes. Use the product action boundary to define identity, then use a digest to validate a reused identity.

Crash between side effect and completion write. A worker can perform the downstream mutation and stop before persisting succeeded. That must become unknown, not an assumed failure. Design a reconciliation lookup, downstream idempotency use, or an operator path before enabling automatic recovery.

Concurrent duplicate delivery. Two workers can receive the same queued event or a client can reconnect while a first request is still active. The exclusive state transition prevents both workers from crossing the tool boundary.

Partial result returned to the model. The tool may complete, but the model response may fail before the application sees the final text. Replaying the stored tool outcome is safer than re-running the tool. Review Classify CometAPI Partial Success Before You Retry when deciding what evidence makes an action terminal, unknown, or safe to resume.

FAQ

Is a provider call_id or tool_use_id enough? No. Those fields associate a tool result with a provider-specific model tool call, as shown in the OpenAI and Claude documentation. Persist them for traceability, but use an application-owned action ID to span provider changes and retries.

Can every tool use this workflow? Read-only tools can usually tolerate ordinary retries, although they still need rate and timeout controls. A tool that changes money, customer communication, infrastructure, records, or access should have an action identity, an outcome record, and a defined unknown-state recovery path.

Can the argument digest choose the action ID? No. It can detect accidental mismatch when an existing action is reused, but it cannot tell whether two identical inputs are one intended request or two. The Amazon source explicitly highlights that parameter equality does not always establish caller intent.

What should an operator do with a long-lived unknown action? First search for the effect using the action identity and any downstream reference. If the effect cannot be determined, keep automatic execution blocked, document the evidence, and use the business-specific recovery process. Do not resolve uncertainty by sending a new tool request through another model route.

Does returning a stored tool result force the model to produce identical wording? No. The safety contract concerns the external effect and its recorded outcome. The application should preserve the tool outcome while allowing the response layer to complete or recover according to its own response contract.

Reader next step

Choose one mutating tool and write its action record contract before the next fallback rollout. In a non-production environment, force a timeout after the tool boundary but before the response is received. The expected result is one action record, one downstream effect, an unknown state during reconciliation, and no second execution when the model request is retried or routed elsewhere.

Then add a dashboard view for the sanitized fields above and assign an owner for unknown actions. This turns an outage from an unsafe retry decision into a recoverable workflow with evidence. Use the two linked reliability guides to align retry limits and tool-call protocol handling with the same action boundary.