Handle SQS Redelivery During Long LLM Calls

Last reviewed: October 31, 2026 (Asia/Shanghai).

Direct answer

Handle SQS redelivery during LLM calls by combining bounded visibility renewal with a durable application job record. Renew visibility while useful work continues, but require a database claim before starting inference and a matching ownership epoch before accepting its result. Delete the queue message only after the job has a durable terminal outcome or responsibility has moved to another durable recovery mechanism.

An SQS visibility timeout controls when a message can become available again. It cannot establish exclusive application ownership. The Amazon SQS visibility guide says a standard-queue message may be delivered more than once even during its visibility period. A duplicate receiver must therefore consult the job record before sending another model request.

This design can protect one accepted application result. It cannot promise one provider execution or one billable request: a worker may lose its connection after the provider has accepted the call. Preserve that uncertainty and reconcile it before authorizing another attempt.

Who this is for

This guide is for engineers running asynchronous LLM work behind an Amazon SQS standard queue: document summaries, extraction jobs, evaluations, or other tasks that can outlast one visibility interval. It assumes a durable transactional store exists outside the worker process.

The queue delivery is the trigger to examine a job. The application record decides whether inference can start, whether a result can become authoritative, and what recovery remains possible after a crash. Those decisions need to survive both another delivery and a replacement worker.

Key takeaways

• Keep the application job ID stable across deliveries and model attempts.
• Treat the receipt handle as a delivery-specific SQS capability. Keep it out of routine logs.
• Claim work atomically and increase an ownership epoch whenever a new owner is authorized.
• Renew queue visibility and application ownership separately, within a fixed job deadline.
• An expired ownership lease authorizes investigation; it does not prove the earlier provider call failed.
• Store the terminal result before deleting the message, and make later deliveries harmless.

Sources checked

These maintained primary references were fetched successfully on October 30 UTC, which is October 31 in Asia/Shanghai. They support an evergreen worker design; this article does not describe a new release, an incident, or a measured deployment outcome.

• Amazon SQS visibility timeout documents redelivery after expiry, heartbeat renewal, the 12-hour limit, and duplicate-delivery caveats.
• SQS ChangeMessageVisibility documents when a replacement timeout starts, its limits, and its behavior on the next receive.
• SQS DeleteMessage documents receipt-handle selection, old-handle ambiguity, and possible delivery after deletion on standard queues.
• PostgreSQL UPDATE documents conditional updates and returning the rows actually updated. The job state machine below is an application recommendation built on those primitives.

Contract details to verify

Separate the three identities

Use one application-created job_id for one intended workload. Bind it to immutable input and an output-contract version. A second intended run gets a new job ID even if its input is identical. A repeated ID with changed input is a conflict.

Keep the SQS message identifier for transport correlation, and retain the receipt handle only in the delivery context that needs it. AWS says each receive returns a different receipt handle. Neither field should replace the application job ID.

Finally, store an integer owner_epoch. A successful ownership transfer increments it. Every result acceptance, retry authorization, and state transition must check that epoch. This is fencing: a worker holding epoch 7 cannot commit after a replacement has advanced the record to epoch 8.

An epoch only protects boundaries that enforce it. It cannot recall a provider request already sent or stop a stale process from consuming tokens. Keep provider submission and external effects behind the same authorization policy, and retain downstream idempotency safeguards where supported.

Make recovery eligibility explicit

A minimal job record needs the input reference and digest, state, owner epoch, ownership expiry, fixed job deadline, cumulative attempt count, provider attempt references, and accepted result reference. Create it durably before enqueueing; use an outbox or another repairable enqueue procedure so a crash cannot strand an accepted job between the database and SQS.

Use states with different permissions. ready permits a first attempt. running records a claimed attempt. unknown means a provider call may have happened without a confirmed outcome. retry_authorized means reconciliation or an explicit replay policy permits another attempt. succeeded, failed_terminal, and expired are terminal application decisions.

Do not let a receiver change an expired running record straight to a fresh provider attempt. First obtain exclusive recovery ownership, fence out the old epoch, and inspect its attempt evidence. If the provider offers a documented retrieval mechanism and you retained its identifier, try that. Otherwise keep unknown for operator review, or apply a preapproved replay policy that explicitly accepts possible duplicate inference and cost. Record that decision against the same job and cumulative attempt budget.

The site’s tool side-effect safeguards cover the additional protection needed when model output can trigger a refund, notification, or other mutation. Job fencing alone does not make those effects safe to repeat.

Set two renewable limits and one fixed deadline

SQS visibility and application ownership are separate limits. A successful renewal of one does not prove the other remains valid. Keep a fixed product deadline above both and stop renewal when the job reaches it, loses ownership, or stops meeting your progress policy.

The ChangeMessageVisibility reference gives a useful timing distinction: the new interval starts when the change is made. It is not added to the previous expiry. An individual adjustment also does not become the default for the next receive.

For an illustrative policy, receive with 90 seconds of visibility, attempt renewal every 25 seconds, and reserve 15 seconds for scheduling delay and the renewal request. Choose production values from observed latency and control-plane behavior. A timer firing is not proof of model progress, and a blocked event loop must not silently stop the renewal mechanism.

The visibility guide caps an extension chain at 12 hours from the initial receive; renewing does not restart that limit. A change exceeding the remaining maximum returns an error rather than being automatically shortened. Bound requests conservatively and split work or use another orchestration design before approaching the cap.

Preserve the job deadline across redelivery and across provider fallback. The site’s deadline propagation guide explains why a replacement attempt must spend the remaining budget. Receiving the same message again creates no new product budget.

Guard every result write

Use a conditional transition for result acceptance. In PostgreSQL, the following illustrative statement assumes an existing job row, bound parameters, a durable result artifact, and an ownership lease measured by the database clock:

UPDATE llm_jobs
SET state = 'succeeded', result_ref = $3
WHERE job_id = $1
  AND owner_epoch = $2
  AND state = 'running'
  AND owner_until > clock_timestamp()
  AND deadline_at > clock_timestamp()
RETURNING job_id;

PostgreSQL documents that WHERE controls which rows change and RETURNING reports rows actually updated. A zero-row result means this worker did not accept an answer. Re-read the record to distinguish lost ownership, expiry, and an already terminal job. Never follow a rejected update with an unconditional write.

Keep database transactions short; do not hold one open while calling the model. The claim, ownership renewal, recovery claim, retry authorization, and terminal transition all need compatible conditional rules. The statement above illustrates one boundary, rather than supplying a complete worker implementation.

If accepting the answer also creates a notification, write its unique outbox entry in the same transaction as the terminal transition. A separate dispatcher handles delivery and its own idempotency. Retain the job record through the queue retention and permitted replay period so a late duplicate still finds its terminal decision.

Acknowledge after durable responsibility

After the result transaction commits, call DeleteMessage with the receipt handle for the current delivery. The DeleteMessage reference warns that using an old handle can return success without deleting the message. It also says a standard-queue copy can occasionally arrive after deletion. A terminal job record makes both cases recoverable without new inference.

A duplicate receiver that finds a live owner should return to a bounded wait or visibility policy without launching inference. Do not automatically delete that delivery unless a durable reconciler has accepted responsibility for the still-running job. Otherwise, deleting the queue trigger and then losing the original worker can strand the job.

Once a durable terminal record exists, any later receiver can acknowledge its own delivery without calling the model. If deletion times out, keep the terminal record and allow redelivery to resolve the transport uncertainty. A queue acknowledgement is cleanup, not permission to regenerate.

Happy-path operator workflow

Consider an illustrative two-minute generation with 90 seconds of initial visibility and a three-minute job deadline. Worker A claims job job-demo at epoch 7 and records the attempt before sending it. At 25 seconds it renews visibility to another 90 seconds and separately renews its ownership lease. At 50, 75, and 100 seconds it repeats those bounded checks while the job remains eligible.

At 120 seconds, A stores the validated answer in durable storage, commits the guarded terminal transition, then deletes the message using its current receipt handle. If another copy arrives later, its receiver reads succeeded, returns the stored result through the normal application path, and acknowledges that delivery.

The expected evidence is one accepted job outcome and a timeline connecting receives, renewals, provider attempts, terminal commit, and deletion. These are design expectations, not reported test results.

Error-path operator workflow

Suppose A pauses long enough for visibility and application ownership to expire. Worker B receives the job while A’s model request may still be running.

If A resumes after B has advanced the epoch, A’s result transition must fail. A may retain its answer as restricted diagnostic evidence, but it cannot publish it, overwrite the accepted result, authorize fallback, or delete the message as if it still owned the job. Cancelling A’s local request may reduce waste; it does not prove the provider stopped or waived billing.

When a renewal request times out, record uncertainty separately from a confirmed renewal. Retry only while a conservative remaining-validity budget allows it. If visibility becomes uncertain but the database lease remains valid, another receiver must defer to that lease. If database ownership cannot be renewed before its expiry, stop new provider attempts and effects, cancel locally where possible, and leave durable recovery evidence. Do not set visibility to zero while an unfenced worker can still commit.

Failure modes

Renewing forever. A healthy heartbeat can surround a hung request. Enforce the fixed deadline, a finite attempt budget, and a progress rule independent of timer activity.

Resetting attempts on receive. A new worker starts its counter at one while earlier provider calls remain billable. Keep cumulative attempts and replay authorizations in the job record.

Deleting before persistence. The worker removes the queue trigger, then crashes before storing the answer. Commit durable responsibility first.

Letting expired owners write. A database lease without conditional result acceptance allows a paused worker to overwrite recovery. Apply the epoch check to every authoritative write.

Replaying a dead-letter queue blindly. A message’s transport failure history does not establish its application outcome. Inspect terminal and unknown jobs before replay, preserving job IDs, deadlines, and attempt budgets.

FAQ

Does a longer visibility timeout prevent duplicate LLM requests?

It can reduce premature redelivery, but standard queues still require duplicate handling. Longer visibility also delays recovery after a crash. Use bounded renewal plus application claims and guarded writes.

Can a successful delete replace the job ledger?

No. AWS documents both old-handle ambiguity and occasional standard-queue delivery after deletion. A retained terminal job record is what lets the application absorb another delivery safely.

Can fencing guarantee exactly one provider execution?

No. It can reject stale application writes where the epoch is checked. An earlier request may already be executing upstream. Reconciliation, documented provider idempotency where available, and explicit replay policy address that separate uncertainty.

Reader next step

Run a non-production drill with a stubbed model call that outlasts the initial visibility interval. Pause its worker until another receiver appears, then resume it after recovery ownership changes. Verify that a live owner suppresses duplicate inference, stale epochs cannot accept results, an uncertain attempt requires a recorded recovery decision, and a committed answer survives failed message deletion.

Capture job reference, delivery count, owner epoch, remaining deadline, renewal outcome, attempt count, recovery reason, result-transition outcome, and deletion status. Use bounded categories in metrics and protected correlation identifiers in traces. Keep prompts, answers, credentials, and full receipt handles out of routine logs. The drill passes on those state transitions; counting a successful queue delete alone is insufficient.