Last reviewed: 2026-08-30

Direct answer

LLM API error-budget burn alerts work best when they measure customer outcomes, not isolated upstream attempts. Define one eligible event as one completed user action, decide what makes that event good, and calculate how quickly bad events are consuming the error budget. Keep provider-attempt metrics for diagnosis, but do not let retries or fallback attempts inflate the customer-facing denominator.

For an SLO target of S, the allowed bad-event ratio is 1 - S. Burn rate is the observed bad-event ratio divided by that allowance. A 99.9% SLO permits a 0.1% bad-event ratio; an observed 1% bad-event ratio is therefore a 10x burn. A burn rate above 1 means the service would exhaust its budget before the reporting period ends if that rate continued.

Do not page from one short window alone. Pair a short window with a longer confirmation window and require both to exceed the same burn threshold. The short window detects abrupt customer impact; the long window filters brief spikes. Use multiple pairs so a severe incident pages quickly while slower erosion creates a warning or ticket.

A burn alert is evidence of user impact, not proof of its cause and not an automatic failover command. An operator should confirm telemetry completeness, identify the affected route and interaction mode, compare customer outcomes with provider-attempt signals, and then choose a mitigation. Start by making sure you bind reliability accounting to the user action .

Who this is for

This guide is for platform engineers, SREs, and on-call owners responsible for production LLM gateways, model routes, or multi-provider fallback. It assumes you already collect request outcomes and latency but need alerts that correspond to meaningful error-budget loss.

It is especially useful when one user action may create several physical API attempts, when streaming and non-streaming calls have different latency contracts, or when a routing change can move traffic without fixing the customer-visible failure.

Key takeaways

  • Build the primary SLO around eligible user actions. Keep upstream attempts in a separate diagnostic SLI.
  • Define availability and latency as ratios of good events to total eligible events; do not mix unlike request populations.
  • Use paired short and long windows for each burn threshold. This improves urgency without paging on every transient spike.
  • Route fast-burn alerts to responders and slow-burn alerts to a ticket or team queue.
  • Treat missing telemetry, low traffic, retries, and fallback amplification as first-class failure modes.
  • Require corroborating evidence before changing routes. Error-budget burn measures impact, not root cause.

Sources checked

  • Google SRE: Alerting on SLOs explains how error budgets, precision, recall, detection time, and reset time shape an alerting strategy. It recommends multi-window, multi-burn-rate alerting over simpler single-window approaches.
  • Prometheus alerting rules documents expression-based alerts, pending and firing states, the for clause, keep_firing_for, labels, annotations, and Alertmanager handoff.
  • OpenTelemetry GenAI Semantic Conventions provides a public vocabulary for GenAI spans, metrics, and events, including client and provider-specific conventions.
  • Grafana burn-rate notification guidance documents concrete fast-burn and slow-burn window pairs, severity labels, and notification routing.

Contract details to verify

Freeze the event contract first

Write down the SLO contract before writing an alert expression. At minimum, verify:

  • Unit: one completed, eligible user action rather than one HTTP attempt.
  • Population: the routes, customers, regions, interaction modes, and request classes included in the denominator.
  • Availability good event: the user receives a response that satisfies the route contract after any permitted retry or fallback.
  • Latency good event: the eligible action completes within its published threshold. For streaming, consider a separate time-to-first-output contract instead of mixing it with full completion time.
  • Reporting period and target: for example, an agreed objective over a rolling period. The exact target must come from the service contract, not from an alerting-tool default.
  • Exclusions: malformed client requests, deliberate cancellations, test traffic, and policy rejections need explicit treatment. Do not silently change exclusions during an incident.
  • Late or missing events: decide when an action is final and how delayed telemetry affects the numerator and denominator.

Separate request populations when the user experience differs. In particular, separate streaming and non-streaming timeout behavior rather than averaging them into one ambiguous latency SLO.

Maintain two layers of indicators. The customer SLI answers whether the user action succeeded. A provider-attempt SLI records upstream response class, timeout phase, provider group, model family, and retry position. The second layer helps explain the first, but it must not redefine it.

Calculate burn from the same numerator and denominator

The core calculation is small enough to review directly:

allowed_bad_ratio = 1 - slo_target
burn_rate = observed_bad_ratio / allowed_bad_ratio
alert = short_burn >= threshold AND long_burn >= threshold

For a 99.9% SLO, allowed_bad_ratio is 0.001. If 10 of 1,000 eligible user actions are bad, the observed ratio is 0.01 and the burn rate is 10. Calculate availability burn and latency burn separately so an operator can tell whether requests are failing, merely slowing down, or doing both.

The refetched Grafana guidance uses two fast-burn pairs: 14.4x over both 5 minutes and 1 hour, and 6x over both 30 minutes and 6 hours. Its slow-burn pairs are 3x over both 2 hours and 24 hours, and 1x over both 6 hours and 72 hours. These are concrete starting points, not universal policy. Confirm that your reporting period, traffic volume, notification paths, and response expectations make the chosen thresholds meaningful.

Implement windows in the expression

Use recording rules or equivalent precomputed series for good events, total events, bad-event ratios, and burn rates. Every paired window must use the same eligibility rules and label set. A route that disappears from one window must not be treated as healthy merely because its series is absent.

Prometheus documents for as a delay that keeps an active alert pending until its expression has remained active for a specified duration. That is not the same as calculating a long-window burn rate. Google’s SLO guidance shows why a duration timer can miss repeated spikes when the condition briefly clears. Put the short and long ranges in the burn expressions. Use keep_firing_for only when a deliberate post-condition hold helps prevent flapping or false resolution; do not use it to alter the budget calculation.

Keep alert labels bounded and operational. Useful dimensions include service, slo, route_class, interaction_mode, provider_group, model_family, region_bucket, window, and severity. Do not attach prompts, response text, raw user identifiers, or unbounded request identifiers as metric labels. The OpenTelemetry GenAI conventions can help align telemetry names, but verify the exact convention version and the fields your instrumentation actually emits.

Add a separate telemetry-gap alert. A missing numerator, denominator, scrape target, or recording rule is an observability failure; it is not a zero burn rate.

Happy-path operator workflow

  1. The gateway records one final outcome for every eligible user action after permitted retries and fallback complete. Each physical attempt is recorded separately for diagnosis.
  2. Recording rules calculate good events, total events, bad-event ratios, and burn rates for every approved window from the same population.
  3. During normal service, both members of each window pair remain below threshold. Dashboards show nonzero denominators, recent evaluation timestamps, and complete route coverage.
  4. In staging or a rule-test harness, replay known good and bad event counts. Confirm that a slow-burn condition creates a warning or ticket, while a fast-burn condition reaches the paging path.
  5. Clear the test condition and verify pending, firing, and resolved transitions. Confirm that notification labels identify the SLO and route without exposing request content.
  6. Record the rule version and test result so a later rule change can be compared with the known-good behavior.

Error-path operator workflow

  1. A fast-burn page arrives. Acknowledge it and capture the first firing time, SLO name, severity, route class, interaction mode, region bucket, and evaluated window pair.
  2. Check data freshness and denominator volume before acting. Confirm that both windows cover the same eligible population and that missing series did not distort the ratio.
  3. Compare the user-action SLI with provider-attempt signals. Slice failures by response class, timeout phase, provider group, model family, and region bucket without introducing raw user identifiers.
  4. Inspect retry and fallback activity. A retry surge can consume capacity while the customer SLI initially looks stable, so review retry-budget evidence alongside the burn alert.
  5. Choose a mitigation tied to the observed failure domain. Hold a route change if the candidate route lacks fresh health evidence, shares the suspected failure domain, or would exceed reserved capacity.
  6. Annotate the mitigation time and watch availability burn, latency burn, traffic share, retry volume, and both members of the firing window pair. Roll back a change that moves traffic without improving customer outcomes.
  7. Resolve the incident only after the short window recovers, the long window is decaying as expected, telemetry is complete, and the operator has preserved the decision evidence.

Log the evaluation without logging user content

A sanitized structured record can make each page reproducible:

event: slo_burn_evaluation
service: llm_gateway
slo: availability
route_class: chat
interaction_mode: nonstreaming
provider_group: primary
model_family: general
region_bucket: region_a
rule_version: v3
short_window: 5m
long_window: 1h
good_events: 989
total_events: 1000
allowed_bad_ratio: 0.001
short_burn_rate: 11.0
long_burn_rate: 4.2
evaluation_lag_seconds: 12
alert_state: pending
severity: critical
fallback_state: held
http_status_class: 5xx
timeout_phase: upstream
request_id_hash: '[REDACTED]'
trace_id_hash: '[REDACTED]'
prompt_content: '[REDACTED]'
response_content: '[REDACTED]'

Retain the rule version, evaluated windows, counts, burn values, alert state, routing state, and coarse failure classification. Redact prompts, outputs, and user-linked identifiers. Keep high-cardinality correlation data in access-controlled logs, not metric labels, and follow the site’s retention policy.

Failure modes

  • Counting attempts instead of actions: A request that succeeds after one retry becomes one good customer outcome, not one bad event plus one good event in the primary SLO. Attempt-level failure still belongs in diagnostic telemetry.
  • Mixing unlike latency contracts: Combining non-streaming completion time with streaming time to first output can hide a regression in either population.
  • Paging on the immediate SLO threshold: A brief error ratio just above the objective can create many alerts while consuming little of the long-period budget. Use burn and paired windows.
  • Using only a long window: Detection may be acceptable, but the alert can remain active long after recovery because old bad events stay in the range.
  • Using for as a substitute for a range: Intermittent spikes can reset the pending timer even while they consume substantial budget.
  • Treating missing data as success: A broken exporter or empty denominator can produce an apparently quiet dashboard during a real outage. Alert on telemetry gaps separately.
  • Ignoring low traffic: One bad event can cause a volatile ratio, while no events provide no customer evidence at all. Show event counts beside burn, define a low-volume review path, and avoid hiding a known outage behind a minimum-volume gate.
  • Aggregating away the fault: A global route average can hide one failing region, model family, or interaction mode. Use bounded diagnostic dimensions and verify every expected slice exists.
  • Exploding label cardinality: Raw request, user, conversation, or prompt values can make the alerting system expensive and difficult to query. Keep those values out of metrics.
  • Duplicating notifications: Fast and slow conditions can overlap. Route by severity, group related alerts, and prevent a warning path from creating a second page for the same SLO and scope.
  • Automating failover from burn alone: Burn confirms budget consumption but does not identify a healthy destination. Automatic rerouting can amplify load or move traffic into the same failure domain.
  • Closing on one recovered sample: A short window may recover while the longer window remains above threshold. Confirm both the current customer outcome and the expected long-window decay.

FAQ

What should count as a bad LLM API event?

Count an eligible user action as bad when its final outcome violates the documented availability or latency contract. Examples can include an unresolved service error, a gateway timeout, or completion beyond the latency threshold. Keep availability and latency as separate indicators. Client-invalid requests and deliberate cancellations should follow explicit eligibility rules rather than being improvised during an incident.

Should an upstream failure count if fallback succeeds?

It should count as a failed provider attempt but, if the user receives a contract-compliant result within the allowed time, it can remain a good customer outcome. This separation prevents retries from overstating user-visible errors while preserving evidence that the primary route is deteriorating.

Can one SLO cover every model and endpoint?

Only if those calls share the same user promise and operating characteristics. In practice, chat, embeddings, batch work, and streaming interactions often need separate populations or thresholds. Keep the number of SLOs manageable, but do not merge contracts that operators cannot interpret consistently.

Are the Grafana burn thresholds mandatory?

No. They are documented examples that implement fast and slow burn detection. Use them as a reviewable baseline, then verify them against your SLO period, traffic profile, error-budget policy, and notification response. Preserve the window pairs when testing so the short and long expressions evaluate the same events.

Can a Prometheus for clause replace the long window?

No. The clause delays firing while the current expression remains active; it does not calculate budget consumption across a long range. A brief return below threshold resets the pending condition. Compute the long-window burn directly and combine it with the short-window burn.

What should happen when traffic is very low?

Show total event counts with the ratio, identify low-volume SLOs explicitly, and add a documented review path. A single failure may create a large burn estimate, while an empty series says nothing about customer success. Synthetic checks can be a separate signal, but they should not be silently mixed into the real-user denominator.

Should a fast-burn page trigger failover automatically?

Not by itself. First verify telemetry, scope, destination health, capacity, and failure-domain independence. Burn rate tells you how urgently the customer SLO is being consumed. It does not prove that another provider or route will improve the outcome.

Reader next step

Choose one high-volume LLM route and write a one-page SLO contract for it. Define the eligible user action, availability good event, latency good event, reporting period, objective, exclusions, and finalization rule. Then:

  1. Build customer-outcome and provider-attempt recording rules from the same documented event lifecycle.
  2. Add paired fast-burn and slow-burn expressions, plus a separate telemetry-gap alert.
  3. Put event counts, both burn windows, evaluation freshness, and bounded route dimensions on one operator view.
  4. Route fast burn to the paging path and slow burn to a ticket or team queue.
  5. Test healthy, abrupt-outage, intermittent-spike, low-traffic, missing-data, retry-surge, and recovered states.
  6. Run a tabletop exercise in which the destination route is unhealthy, ensuring the burn page does not cause an unsafe automatic failover.
  7. Record the approved rule version, thresholds, owner, and runbook link, then schedule an independent review before release.

The practical finish line is not merely a firing alert. It is an alert whose numerator, denominator, urgency, evidence, and operator response can all be explained from one user action to one reviewed decision.