Own LLM HTTP Clients with FastAPI Lifespan

Last reviewed: November 2, 2026 (Asia/Shanghai).

Direct answer

Create the reusable HTTPX client inside FastAPI’s lifespan context, pass it to handlers through lifespan state, and close it when that context exits. Keep the client within the event loop and application lifecycle that own it. Request handlers borrow the client; they close their own streaming responses and leave whole-client shutdown to the lifespan owner.

The ASGI Lifespan Protocol specifies one lifespan per event loop processing requests, with a lifespan in each process in a multi-process deployment. Lifespan and request processing use the same loop so connection pools do not move between loops. “One shared client” therefore needs an ownership scope: a deployment with several workers has several client lifetimes.

The HTTPX Async Support guide recommends avoiding repeated client construction in a hot loop to preserve pooling benefits. FastAPI’s Lifespan Events guide provides the startup and shutdown boundary for shared resources. Together, these references support a client that is reusable during service and explicitly closed afterward.

Who this is for

This guide is for Python gateway maintainers and SREs running asynchronous LLM adapters in FastAPI. Use it when a client appears to be closed during ordinary requests, tests behave differently when run together, or client construction is scattered through routes and retry helpers.

The task is whole-client ownership. Provider response validation, individual stream cleanup, admission control, and deployment draining remain separate responsibilities. A correctly owned client can still carry an invalid request or exhaust its available connections.

Key takeaways

• Construct the client when the owning application’s lifespan starts.
• Reuse that client only within its intended lifecycle and event loop.
• Give handlers access through lifespan state without giving them shutdown ownership.
• Allocate connection limits per client and account for worker replication.
• Exercise startup and teardown in tests; successful route dispatch alone proves neither.
• Record client closure separately from response completion and remote execution.

Contract details to verify

Name the owner before choosing a sharing pattern

Write down the resource hierarchy before changing code:

A module can hold an application definition without holding a live HTTP client. The factory below delays client creation until lifespan entry, making the ownership boundary visible and allowing tests to supply a controlled client factory.

Avoid retaining a client from one test session for a later session with another event loop. Likewise, a synchronous entry point that starts a fresh loop for each invocation must not silently keep using a network client from an earlier loop. Symptoms depend on the runtime and transport; an event-loop error is a diagnostic lead, rather than a universal signature of this defect.

Use lifespan state to lend the client to requests

Save this as client_lifetime.py in an environment with FastAPI and HTTPX installed:

from contextlib import asynccontextmanager

import httpx
from fastapi import FastAPI


def create_app(client_factory=httpx.AsyncClient):
    @asynccontextmanager
    async def lifespan(app):
        async with client_factory() as client:
            yield {"llm_http": client}

    return FastAPI(lifespan=lifespan)


app = create_app()

In an asynchronous route, obtain the borrowed client with request.state.llm_http, where request is a FastAPI Request. Pass it into the existing LLM adapter. Keep the adapter’s approved endpoint, request-specific authorization, response-size checks, overall deadline, and response contract in place.

The Starlette Lifespan guide documents this pattern with an httpx.AsyncClient and a dictionary yielded by the lifespan handler. It states that request state is a shallow copy of lifespan state. The dictionary is copied, while the referenced client remains the shared object. Calling aclose() from one handler can therefore affect other handlers borrowing that object. Do not close the shared client merely because one upstream request failed.

The example uses attribute access, which the Starlette guide supports. That guide dates dictionary-style typed state access to Starlette 0.52.0; do not copy newer syntax into an older installation without checking compatibility. The local verification below used Starlette 0.46.2 and attribute access.

The default factory deliberately leaves transport policy to the surrounding application. Supply a factory configured with reviewed TLS and proxy settings, connection limits, and timeouts before deployment. Build provider or traffic-class partitions when isolation is needed. The site’s bulkhead pool guidance explains that separate capacity decision.

Account for every worker and lifecycle participant

A per-client connection setting is not a deployment-wide admission limit. For an illustrative deployment with four workers and one client per worker, setting max_connections=8 on each client gives those four clients a combined configured ceiling of 32 connections. This is arithmetic about the configuration, not a measured provider capacity or a guarantee about concurrent HTTP/2 requests. Add any separate clients and other replicas to the inventory, and enforce the intended shared request budget separately.

Put required initialization before the lifespan’s yield. The ASGI specification requires a server to wait for startup completion before processing connections. Verify that the deployed server actually runs lifespan and that a required-resource failure prevents normal readiness. A local constructor can succeed without proving that DNS, credentials, quotas, or the selected model work. Decide separately whether startup needs an authorized upstream probe; it can introduce cost and a new startup dependency.

FastAPI recommends the lifespan parameter and warns that providing it disables the older startup and shutdown event handlers. During a migration, move required initialization and cleanup into the chosen lifespan arrangement together. Keeping a database initializer only in the old event handler can leave the new HTTP client available while another required resource never starts.

FastAPI also says these events run for the main application rather than mounted subapplications. Inventory mounted apps explicitly. Give each needed resource an owner in the active startup path instead of assuming every mounted application’s lifespan will run automatically.

Preserve the response and task boundaries

Reusing the client does not extend a response’s valid consumption scope. Keep each streamed response open until its consumer ends, then close that response through the supported interface. The site’s HTTPX response cleanup guide covers that narrower lifetime.

Starlette documents lifespan teardown after connections close and in-process background tasks complete. Arbitrary tasks created elsewhere still need an explicit owner and completion policy. Ensure background users of the HTTP client have ended before closing it. Keep shutdown bounded through the server and deployment contract, and test forced termination separately: a process killed before teardown cannot demonstrate successful cleanup.

Happy-path operator workflow

Trace one request from lifespan startup to adapter dispatch. Confirm that creating the application does not create the client, startup creates the intended clients, and overlapping handlers receive the appropriate shared reference. Then finish the requests and verify orderly teardown closes each client once its borrowers have ended.

Record lifecycle start, worker identity, configuration revision, readiness, teardown start, and close outcome in sanitized events. Keep prompts, completions, credentials, full request objects, and raw exception text out of routine lifecycle logs. Client counts and close outcomes are useful without exposing content.

Run the focused offline check below, then repeat through the actual server and middleware. A successful restart is insufficient evidence: verify that active work completes according to the deployment contract and that replacement workers get fresh clients.

Error-path operator workflow

When a request encounters a closed-client or loop-related error, preserve the exception class and lifecycle identifiers before routing again. Check whether a handler closed the shared client, a test retained an old reference, startup was skipped, or background work outlived teardown. Starting another model call through the same invalid client does not repair ownership.

Classify a local lifecycle defect separately from provider health. Determine whether the affected attempt was dispatched before applying retry policy. If dispatch occurred, client closure alone does not establish remote cancellation or zero charges. Contain the worker through the normal readiness and replacement procedure, and reconcile the operation’s outcome before any permitted replay.

For required-resource startup failure, verify that normal traffic is withheld and partial initialization is cleaned up. For teardown failure, retain a distinct close-failed outcome; do not mark closure successful merely because the process subsequently exits.

Verify the lifecycle offline

Save this as lifetime_probe.py beside the factory and run python3 lifetime_probe.py. Every outbound request is intercepted by MockTransport; the reserved .invalid hostname is a fixture.

import httpx
from fastapi import Request
from fastapi.testclient import TestClient

from client_lifetime import create_app


clients = []
sends = []


def factory():
    def reply(request):
        assert request.url.host == "fixture.invalid"
        sends.append(request.url.path)
        return httpx.Response(200, json={"ok": True})

    client = httpx.AsyncClient(transport=httpx.MockTransport(reply))
    clients.append(client)
    return client


app = create_app(factory)


@app.get("/fixture")
async def fixture(request: Request):
    response = await request.state.llm_http.get(
        "https://fixture.invalid/check"
    )
    response.raise_for_status()
    return response.json()


assert clients == []
with TestClient(app) as browser:
    assert len(clients) == 1
    first = clients[0]
    for _ in range(2):
        assert browser.get("/fixture").json() == {"ok": True}
    assert len(clients) == 1 and not first.is_closed
assert first.is_closed

try:
    with TestClient(app) as browser:
        assert len(clients) == 2
        second = clients[1]
        assert second is not first and not second.is_closed
        assert browser.get("/fixture").json() == {"ok": True}
        raise RuntimeError("controlled-scope-failure")
except RuntimeError as exc:
    assert str(exc) == "controlled-scope-failure"
else:
    raise AssertionError("scope failure was lost")

assert second.is_closed and sends == ["/check"] * 3
print("lazy creation, reuse, fresh lifespan, and closure: passed")

The Starlette lifespan reference requires using TestClient as a context manager to run lifespan. Both sessions above do so. The second deliberately raises inside the surrounding test scope to check closure during that scope’s unwinding; it does not simulate a failed model call or a server crash.

A local run on November 2, 2026 with Python 3.12.3, FastAPI 0.115.12, Starlette 0.46.2, HTTPX 0.28.1, and AnyIO 4.16.0 printed the stated pass line. The check established lazy construction, reuse within the first lifespan, a fresh client in the second, and closed clients after both exits. These are tested versions, rather than a claim that they are the latest releases.

The mock does not exercise real sockets, connection reuse, cross-loop transport failures, provider behavior, or production shutdown. Before rollout, test those boundaries on the deployed versions. Include skipped lifespan, required initialization failure, a controlled outbound failure, background work during shutdown, and a request active during worker retirement. Require each resource to have a known owner and terminal outcome.

Failure modes

A request closes the shared client. Later requests fail even though startup succeeded. Handlers must close their responses while leaving client shutdown to the lifespan owner.

A new client appears in every retry helper. Requests complete, but reuse disappears and the configured resource inventory is wrong. Inject the lifecycle-owned client into every attempt path.

A test dispatches a route without startup. The state entry is absent or a fallback global masks the defect. Exercise lifespan explicitly and avoid a hidden global client that makes the test pass.

Old startup hooks are stranded during migration. Adding a lifespan parameter disables legacy event handlers. Migrate dependent initialization and cleanup as one reviewed change.

Worker replication multiplies capacity. A setting intended as a service limit is applied independently to every client. Inventory instances and apply shared admission controls where required.

Teardown races a detached task. The client closes while unowned work is still using it. Register that work with a lifecycle owner and join it before client closure.

FAQ

Is a global AsyncClient always wrong?

HTTPX documents a global client as one way to avoid repeated construction. Its validity depends on the lifecycle that creates, uses, and closes it. Lifespan makes that ownership explicit in an ASGI application; a module variable alone does not establish it.

Does every model need its own client?

No. Choose partitions from transport settings, credentials policy, and required capacity isolation. Creating a client for every model can multiply pools without improving lifecycle ownership.

Does the probe prove connections were pooled?

No. It proves reuse of the client object and its context-managed closure using an in-memory transport. Verify real connection behavior with a controlled network upstream separately.

Will lifespan cleanup always run when a worker stops?

The orderly lifecycle provides a cleanup boundary. Forced process termination can prevent it from finishing or running. Verify server signaling, drain behavior, and termination limits in the actual deployment.

Reader next step

Inventory the HTTP clients in one worker, name each lifespan owner, and locate every call to whole-client closure. Move construction and teardown into that owner, inject the borrowed client into handlers, and run the offline check. Then test an active request across orderly worker retirement before accepting the deployment change.

Sources checked

• ASGI: Lifespan Protocol — fetched November 2, 2026. Supports per-event-loop and per-process lifespan, state transfer, startup completion, and shutdown messages. The page identifies specification version 2.0, dated March 20, 2019.
• FastAPI: Lifespan Events — fetched November 2, 2026. Supports the lifespan context manager, shared-resource initialization and cleanup, replacement of legacy event handlers, and the mounted-subapplication caveat.
• HTTPX: Async Support — fetched November 2, 2026. Supports reusable async clients, context-managed or explicit closure, and separate streaming response closure.
• Starlette: Lifespan — fetched November 2, 2026. Supports the HTTPX lifespan-state pattern, shallow state copying, teardown order, test context management, and state-access version differences.

These current primary technical references support an evergreen implementation task. This guide does not announce a recent release or incident. The ownership audit, capacity inventory, containment workflow, and rollout acceptance criteria are engineering recommendations; the local observations are limited to the versions and mock transport stated above.