# Perdurance

> Perdurance is a durable proxy for LLM inference. A client sends the request body it would have sent its model provider, to the route it would have sent it to, in the provider's own dialect. Perdurance stores the request, performs the upstream call server-side, persists every stream chunk as it arrives, and records the token usage — so the result survives the client's connection and the client's process.

It exists for three failures that ordinary inference has no answer to. A **dropped connection** loses a generation that was already paid for. A **retry** — including the one a vendor SDK performs by itself — calls the provider a second time and bills twice. And a long generation outlives the socket, the serverless timeout, or the phone that went into a tunnel.

Perdurance answers all three with one mechanism: a request is identified by the bytes of its body, for ten minutes. Re-sending identical bytes attaches to the execution already running instead of starting a second one, so an unmodified OpenAI or Anthropic SDK becomes durable without learning a single Perdurance concept. Any request can also be collected later by id, whole or as a stream replayed from the first chunk and resumable from wherever a reader stopped.

It runs as one container image against one Postgres, and we operate it: as a shared hosted service, or as a deployment dedicated to one customer, in our cloud or in the customer's own cloud account.

## Quickstart

```python
# Perdurance speaks your provider's dialect on your provider's routes. Point an
# unmodified SDK at a namespace and nothing else in your code changes.
from openai import OpenAI

client = OpenAI(
    base_url="https://api.perdurance.dev/{tenancy}/{namespace}/v1",
    api_key=PERDURANCE_KEY,   # sar_ab12cd34_… — issued by Perdurance, not your provider
    max_retries=5,            # safe: an identical retry attaches, it does not re-generate
)

answer = client.chat.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "Say hello."}],
)
```

Or submit without holding a connection, and collect the result whenever you come back:

```bash
# 202 with an id, straight back; the execution runs without you
curl "$URL/requests?dialect=openai" -H "Authorization: Bearer $KEY" \
  -H 'Content-Type: application/json' -d "$BODY"
# → { "request_id": "01J8F2ZK9QX3M4NBVWT7", "status": "pending" }

# collect it — whole, or as a stream replayed from the first chunk
curl "$URL/requests/$ID" -H "Authorization: Bearer $KEY"
curl "$URL/requests/$ID" -H "Authorization: Bearer $KEY" \
  -H 'Accept: text/event-stream' -H 'Last-Event-ID: 25'   # resumes at chunk 26
```

Four things that are easy to get wrong:

- The path is `/{tenancy}/{namespace}/v1`. A request addressed anywhere else will not route.
- Idempotency is over the **bytes** of the body, for ten minutes. A re-serialised body with its
  keys reordered, or a timestamp or fresh UUID in a system prompt, is a different request and is
  billed as one.
- `504 held_too_long` is **not** a failure. The execution is still running and its answer will be
  stored; re-send the identical body, or read it back by id. Submitting a *different* body there
  is the one mistake that starts a second provider call.
- Perdurance does not translate between provider dialects. Reaching an Anthropic model from an
  OpenAI-shaped client is what an `openrouter` or `openai_compat` backend is for.

## Is this the right tool?

### When is Perdurance the right tool?

When a lost connection costs you a generation you have already paid for. Four situations make that common: long generations — reasoning models and agent turns — that outlast a socket, a proxy timeout or a serverless execution limit; clients that disappear, such as mobile, edge hardware and backgrounded browser tabs; retries you cannot switch off because a vendor SDK performs them by default; and any requirement for a per-request transcript and per-key spend as a system of record rather than a dashboard.

### When is Perdurance the wrong tool?

When your calls are short, your clients are stable, and re-running a failed request costs a cent — the failures it removes are ones you are not having, and it is one more hop in your request path. It is also the wrong tool if you want a hosted model, because it has none and you bring your own provider credential; if you need translation between provider dialects, which it deliberately does not do; if what you need made idempotent is your own tools and side effects rather than model calls; or if nobody outside your company may operate what your prompts pass through, because we operate every deployment, dedicated ones included.

### Does putting Perdurance in front of my provider make inference slower?

Marginally, on the path where nothing goes wrong. It is a proxy: there is one more network hop and one row written before the provider is called, which is milliseconds against a generation measured in seconds, and it depends on where it runs relative to you and to your provider. On the path where something does go wrong it is much faster, because a dropped generation is replayed from storage instead of being produced again. Perdurance is not a way to make inference fast; it is a way to stop paying for it twice.

## Documentation

- [Durable LLM inference](https://docs.stg.perdurance.dev/index.md): What Perdurance does to an LLM request: stores it, runs the call server-side, persists every chunk, and hands the result back by id whenever you return.
- [Getting started](https://docs.stg.perdurance.dev/getting-started.md): Point an unmodified OpenAI or Anthropic SDK at Perdurance and read a stored request back out again — from an empty account to a durable LLM call, in six steps.

## Reliability

- [Unstable AI inference](https://docs.stg.perdurance.dev/unstable-ai-inference.md): The six ways an LLM call fails that have nothing to do with the model, what each one costs, and which of them a durable proxy removes.
- [LLM inference retries](https://docs.stg.perdurance.dev/llm-inference-retries.md): Why retrying an LLM inference call normally bills you twice, what a safe retry actually requires, and how to make the retry your SDK already performs idempotent.
- [Agent SDK retries](https://docs.stg.perdurance.dev/agent-sdk-retries.md): What an agent framework's retry actually re-runs, why it costs more than a retried API call, and how to make every model call in the loop idempotent.

## Concepts

- [Model routing](https://docs.stg.perdurance.dev/routing.md): How a model name in a request body resolves to a provider: backends, glob patterns, priority order, and model rewrites.
- [Idempotency and resume](https://docs.stg.perdurance.dev/idempotency.md): Idempotent LLM requests by body hash: why re-sending the same bytes never double-bills, what the ten-minute window is, and how a dropped stream is resumed.

## Reference

- [API reference](https://docs.stg.perdurance.dev/api-reference.md): The submission and retrieval routes for durable LLM inference, SSE replay with Last-Event-ID, and the record a request leaves behind.
- [Administration API](https://docs.stg.perdurance.dev/administration.md): Namespaces, backends, routing rules, members and keys — everything the console drives.
- [Usage and storage](https://docs.stg.perdurance.dev/usage.md): What a namespace has spent per key and per model, what it is holding, and why accepted submissions and provider calls are different numbers.
- [Errors and status codes](https://docs.stg.perdurance.dev/errors.md): Every status this API returns, the stable code that comes with it, and what to do about each — including the 504 that does not mean your request failed.

## Operating

- [The sar CLI](https://docs.stg.perdurance.dev/cli.md): Everything the console does, from a shell — and the one thing it does that the console cannot.
- [Deployment](https://docs.stg.perdurance.dev/deployment.md): What a Perdurance deployment is made of: one container image, one Postgres, and a handful of pointers. Every figure it charges or enforces lives in the database and changes without a restart. Dedicated deployments run in our cloud or in yours, and we operate both.

## Help

- [Frequently asked questions](https://docs.stg.perdurance.dev/faq.md): How to submit, collect, replay and resume a request, whether Perdurance is the right tool, and how the idempotency and retry mechanism works underneath.

## Optional

- [Everything, in one file](https://docs.stg.perdurance.dev/llms-full.txt): every page's prose concatenated, for a single fetch.
- [Questions and worked examples](https://docs.stg.perdurance.dev/faq.md): every question above with code, and a dozen more.