Module retry 

Retry

Shared backoff policy for retryable upstream failures.

decide is a pure function: the caller owns the clock (passing elapsed) and the randomness (passing jitter_unit), so the policy has no dependency on a timer, an RNG, or any I/O. Execution — sleeping, re-issuing the request — belongs to the adapter layers that call it.

Two consumers share this one policy so that client and server agree on backoff shape by construction rather than by convention:

• the in-process LLM completion adapter, retrying a 503 from the proxy, and
• the proxy itself, waiting out model-startup contention before it ever emits a 503 to an external OpenAI-compatible client.

Delay derivation

Without a server hint the delay is full jitter — random(0, min(cap, base·2ⁿ)). The failure mode being defended against is several clients colliding on a single model’s startup; full jitter is the variant that decorrelates them most aggressively, where fixed backoff would have every waiter wake together and collide again.

With a Retry-After the server’s value becomes a floor rather than being replaced by jitter — retrying earlier than the server asked only burns an attempt against a resource known to be unready. Jitter of up to initial_backoff is added on top so clients handed an identical Retry-After still spread out, and the floor is clamped to max_backoff so a buggy upstream cannot park a request indefinitely.

let policy = RetryPolicy::from_env();
match retry::decide(&policy, attempt, retry_after, elapsed, retry::jitter_unit()) {
    RetryDecision::Retry { after } => tokio::time::sleep(after).await,
    RetryDecision::GiveUp(reason) => return Err(anyhow!("{}", reason.as_str())),
}

Configuration

RetryPolicy::default() is the tuned budget — 4 attempts inside 60 s. RetryPolicy::from_env() layers the GGLIB_* escape hatches over it, resolved once per process:

Variable	Overrides
GGLIB_LLM_RETRY_MAX_ATTEMPTS	attempts, including the first
GGLIB_LLM_RETRY_DEADLINE_SECS	wall-clock ceiling on the whole sequence

An unset or unparseable value leaves that field alone, so a typo degrades to standard behaviour rather than disabling retry. Shortening the deadline pulls max_backoff down with it, because otherwise the very first backoff would overrun the budget and silently turn retrying off for someone who thought they were only tightening it.

RetryPolicy::disabled() is the one-attempt policy gglib chat --no-retry resolves to — asked for by name rather than by knowing that max_attempts: 1 means “off”.

Modules

Module	LOC	Complexity	Coverage
env.rs
env_tests.rs
jitter.rs
policy.rs
policy_tests.rs

Modules

env 🔒

Operator overrides for the retry budget.

jitter 🔒

Jitter source for decide.

policy 🔒

Pure backoff policy — no clock, no randomness source, no I/O.

Structs

RetryPolicy

Bounds on a retry sequence.

Enums

GiveUpReason

Why a retry sequence stopped.

RetryDecision

The outcome of consulting the policy after a failed attempt.

Constants

DEADLINE_ENV_VAR

Overrides the wall-clock ceiling on a whole retry sequence, in seconds.

MAX_ATTEMPTS_ENV_VAR

Overrides how many attempts a retry sequence may make, including the first.

Functions

decide

Decide what to do after attempt attempts have failed.

jitter_unit

A value in [0.0, 1.0) for use as jitter_unit.