Module context_fit 

How large a context this machine can actually serve.

gglib up already does this arithmetic: it picks a model by asking what fits in VRAM at 32k and prints the answer as the number that earns the user’s trust. The launch path then resolved its context from a chain whose lowest reachable rung was a flat 4096 and served that instead — so the number shown and the number used were unrelated.

This is the launch-time half of that arithmetic. up’s shortlist still asks a different question — “does this model fit at 32k?” — so the two are not one calculation and cannot be made one cheaply: they are answered against different budgets, and the shortlist runs before the model is downloaded, when its real KV geometry is not yet readable.

What changed is that the banner no longer implies otherwise. It reports 32k as the bar the model had to clear, and says the served context is sized at launch, which is true and knowable. Naming a rung there would have been the same error in the other direction.

Why it snaps to rungs

A resident is identified partly by the context it was launched with, and a request that resolves to a different one evicts and relaunches. A value computed from a live free-memory reading would wobble between requests and recycle the server — blowing the prefix cache and every saved slot file — on essentially every turn. Snapping to a fixed ladder makes the result a step function that changes only when the machine genuinely changes.

Structs

FitInputs

What fit_context worked from, for a person reading a launch log.

Constants

RUNGS 🔒

The context sizes a fitted value is allowed to take.

Functions

fit_context

The largest context weights_bytes can serve inside budget_bytes.

fit_context_explained

The same calculation, reporting what it worked from.

scale 🔒

bytes * factor, saturating and rounding down.