Expand description
What the admission queue and the VRAM resident set look like right now.
The serializable half of admission control. gglib-runtime owns the live
state and the scheduling decisions; this module owns the shape those
decisions are reported in, so gglib-proxy can put them on
GET /v1/proxy/status without depending on the runtime crate.
Everything here is a point-in-time projection, the same way the proxy’s active-connection registry projects itself for the dashboard. Nothing here is authoritative; reading a stale snapshot is always safe.
§Why the reasons are carried, not just the numbers
A user looking at an empty second slot on a card with 12 GB free needs to be
told why. So SecondarySlotStatus carries a stable label for styling
and a ready-to-render sentence, in the same shape
CacheRamHealth established for the prompt
cache. Consumers branch on the label and print the detail; they never parse
the prose.
Structs§
- Admission
Snapshot - Everything the admission queue and resident set look like right now.
- Queued
Model Snapshot - Requests waiting for one model that is not currently resident.
- Resident
Slot Snapshot - One model resident in VRAM.
- Secondary
Slot Status - Why the second VRAM slot is or is not in use.
Functions§
- format_
bytes 🔒 - Render a byte count the way the launch banner does — one decimal, GiB above a gibibyte and MiB below, so “307 MiB” does not print as “0.3 GiB”.