Module residency 

Whether a second model may stay resident in VRAM alongside the first.

One model at a time is the safe default, not the good one. An embedding model is often two orders of magnitude smaller than the chat model it keeps displacing: nomic-embed-text is ~275 MB against a 9 GB coder model, and on a 16–24 GB card there is room for both several times over. Every swap between them costs a full process teardown, weight reload, and prompt re-prefill — paid twice per alternation, for want of a few hundred megabytes.

This module answers the one question that decides whether that cost is paid: given what is free right now, can this candidate simply stay loaded too?

It is deliberately pure. The live VRAM figure is supplied by the caller (gglib_runtime::system::free_gpu_memory_bytes), so the arithmetic is testable on a machine with no GPU at all — which is every CI runner this workspace has.

What it does not do

It does not decide which model is a good co-resident, and it consults no tags. A model earns the second slot by fitting, full stop. That keeps the rule honest: a 275 MB embedding model and a 900 MB title generator are the same problem, and a 7B chat model is refused by the ceiling rather than by a category judgement that would be wrong as often as it was right.

It also does not model host RAM. The secondary’s --cache-ram budget is the caller’s problem (see gglib_runtime’s residency module), because that figure depends on what the primary already took.

Structs

SlotFootprint

What one resident model is expected to occupy in VRAM.

Enums

SecondarySlotDecision

Whether a candidate may take the second resident slot, and why not when it may not.

Constants

RESIDENCY_UTILISATION

Fraction of free VRAM a co-resident candidate is allowed to claim.

SECONDARY_MAX_BYTES

Hard ceiling on a co-resident model’s footprint, regardless of free VRAM.

Functions

decide_secondary_slot

Decide whether candidate may stay resident alongside what is already loaded.