Expand description
Picking a first model that actually fits the machine it will run on.
gglib up has to answer a question a new user cannot: which GGUF should
land on this box. Getting it wrong is worse than not answering — a model
that overflows VRAM does not fail, it swaps to host memory and runs at a
tenth of the speed, which reads as “gglib is slow” rather than “that model
was too big”.
The answer is a small hand-maintained shortlist rather than a live search. Hugging Face has tens of thousands of GGUF repos and no reliable signal for “this one tool-calls properly”; a curated table is deterministic, testable offline, and needs no network round-trip before the confirmation prompt. Its cost — someone has to revisit it as models age — is paid once per release rather than once per user.
Candidates are biased towards models whose tool-call dialect
crate::normalize already parses. Recommending a model gglib cannot
normalize would sell the user the exact failure the proxy exists to fix.
This module decides what to suggest; it does not download, and it has no
opinion on what to do when nothing fits — recommend returns None
and the caller reports the hardware it found.
Structs§
- Model
Candidate - One entry in the shortlist.
- Recommendation
- A candidate plus the reasoning that selected it.
Enums§
- Budget
Source - Which pool of memory the recommendation was sized against.
Constants§
- BUDGET_
UTILISATION 🔒 - Fraction of the memory budget a candidate is allowed to occupy.
Statics§
- SHORTLIST 🔒
- The shortlist, largest first.
Functions§
- recommend
- Recommend the largest shortlisted model that fits this machine.
- resolve_
budget 🔒 - Resolve the memory figure to size against, and say where it came from.