pub struct FitInputs {
pub budget_bytes: Option<u64>,
pub weights_bytes: Option<u64>,
pub kv_bytes_per_token: Option<u64>,
pub trained_ctx: Option<u64>,
pub unsnapped: Option<u64>,
}Expand description
What fit_context worked from, for a person reading a launch log.
The two constants governing this — BUDGET_UTILISATION and the caller’s
co-resident reservation — are judgement calls, not measurements. Nothing
acts on this record; it exists so the numbers behind a fitted context are
visible when someone asks whether those judgements were right, rather than
having to be re-derived from the rung alone.
gglib model explain is where a person reads it, through
residency::explain::explain_fit, which supplies the same two budgets
admit does. Before that it reached only a debug! line written after a
launch, which is not a reading anyone could take across a catalog — and
ADR 0009’s first kill criterion needs exactly that.
Fields§
§budget_bytes: Option<u64>Device memory the fit was allowed to spend against.
weights_bytes: Option<u64>The model’s weights, as summed across shards.
kv_bytes_per_token: Option<u64>Bytes of KV cache each token of context costs at the resolved types.
trained_ctx: Option<u64>The model’s trained ceiling.
unsnapped: Option<u64>The context that fit before snapping to a rung — the difference between this and the chosen rung is what the ladder costs.