pub fn resolve_sampling(
body: &mut Value,
ctx: &ModelContext,
layers: &SamplingLayers,
) -> SamplingDecisionExpand description
Resolve the sampling hierarchy into body, then pin cache_prompt.
This doc block used to sit above read_client_layer, where a split left
it fused to that function’s own first line — so the entry point of the
whole stage was undocumented while a private helper carried a description
of something else. Restored here; read_client_layer keeps its own.
§Force-insert, not or_insert
The client’s own parameters are extracted from body first, folded
through InferenceConfig::resolve_layers_with_sources alongside cli / profile /
model / global, and the fully-resolved result is then written back over
the top. Client parameters still win — they win by being the
highest-priority layer in the fold, not by surviving an or_insert.
Rewriting this as or_insert looks equivalent and silently breaks the
hierarchy: every layer below the client would stop applying to any key
the client happened to send.
§Client trust
layers.trust_client_sampling gates which of the client’s own fields
enter that layer at all. When false (the default — see
Settings::trust_client_sampling), only CLIENT_AUTHORITATIVE_KEYS
survive; the rest of body’s sampling keys are read but discarded before
the fold, so a client with a hardcoded temperature can no longer outrank
this server’s own configuration, and every field it left unset still
gap-fills from below exactly as if it had never sent that key.
That carve-out is a category, not a list of exceptions, and the two
reasoning controls land on opposite sides of it: the budget is a budget and
survives, the effort level is taste and does not. The reasoning is on
CLIENT_AUTHORITATIVE_KEYS.
The gate covers modelled fields; sampler keys the ladder has no field
for (UNMODELLED_SAMPLER_KEYS) are stripped from the untrusted body
itself, because a key with no layer has nothing to be discarded from and
would otherwise ride the body to llama-server ungoverned.
§What leaves the body
erase_unadopted_client_keys is the one place keys are deleted, and it
deletes three things — only the first of which is about trust:
- What the gate binned — empty when the client is trusted.
thinking_budget_tokens— always. It is upstream’s alias for the budget, gglib reads it and then emits the canonical key alone, and two spellings of one parameter in one body is a disagreement waiting to be resolved by somebody else’s parse order.- A refused
reasoning_effort— on both sides of the gate, and it is the only field anissuesentry removes. Every other refused value is forwarded exactly as before this PR, because upstream 400s on it and that 400 is a better answer to the client than gglib quietly rewriting the request.reasoning_effortis the exception because upstream validates it not at all: a refused"banana"left in the body is not rejected downstream, it is rendered into the prompt.
So a client’s top_k: "5" still reaches llama-server and still earns its
HTTP 400, unchanged by the reasoning work. The helper carries the full
argument and ADR 0007’s finding behind it.
A body that is not a JSON object is left alone.