pub fn resolve_sampling(
body: &mut Value,
ctx: &ModelContext,
layers: &SamplingLayers,
)Expand description
Resolve the sampling hierarchy into body, then pin cache_prompt.
§Force-insert, not or_insert
The client’s own parameters are extracted from body first, folded
through InferenceConfig::resolve_layers alongside cli / profile /
model / global, and the fully-resolved result is then written back over
the top. Client parameters still win — they win by being the
highest-priority layer in the fold, not by surviving an or_insert.
Rewriting this as or_insert looks equivalent and silently breaks the
hierarchy: every layer below the client would stop applying to any key
the client happened to send.
§Client trust
layers.trust_client_sampling gates which of the client’s own fields
enter that layer at all. When false (the default — see
Settings::trust_client_sampling), only max_tokens survives; the rest
of body’s sampling keys are read but discarded before the fold, so a
client with a hardcoded temperature can no longer outrank this
server’s own configuration, and every field it left unset still
gap-fills from below exactly as if it had never sent that key.
A body that is not a JSON object is left alone.