Skip to main content

THINKING_BUDGET_TOKENS_KEY

Constant THINKING_BUDGET_TOKENS_KEY 

Source
pub(crate) const THINKING_BUDGET_TOKENS_KEY: &str = "thinking_budget_tokens";
Expand description

Upstream’s accepted alias for REASONING_BUDGET_TOKENS_KEY.

llama-server reads either name into the same parameter, so a client may legitimately send this one and mean the budget. gglib therefore reads it (see read_reasoning_budget_tokens) and never emits it: the resolved value is force-inserted under the canonical key alone, and this key is erased from every forwarded body by request_pipeline::sampling whatever the trust setting says.

Both halves are load-bearing. Not reading it left an untrusted client’s thinking_budget_tokens: 100000 riding the body past a gate that governs the canonical spelling — the #779 shape, ungoverned and unrecorded. Not erasing it would leave two keys upstream reads as one, with gglib’s own resolved value in only the first: llama-server’s own parse order, not gglib’s ladder, would decide the budget.