Skip to main content

grammar_call_limit

Function grammar_call_limit 

Source
fn grammar_call_limit() -> usize
Expand description

How many tool calls the originated grammar may express in one response.

§Why bound it at all

The rule was root ::= sp call (sp call)* sp. Nothing in * says stop, and nothing else did either: tool_choice must be "none" beside a custom grammar (llama-server accepts no other combination), so the model’s own trained stop behaviour is not in play, and a request carrying no max_tokens has no ceiling below the context window. Measured 2026-08-29: 606 calls in one response for a task expecting one, 853s against 6s unconstrained, scored 1.0 either way because extra calls cost nothing.

§Why the ceiling

Calls past MAX_PARALLEL_TOOLS_CEILING can never be executed — no configured limit may exceed it — so generating them is waste the loop then pays to discard. The grammar should not be able to express what the runtime will certainly reject.

This is the ceiling, not the user’s configured max_parallel_tools, which the request pipeline cannot see: ModelContext carries per-model facts, not agent settings. Threading the live setting through would tighten this further and is the natural follow-up. The env override exists so the bound can be tested against a real model without a rebuild.