Expand description
Spec-driven delimited tool-call parser.
Tier A — Compensation (ADR 0001). This parser exists because llama-server handed dialect tool calls to the client as raw text. It is not gglib’s job in principle; it is gglib’s job until upstream does it correctly.
Deletion criterion: llama.cpp’s peg-native parser handles the
delimited dialects gglib tags, for every tagged model, and the
failure modes this parser was hardened against no longer reproduce —
specifically a parameter value containing a literal </parameter>
(#24807) and a reasoning model emitting prose before the open marker
(#20260). Evidence is the drift alarm (crate::normalize::residue)
reporting zero residue across a release cycle with this parser bypassed,
not the mere presence of RuntimeFlags::PEG_NATIVE_TOOL_CALLS.
Rewrites OPEN...CLOSE tool-call markup — emitted inside the text or
reasoning channel — into proper ToolCall values, where the envelope
markers come from a DialectSpec rather than being hardcoded. Bytes
outside envelope regions are forwarded verbatim on the channel they
arrived on. The built-in DialectSpec::qwen_xml spec reproduces the
historical Qwen 2 / 2.5 / 3 behaviour (<tool_call> markers); template-
derived specs drive the exact same machine with their own markers.
Body decoding is selected by the spec’s ordered BodyCodec list:
BodyCodec::Json—{"name":"foo","arguments":{...}}(Qwen 2 / 2.5, Hermes, and template-derived dialects).BodyCodec::FunctionXml—<function=NAME><parameter=KEY>VALUE</parameter>...</function>, one or more back-to-back inside a single wrapper (Qwen 3 +--jinja, Hermes-style). The inner markers are codec-internal constants, invariant across models that use the codec.
§Chunk safety
Either marker may straddle SSE chunk boundaries. The parser holds back
at most marker.len() - 1 bytes per channel as a lookahead buffer. The
buffered bytes are flushed on the next push or at
ToolCallParser::finish.
§Cross-channel handling
In practice a tool call appears entirely on one channel — either text
(no reasoning split) or reasoning (when --reasoning-format is on).
Each channel therefore maintains its own independent parser state
(ChannelState) so that markup never crosses channels. The synthesised
tool-call IDs share a single monotonic counter across both channels.
Structs§
- Channel
State 🔒 - Per-channel scanning state. The text and reasoning channels each own one of these; they never share buffers.
- Delimited
Tool 🔒Call Parser - Parser for delimited tool-call dialects, configured by a
DialectSpec. See module docs.
Enums§
- Channel 🔒
- Output channel selector — keeps
scanchannel-agnostic.
Functions§
- finalize_
tool_ 🔒call - Parse the accumulated tool-call body and push the resulting
ToolCalls (or aNormalizationError) ontoout. - find_
own_ 🔒close - Find this tag’s own closing marker inside
rest: the LAST occurrence ofclosebefore the next siblingnext_openmarker (or before the end ofrest, if there is no next sibling). - forward 🔒
- Append
bytesto the channel-appropriate field ofout. - parse_
function_ 🔒xml_ body - Try to interpret
bodyas one or more back-to-back Hermes/Qwen3 inner-XML tool calls:<function=NAME><parameter=KEY>VALUE</parameter>...</function>, repeated. - parse_
json_ 🔒body - Try to interpret
bodyas a Qwen JSON tool call. - parse_
param_ 🔒value - Best-effort coercion of a
<parameter>body to a JSON value. Falls back to a string literal when the body is not valid JSON. - partial_
suffix_ 🔒len - Largest
nin[0, marker.len())such that the lastnbytes ofbufare a prefix ofmarker. Used as the lookahead window for chunk-safe marker detection.