Expand description
Do the emitted tool calls actually match the schemas the client advertised?
Tier B — Policy (ADR 0001). llama-server has no view of what a client
does with a malformed call, so deciding whether to forward one is gglib’s
call regardless of how good upstream’s grammar becomes. Nothing here is
gated on RuntimeCapabilities.
§Why this exists
tool_choice: "auto" is the path every agentic client uses, and on some
model/build pairs llama.cpp installs no grammar for it. Measured on
b10327 (ADR 0002, findings 4-5):
| model | auto conformance | required conformance |
|---|---|---|
| Qwen3.5-4B | 30/30 | 30/30 |
| Llama 3.2 3B | ≤ 4/30 | 30/30 |
On Llama 3.2, 26 of 30 calls put max_lines as the string "42" where the
schema declares an integer. The client’s executor then fails, reports the
error back to the model, and the model tries again — one of the ways a
local agentic session dies, and nothing in gglib noticed it happening.
This module is the detection half. The repair half — re-issuing under a
grammar, upstream’s with tool_choice: "required" or gglib’s own on a turn
it constrained — lives in the proxy, because only it can make a second request.
See Tool-call repair.
§Deliberately not a JSON Schema engine
Only the constraint kinds small models demonstrably get wrong are checked:
types, required, enum, additionalProperties: false, and the same
checks recursively through nested objects and array items.
$ref, anyOf/oneOf/allOf, not, $defs and prefixItems yield
Verdict::Unvalidatable and the response is forwarded untouched, and
pattern is not checked at all. Half-implementing those constructs would
produce false violations, and a false violation costs a wasted generation
and replaces a working call with a re-rolled one.
§Recursion is not optional
The experiment that motivated this module checked nested presence but not
nested types, so options: {"follow_symlinks": "null"} passed a
validator that should have rejected it and the measured conformance rate
came out flattering. Pinned by
validate_tests::a_nested_wrong_type_is_caught so
the same gap cannot reappear where it would cost a real repair.
Structs§
- Violation
- What a single tool call got wrong.
Enums§
- Verdict
- The outcome of validating one response’s tool calls.
- Violation
Kind - The constraint a
Violationbroke.
Constants§
- SUBSCHEMA_
KEYWORDS 🔒 - Where a schema holds subschemas, besides the values of
properties. - UNSUPPORTED_
KEYWORDS 🔒 - Schema keywords this validator does not implement.
Functions§
- check_
object 🔒 - The object-shaped checks:
required, declared properties, andadditionalProperties: false. - check_
value 🔒 - Check
valueagainstschema, appending(pointer, kind)for each violation found at or below this point. - schema_
for 🔒 - The
parametersschema forname, from the advertised tools. - type_
label 🔒 - The
typekeyword as a violation names it:integer, orstring or null. - type_
matches 🔒 - Whether
valuesatisfies one JSON Schema type name. - type_
name 🔒 - The JSON type name of
value, for violation reporting. - type_
satisfied 🔒 - Whether
valuesatisfies atypekeyword, written as one type or a list. - unsupported_
reason 🔒 - The first unsupported keyword in
schemaor in a subschema under it. - validate_
tool_ calls - Validate a response’s
tool_callsagainst the request’stools.