Module request_pipeline

Expand description

§Request Pipeline

LOC Complexity

Request shaping for every inference pipeline: what we know about the model, and what we do to the request because of it.

gglib has two request paths that historically diverged: gglib proxy, which applied a full shaping pipeline, and the agent path used by gglib chat, gglib q and the web UI, which applied almost none of it. Both start from the same question — what do we know about this model? — and both need the same answer applied to the outgoing body. This module is the one place that does either.

§Module map

Routing — which model the request names

profile_route — resolve_route(), which reads a {model}:{profile} suffix off a requested id and decides whether it names a model outright, a model plus a configured profile, or a profile that does not exist.

This one runs before the pipeline rather than inside it. Every stage of apply() shapes a request already known to belong to some model; resolve() cannot even build a ModelContext until the base name is known, and stripping the suffix is what produces that name. So the order for a caller that supports profiles is: route, then resolve the base name it returns, then apply. A caller that does not support profiles skips straight to resolve(), which is why this is a separate entry point rather than a stage — and why an id with no : costs no catalog access at all.

Resolution — what the model is

model_context — ModelContext, the resolved per-model facts (capabilities, format:* tags, inference defaults, context length) that the request and response stages are built from, plus the inert ModelContext::passthrough fallback.
resolve — resolve(), the single catalog round-trip that produces one.
request_shape — carries_tools(), the one thing the stages need to know about the request rather than the model: whether it is asking for a tool call. Read by two stages for different purposes, so it lives in neither.
content — text_len() and for_each_text_mut(), the two shapes a message’s content takes (a string, or an array of parts), read and rewritten in one place for every stage that handles message text.

Shaping — what happens to the request

apply — apply(), the whole ordered pipeline as one call, and the one place the stage order and its rationale are written down. Read this first.
messages — shape_messages(), stages 1–2: reasoning strip and capability coalescing. Everything that rewrites the messages array.
truncation — truncate_history(), stage 3: trimming stale tool results and oversized assistant turns to fit the model’s context budget, and rejecting the request when it cannot be made to fit.
truncation_parts — elide(), the elision of one message in either content shape; kept beside truncation, which is at its file budget.
sampling — resolve_sampling() and SamplingLayers, stages 4–5: the sampling hierarchy, the floor selection (neutral / reasoning / tool-call), and the cache_prompt pin. Everything that touches top-level keys.
effort_gate — stage 5b: deleting a resolved reasoning_effort the model’s observed template does not read, and writing down what was deleted so a surface can say whose setting went nowhere.
sampling_log — no stage of its own; the single sampling resolved debug line, rendered after 5b so it describes what was sent rather than what stage 4 folded. Its module docs carry the argument for that placement.

Every request path calls apply(). The proxy used to run the stages by hand with its own truncation pass spliced between them, because that pass gated on the payload’s size in wire bytes and could reject the request with an axum response — neither of which fits here. Measuring the serialized Value and returning a domain error removed both obstacles, so there is one implementation of the order and no second route to keep in sync.

§The truncation budget

Stage 3 needs a character budget, and it comes from the model: ModelContext::context_budget_chars converts the model’s context length at CHARS_PER_TOKEN_APPROX. There is no floor — a 4,096-token model gets a ~16,000-character budget and a 262,144-token model gets a ~1,000,000-character one — so the same conversation is treated differently on different models, which is the point.

Callers holding better information pass their own number instead. Only one does: gglib-proxy knows the live serving context of the running llama-server and learns a per-model chars-per-token ratio from observed usage frames. That calibration is stateful and tied to the proxy’s request lifecycle, so it stays there.

None means do not truncate, not truncate at zero. An unresolvable model has no context length, and guessing one would risk rejecting a request over a number nobody knows.

§Why the fields travel together

They feed four different stages — capabilities drive request-side transforms, tags drive response-parser selection, defaults are the per-model layer of the sampling hierarchy, context length is the truncation budget — but they all come from one catalog row. Resolving them separately is what produced the split-brain this module exists to close.

Identifier resolution itself is not decided here: resolve() goes through crate::ports::ModelCatalogPort, whose implementations delegate to crate::ports::ModelRepository::get_by_identifier — the workspace’s single lookup-key policy.

§Fallback policy

Exactly one, applied by resolve(): an unresolvable model yields ModelContext::passthrough, so it loses its model-specific handling and nothing else. Unknown models log at debug (routine — clients name models the catalog has never seen); catalog errors log at warn (something is broken).

Shaping inherits it for free: a passthrough context has empty capabilities, so every message-level stage is a no-op, and no per-model defaults, so the sampling hierarchy simply resolves one layer shallower. An unknown model never costs the request itself.

Modules

Module	LOC	Complexity	Coverage
`apply.rs`
`apply_tests.rs`
`constrain.rs`
`constrain_tests.rs`
`content.rs`
`content_tests.rs`
`effort_gate.rs`
`effort_gate_stored_tests.rs`
`effort_gate_tests.rs`
`explain.rs`
`messages.rs`
`model_context.rs`
`profile_route.rs`
`profile_route_tests.rs`
`request_shape.rs`
`resolve.rs`
`sampling.rs`
`sampling_log.rs`
`sampling_log_tests.rs`
`sampling_tests.rs`
`tools.rs`
`truncation.rs`
`truncation_parts.rs`
`truncation_parts_tests.rs`
`truncation_tests.rs`
`validate.rs`
`validate_tests.rs`
`validate_unjudged_tests.rs`

Re-exports§

pub use apply::PipelineReport;
pub use apply::apply;
pub use explain::explain_stored;
pub use profile_route::ModelRoute;
pub use profile_route::resolve_route;
pub use resolve::resolve;
pub use validate::Verdict;
pub use validate::Violation;
pub use validate::ViolationKind;
pub use validate::validate_tool_calls;

Modules§

apply: The ordered request-shaping pipeline, and the one statement of its order.
constrain 🔒: Stage 6: decode-time enforcement of dialect tool calls.
content 🔒: The two shapes a message’s content takes, read in one place.
effort_gate 🔒: Stage 5b: suppress a resolved reasoning_effort when the model’s observed template never reads it — and say so.
explain: Resolving a model’s stored sampling configuration, with provenance.
messages 🔒: Stage 1–2: shaping the conversation itself.
model_context 🔒: The resolved per-model context every request pipeline is built from.
profile_route: Routing of {model}:{profile} request ids.
request_shape 🔒: What kind of turn a request body represents.
resolve: The single catalog round-trip that produces a ModelContext.
sampling 🔒: Stage 4–5: resolving what the model is asked to sample with.
sampling_log 🔒: The one debug! line that describes a request’s whole sampling decision.
tools 🔒: Stage: strip tools from requests to models that cannot call them.
truncation 🔒: Stage 3: history truncation.
truncation_parts 🔒: The elision of one message, in either content shape.
validate: Do the emitted tool calls actually match the schemas the client advertised?

Structs§

ModelContext: Everything a request pipeline needs to know about the target model, gathered in a single catalog round-trip.
SamplingDecision: Everything resolve_sampling decided, and why.
SamplingLayers: The sampling layers that sit below the client’s own request parameters.
SuppressedEffort: A resolved effort level this stage threw away, and where it came from.
TruncationReport: Summary of what truncate_history did to a request body.

Enums§

FloorClass: Which class floor sat beneath the ladder.
TruncationError: The request cannot be made to fit its context budget.

Constants§

CHARS_PER_TOKEN_APPROX: Character-to-token conversion factor used to translate a model’s token context size into the character budget truncate_history measures.
CLIENT_AUTHORITATIVE_KEYS: The client’s own fields that survive an untrusted request.
DISABLE_AGENTIC_SAMPLING_ENV: Environment kill switch for the agentic-turn adjustments.
DISABLE_GRAMMAR_ENV: Environment kill switch. Truthy values (case-insensitive 1, true, yes, on) disable grammar origination entirely — the same contract as GGLIB_DISABLE_MTP and GGLIB_DISABLE_CACHE_REUSE.
LADDER_RUNGS: Rungs in the pipeline’s ladder: cli, client, profile, model, global, model (auto-detected).

Functions§

append_text: Append text to content as a trailing piece of text, in either shape, and say whether it could be.
carries_tools: Whether the request carries a non-empty tools array.
constrain_tool_calls: Originate a decode-time grammar for a demanded dialect tool call.
for_each_text_mut: Apply f to every piece of text in content, in either shape, and say how many pieces it visited.
resolve_sampling: Resolve the sampling hierarchy into body, then pin cache_prompt.
shape_messages: Apply every message-level transform, in order.
strip_unsupported_tools: Remove tools and tool_choice when the resolved model cannot use them.
suppress_stored_effort: Stage 5b’s rule, applied to a resolution with no request in hand.
suppress_unsupported_effort: Remove a resolved reasoning_effort the observed template cannot read, and return what was removed.
text_len: The number of characters of text content carries, in either shape.
text_parts: The number of pieces of text content carries: 1 for a string, one per text part for an array, 0 for anything else.
truncate_history: Trim stale history in place so the request fits within limit_chars, aiming past the budget for the LOW_WATERMARK_PCT watermark once triggered.