Module inference 

Inference configuration types.

Defines shared types for configuring LLM inference parameters (temperature, top_p, top_k, max_tokens, repeat_penalty, presence_penalty, min_p).

Tier B — Policy (ADR 0001) for the hierarchy: the ordered fold, profiles, the user-set versus auto-detected split and the class floors are decisions llama-server is structurally not in a position to make, so nothing here gates on RuntimeCapabilities.

with_hardcoded_defaults is the exception, and it is the same shape as ADR 0001’s truncation caveat: the policy of having a floor is gglib’s, but a floor value that equals llama.cpp’s own default is a redundant assertion rather than a decision. Six of the seven were measured to be exactly that. ADR 0003 decides they are deferred, leaving temperature — the one genuine divergence, 0.7 against upstream’s 0.8 — plus reasoning_floor’s class-aware overrides.

This module provides the core InferenceConfig type that is reused across:

• Per-model defaults (Model.inference_defaults)
• Global settings (Settings.inference_defaults)
• Request-level overrides (flattened in ChatProxyRequest)
• gglib proxy — per-request injection into OpenAI-format request bodies
• gglib chat / gglib q — hierarchy resolution for the agentic loop

All surfaces resolve inference parameters through InferenceConfig::resolve_with_profile, which is the single source of truth for the hierarchy. InferenceConfig::resolve_with_defaults is the same resolution with no profile selected, for surfaces that have no notion of one.

Structs

CoupledLayers 🔒

Which ladder rung supplied each member of the temperature-coupled set.

InferenceConfig

Inference parameters for LLM sampling.

ModelSamplingContext

Everything about the target model that changes how sampling resolves, independent of any specific request.

Enums

DefaultsOrigin

Whether a model’s stored inference_defaults were set by the user or written automatically at import time.

FieldIssue

What reading one client-supplied sampling field did, when it was not simply “read it”.

Constants

REASONING_BUDGET_TOKENS_KEY 🔒

Wire key for InferenceConfig::reasoning_budget_tokens, and the only spelling gglib ever emits.

REASONING_EFFORT_KEY 🔒

Wire key for InferenceConfig::reasoning_effort.

THINKING_BUDGET_TOKENS_KEY 🔒

Upstream’s accepted alias for REASONING_BUDGET_TOKENS_KEY.

Functions

brief 🔒

Render a JSON value compactly enough for a log line.

camel_to_snake 🔒

Convert a camelCase string to snake_case.

narrow 🔒

Narrow a JSON number to the f32 every sampling field stores.

read_f32 🔒

Read one float field. Absent and null are both “no opinion”.

read_i32 🔒

Read one integer field.

read_i32_raw 🔒

The integer read behind read_max_tokens, without issue reporting — its caller reports in terms of max_tokens’ own accepted range.

read_max_tokens 🔒

Read max_tokens, which is u32 internally and -1 on the wire.

read_reasoning_budget_tokens 🔒

Read the reasoning_budget_tokens field, under either name upstream accepts for it.

read_reasoning_effort 🔒

Read the reasoning_effort field.

read_seed 🔒

Read the seed field.