Struct InferenceConfig 

pub struct InferenceConfig {

    pub temperature: Option<f32>,
    pub top_p: Option<f32>,
    pub top_k: Option<i32>,
    pub max_tokens: Option<u32>,
    pub repeat_penalty: Option<f32>,
    pub presence_penalty: Option<f32>,
    pub frequency_penalty: Option<f32>,
    pub min_p: Option<f32>,
    pub dynatemp_range: Option<f32>,
    pub dynatemp_exponent: Option<f32>,
    pub top_n_sigma: Option<f32>,
    pub dry_multiplier: Option<f32>,
    pub dry_base: Option<f32>,
    pub dry_allowed_length: Option<i32>,
    pub dry_penalty_last_n: Option<i32>,
    pub seed: Option<u32>,
    pub reasoning_effort: Option<ReasoningEffort>,
    pub reasoning_budget_tokens: Option<i32>,
}

Inference parameters for LLM sampling.

All fields are optional to support partial configuration and fallback chains. Intended to be shared across model defaults, global settings, and request overrides.

Hierarchy Resolution

When making an inference request, parameters are resolved in this order:

1. Request-level override (user specified for this request)
2. Selected profile (Settings.inference_profiles, chosen as {model}:{profile}; absent on surfaces without profiles)
3. Per-model defaults (stored in Model.inference_defaults)
4. Global settings (stored in Settings.inference_defaults)
5. Hardcoded fallback (e.g., temperature = 0.7)

Examples

use gglib_core::domain::InferenceConfig;

// Conservative settings for code generation
let code_gen = InferenceConfig {
    temperature: Some(0.2),
    top_p: Some(0.9),
    top_k: Some(40),
    max_tokens: Some(2048),
    repeat_penalty: Some(1.1),
    ..Default::default()
};

// Creative writing settings
let creative = InferenceConfig {
    temperature: Some(1.2),
    top_p: Some(0.95),
    ..Default::default()
};

Fields

temperature: Option<f32>

Sampling temperature (0.0 - 2.0).

Controls randomness in token selection:

• Lower values (0.1-0.5): More deterministic, focused
• Medium values (0.7-1.0): Balanced creativity
• Higher values (1.1-2.0): More random, creative

top_p: Option<f32>

Nucleus sampling threshold (0.0 - 1.0).

Considers only the top tokens whose cumulative probability exceeds this threshold. Common values: 0.9 (default), 0.95 (more diverse)

top_k: Option<i32>

Top-K sampling limit.

Considers only the K most likely next tokens. Common values: 40 (default), 10 (focused), 100 (diverse)

max_tokens: Option<u32>

Maximum tokens to generate in response.

Hard limit on response length. Does not include input tokens.

repeat_penalty: Option<f32>

Repetition penalty (> 0.0, typically 1.0 - 1.3).

Penalizes repeated tokens to reduce repetitive output.

• 1.0: No penalty (default)
• 1.1-1.3: Moderate penalty

• 1.3: Strong penalty (may hurt coherence)

presence_penalty: Option<f32>

Presence penalty (0.0 - 2.0).

Penalizes tokens that have already appeared in the output, encouraging the model to cover new ground. Effective at preventing repetitive reasoning loops in thinking models.

• 0.0: No penalty (default; disabled)
• 1.5: Recommended for reasoning/thinking models (e.g. Qwen3.6, DeepSeek-R1)

• 2.0: Avoid; may degrade coherence

frequency_penalty: Option<f32>

Frequency penalty (-2.0 - 2.0), presence_penalty’s twin.

Penalizes tokens in proportion to how often they have already appeared, where presence_penalty is a flat once-seen offset. An OpenAI-standard field that llama.cpp supports; until it was modelled here it passed through the proxy ungoverned, so an untrusted client could steer sampling with it while every modelled twin was gated (ADR 0003’s frequency_penalty follow-up).

• 0.0: No penalty (llama.cpp’s default)
• Negative values encourage reuse; valid upstream, rarely wanted

Unset defers to llama.cpp’s own default (0.0, disabled).

Deliberately not part of the temperature-coupled trio, although the trio’s rationale (a flat logit offset competing with temperature’s sharpening) applies to it literally. The cost that evicted DRY from the coupled set (#746) applies literally too: coupling would make a layer naming only a frequency_penalty lose it to any lower layer naming a temperature — the default state of every reasoning-tagged model — and no shipped profile pairs a frequency penalty with a temperature, so coupling would protect nothing in exchange. The trio stays a closed set; joining it needs sweep data, not symmetry.

min_p: Option<f32>

Minimum-probability sampling threshold (0.0 - 1.0).

Removes tokens whose probability is below min_p × P(top token).

• 0.0: Disabled (explicit off; recommended by Qwen3.6, and the floor for reasoning-tagged models — see reasoning_floor)
• 0.05: llama.cpp’s own default, and the neutral floor here — see with_hardcoded_defaults

dynatemp_range: Option<f32>

Dynamic-temperature half-range — entropy-adaptive temperature.

llama.cpp scales the effective temperature within [temperature − range, temperature + range] by the entropy of each step’s token distribution: confident (low-entropy) steps decode cooler, uncertain (high-entropy) steps decode hotter. On a completion that mixes free-form reasoning with structured tool-call tokens this is a per-token soft version of phase-aware sampling, with no phase detection required.

• 0.0: Disabled (llama.cpp’s default)
• 0.3–0.75: Exploratory range around a 0.6–1.0 base temperature

Unset defers to llama.cpp’s own default (0.0, disabled).

Deliberately not part of the temperature-coupled trio, for the reason DRY is not (#746): coupling would make a layer naming only a dynatemp_range lose it to any lower layer naming a temperature — the default state of every reasoning-tagged model — which costs the most natural way of switching it on. Unlike the trio, its meaning is anchored to whatever base temperature actually resolves, so an orphaned range stays coherent.

dynatemp_exponent: Option<f32>

Dynamic-temperature exponent, shaping how sharply the effective temperature responds to entropy. Inert while dynatemp_range is unset or zero. Unset defers to llama.cpp’s own default (1.0).

top_n_sigma: Option<f32>

Top-n-sigma: keep only tokens whose pre-softmax logit is within n × σ of the maximum (arXiv 2411.07641). Because it truncates the unscaled logits, the candidate set does not widen as temperature rises — the property that makes it a candidate for keeping tool-call tokens stable while a reasoning model decodes hot.

• ≤ 0.0: Disabled (llama.cpp’s default is −1.0)
• 1.0–2.0: The paper’s evaluated range

Unset defers to llama.cpp’s own default (−1.0, disabled).

dry_multiplier: Option<f32>

DRY (Don’t Repeat Yourself) penalty strength.

Penalises tokens that would extend a sequence already present in the context, which catches the multi-token degenerate loops that repeat_penalty — a flat per-token penalty — cannot see.

• 0.0: Disabled (llama.cpp’s default, and the floor here)
• 0.8: A common starting point for long agentic sessions

Left alone on agentic turns, deliberately. An earlier version forced this to 0 whenever a request carried tools, reasoning that structured output legitimately repeats tokens. Both halves were wrong: llama.cpp’s sequence breakers already default to \n, :, ", * — two of which are pervasive in JSON — and agentic clients send tools on every request, so the pin would have disabled DRY for whole sessions, which is the workload it exists for.

llama.cpp’s fifth DRY parameter, --dry-sequence-breaker, is not modelled: it is a list of strings, and every layer of this hierarchy — merge, coupling, the CLI flags, the settings mirror — is built for scalars. It is also the right lever if DRY is ever seen mangling a tool call, so modelling it is the follow-up, not switching DRY off.

dry_base: Option<f32>

DRY penalty base, the exponent applied per token of matched sequence length. Higher grows the penalty faster on longer repeats. Unset defers to llama.cpp’s own default (1.75).

dry_allowed_length: Option<i32>

Sequence length, in tokens, that DRY tolerates before penalising. Unset defers to llama.cpp’s own default (2).

dry_penalty_last_n: Option<i32>

How far back DRY scans for repeats, in tokens. 0 disables the penalty; llama.cpp resolves negative values against the context size. Unset defers to llama.cpp’s own default (64).

seed: Option<u32>

RNG seed for the sampler.

Unset means llama.cpp draws a fresh random seed per request, which is what it reports as 4294967295 (u32::MAX) in /slots.

Request-scoped by design, like max_tokens

This is the second field in this struct that is not a sampling policy. Every other one answers “how should this model sample?” and is worth storing per model; a seed answers “make this run reproducible”, and a seed stored per model would pin every response that model ever produces to the same text. So:

• no floor names it, on either class floor;
• nothing gglib writes at import names it;
• the CLI, profile and settings surfaces do not expose it.

It lives here anyway, rather than beside the hierarchy, for the reason max_tokens does: this struct is the single thing that becomes the request body (Self::to_openai_json_patch), and a value that reaches the wire any other way is a value the ladder cannot explain and the readback cannot check. llama.cpp reports the applied seed in /slots, so routing it through here is what makes “did my seed actually land?” an answerable question rather than an assumption — which is the whole point of seeding a benchmark.

reasoning_effort: Option<ReasoningEffort>

How hard the model is asked to think — a prompt-shaping template control, not a sampler.

Every other field in this struct configures llama.cpp’s sampler chain. This one does not touch it. It is parsed off the top-level OpenAI body, stored as a Jinja kwarg and handed to the chat template, which may render it into the prompt, may branch on it, or — on most models — may never read the variable at all. It changes what the model is shown, not how its logits are cut.

The wire, measured

Against the pinned build (ADR 0007 finding 7c):

sent	llama-server	why it matters here
"high"	200, kwarg set	the ordinary case
"banana"	200, rendered into the prompt verbatim	nothing upstream validates this field
42	200, kwarg dropped, template default applies	a wrong type degrades silently
""	200, ignored	already means “no opinion”
"none"	200, kwarg erased	on gpt-oss the template’s own medium fallback then fires

An enum makes gglib stricter than upstream, deliberately

extract_client_sampling’s doctrine is to “accept what upstream accepts and reject what upstream rejects, so gglib never becomes the stricter of the two”. A closed ReasoningEffort plainly violates it: llama-server takes "banana" and gglib will not.

The doctrine’s premise is that upstream is the authority on what a field means, so disagreeing with it can only cost a value that would have worked. That premise is false for exactly this field. ADR 0007 finding 7c measured the asymmetry: upstream governs the budget — reasoning_budget_tokens: -2 is a clean HTTP 400 naming the range — and does not govern effort at all. There is no allowlist, no type check, and no check that the loaded template reads the variable. Where upstream has no opinion, gglib’s governance is not the stricter of two; it is the only one there is.

And the failure mode is not a rejected-but-valid value. "banana" does not fail, it renders: the user’s own prompt gains a line reading Reasoning: banana, and the model answers as if a person had typed it. Passing that through is not accepting what upstream accepts, it is forwarding a typo into a prompt. A rejected field costs the client its reasoning_effort and is reported by name (FieldIssue::Rejected); a forwarded typo costs the answer and is reported nowhere.

That trade only exists if the rejection also removes the key, and it is this field that forces the point. Every other reader here rejects only what upstream would have rejected too, so a forwarded reject came back as an HTTP 400 and the damage was visible; and floored fields are overwritten by the resolved patch anyway. This field is neither — no layer or floor ever names it (no_floor_names_a_reasoning_control) and upstream validates nothing — so a rejection that stopped at the layer would be a report with no effect, and "banana" would reach the prompt with gglib’s own client_fields_rejected recording that it had been stopped. See the body cleanup in request_pipeline::sampling, and a_rejected_effort_level_never_reaches_the_wire for the pin.

The narrower rule the doctrine actually encodes still holds and is obeyed by this field’s twin: see reasoning_budget_tokens, which accepts upstream’s full -1..=i32::MAX and rejects precisely what upstream 400s on.

Permanently Blind

Neither this nor the budget appears in /slots or /props; task_params::to_json exports no reasoning field in either branch. ADR 0004’s readback can confirm a top_k arrived and can never confirm this did. The provenance record is the whole account — which is why the field is modelled here rather than left to ride the body ungoverned (ADR 0007 finding 6, the #779 shape under a new name).

No floor names one

See with_hardcoded_defaults. Several templates default themselves (gpt-oss to medium); a floor here would override each template’s own choice with a value nobody made.

reasoning_budget_tokens: Option<i32>

How many tokens of thinking the model is allowed before it is cut off — a budget, not a sampler and not a taste.

It says what this request is (a turn that may spend at most n tokens reasoning), the same category max_tokens occupies, and it is enforced by llama.cpp itself (common/reasoning-budget.{h,cpp}) rather than by a template that may or may not read a variable. That is what puts it on the client-authoritative side of the trust gate while its twin reasoning_effort is gated — see crate::request_pipeline::sampling, which states the split where the carve-out is coded.

The wire, measured — and the range is upstream’s, not gglib’s

Range-validated on the pinned build: -1 <= v <= 2147483647, and -2 comes back as a clean HTTP 400 naming the range (ADR 0007 finding 7c). So the full -1..=i32::MAX is accepted here and only values below -1 are rejected — gglib matches upstream’s own 400 exactly and adds nothing. This is the doctrine at extract_client_sampling applied straight, and it is the reason the departure argued on reasoning_effort is a departure about that field rather than about reasoning controls in general.

• -1 — defer to the launch-time --reasoning-budget default.
• 0 — valid, and the honest spelling of “stop thinking immediately”. It is what gglib offers instead of reasoning_effort: "none", which yields medium thinking on gpt-oss (ADR 0007 decision 4).

A value gglib refuses (below -1) is dropped from the layer and left in the forwarded body, unlike its twin: upstream’s own 400 names this field and its range, which is a better answer to the client than a silent substitution and keeps gglib no stricter than upstream. See FieldIssue::Rejected.

The alias

Upstream also accepts thinking_budget_tokens (THINKING_BUDGET_TOKENS_KEY) as a second spelling of this parameter. gglib reads it — a name it did not read was a name the trust gate could not govern — and emits the canonical one only, erasing the alias from every forwarded body so the two spellings cannot disagree about the resolved value. With both sent, the canonical key wins. See read_reasoning_budget_tokens.

Permanently Blind, like its twin

It is parsed into params.sampling, but task_params::to_json serialises no reasoning_budget_* field, so nothing echoes it in /slots or /props either. ADR 0007’s finding 7a is a correction to that ADR’s own earlier claim; do not re-derive an echo from the request parse table.

No floor names one

See with_hardcoded_defaults. A fleet-wide thinking budget is a tuning decision with measurement behind it, and -1 — the “defer” sentinel — is exactly what omitting the key already means.

Implementations

impl InferenceConfig

pub const fn merge_with(&mut self, other: &Self)

Merge another config into this one, preferring values from other.

For each field, if other has Some(value), use it; otherwise keep self’s value. This is useful for applying fallback chains.

Example

use gglib_core::domain::InferenceConfig;

let mut request = InferenceConfig {
    temperature: Some(0.8),
    ..Default::default()
};

let model_defaults = InferenceConfig {
    temperature: Some(0.5),
    top_p: Some(0.9),
    ..Default::default()
};

request.merge_with(&model_defaults);
assert_eq!(request.temperature, Some(0.8)); // Request value wins
assert_eq!(request.top_p, Some(0.9));      // Fallback to model default

fn resolve_coupled( layers: &[Option<&Self>], temperature: Option<usize>, result: &mut Self, ) -> CoupledLayers

Write the temperature-coupled set into result and report which rung supplied each member.

Split out of resolve_layers_with_sources only for length; that is also where the rule it implements is documented. temperature is the rung that claimed the temperature, if any — the whole coupling rule hangs off whether that is Some.

pub fn resolve_layers_with_sources( layers: &[Option<&Self>], floor: &Self, ) -> (Self, FieldSources)

Resolve an ordered list of sampling layers (highest priority first) into a single fully-resolved config, filling anything still unset from floor, and report which layer supplied each field.

This is the one fold every multi-layer resolution surface goes through: resolve_with_profile wraps it for the simple request/profile/model/global shape, and crate::request_pipeline::sampling builds its own six-layer (cli, client, profile, model, global, model auto-detected) array and calls it directly. There is exactly one place that decides what “wins” means.

Values and provenance come from one pass over one ladder and so cannot disagree — a second function that re-derived the rules would eventually explain a decision the resolution did not take, which is exactly what the describe_provenance helper this replaced had already started doing. See FieldSources for how to read the second half of the return; callers wanting only the values take .0.

Uncoupled parameters

top_p, top_k, max_tokens and the four DRY parameters gap-fill independently: each takes the first Some value found scanning the layers top to bottom.

Coupled parameters

presence_penalty, repeat_penalty and min_p are only meaningful relative to how sharp the sampling distribution is, so they travel with the temperature they were chosen for. reasoning_profile pairs temperature 1.0 with presence_penalty 1.5 deliberately; a sparse profile that sets temperature 0.2 and leaves the penalty unset must not inherit that 1.5 — that would run a recipe no layer ever intended, a penalty tuned for a broad distribution applied to a near-greedy one.

So: temperature resolves to the first layer that sets one. If some layer does, the coupled trio comes only from that same layer — never a layer beneath it — falling to floor for anything that layer itself left unset. If no layer sets a temperature at all, nothing has been tuned against anything, so the trio gap-fills normally, exactly like the uncoupled parameters.

Why DRY is not coupled

It was, briefly, on the symmetry argument that a repetition penalty is a repetition penalty. Verification showed the symmetry is false and the cost is real. presence_penalty and repeat_penalty are flat logit offsets competing directly with temperature’s sharpening; DRY’s strength is governed by its own dry_base and dry_allowed_length, and it targets verbatim sequence repetition — a failure mode that is worse at low temperature, not milder.

Coupling it meant a layer naming a DRY value but no temperature lost that value silently whenever any lower layer named one, which is the default state of every reasoning-tagged model. Since no shipped profile and not reasoning_profile itself pairs a temperature with DRY values, the coupling protected nothing and cost the most natural way to switch DRY on. See #745.

pub const fn with_hardcoded_defaults() -> Self

The floor beneath every sampling ladder: what gglib asserts when no layer named a value.

It asserts one parameter, not seven

ADR 0003 measured this floor against a bare llama-server on the pinned build and found six of its seven values were exactly the upstream default:

  parameter          gglib floor   upstream   verdict
  temperature                0.7        0.8   DIVERGES -> policy
  top_p                     0.95       0.95   EQUALS   -> deleted
  top_k                       40         40   EQUALS   -> deleted
  repeat_penalty             1.0        1.0   EQUALS   -> deleted
  presence_penalty           0.0        0.0   EQUALS   -> deleted
  min_p                     0.05       0.05   EQUALS   -> deleted
  dry_multiplier             0.0        0.0   EQUALS   -> deleted

Restating a value that is already the answer is not a decision, it is a redundant assertion — and a costly one, because it silently overrides whatever upstream chooses next. #739 was exactly that failure: a floor of min_p: 0.0 disabled the tail cut on every untuned request, and nothing in the system was positioned to notice. Six such overrides are now impossible.

The six are deferred, not disabled. Nothing is emitted for them, so llama.cpp applies its own default — which on this build is the same number that used to be written here. Provenance reports them as ParamSource::Unset, which is precisely what deferral is: gglib names no value.

temperature: 0.7 stays, and upstream’s is 0.8

The one genuine policy choice in the set, and stated here because an undocumented divergence is how the other six became invisible in the first place. gglib decodes slightly more conservatively than llama.cpp’s default for agentic work.

The floor is no longer uniform

reasoning_floor still asserts presence_penalty: 1.0 and min_p: 0.0 for reasoning-tagged models, which are class-aware policy llama.cpp has no notion of. So after this change min_p is asserted for reasoning models and deferred for every other model. That is the correct shape and it needs saying out loud, because it is the first time the floor has differed by model class in what it names rather than only in what it names it as.

Deferral is safe only while the build is pinned

ADR 0002 pins the llama.cpp build; ADR 0004’s /props baseline check reads the default table back and flags any field that moves. That pairing is what makes deleting a value behaviour-preserving rather than hopeful.

max_tokens has no fallback

It is deliberately None. Resolution force-writes every Some field into the outgoing request, so a value here would cap every request that did not name its own — silently truncating long answers. Left unset, no max_tokens key is emitted and llama-server applies its own n_predict default of -1, generating until a stop token or the context limit.

Omitting the key is exactly equivalent to sending -1 (llama.cpp’s has_budget() treats -1 as limitless) and is the better of the two: max_tokens: -1 is invalid under the OpenAI schema, which requires a positive integer, so a strict client or intermediary proxy may reject it. Omission keeps the forwarded body OpenAI-legal.

Explicit per-request, per-profile, and per-model values are unaffected — reasoning_profile still sets its own ceiling.

A note on min_p, because it moved twice

#739 changed it from 0.0 to 0.05, correctly: 0.0 reads like an absence but was not one — to_openai_json_patch drops only None, so the floor was explicitly disabling the tail cut on every untuned request. The fix was right, and the mechanism it used was the problem. #739 restated upstream’s value to keep it “visible as min_p=floor in sampling provenance instead of reporting as unset”, which bought visibility at the price of a permanent silent override. Deferral is the better answer to the same objection: it reports as unset because it is unset, and ADR 0004’s readback names llama.cpp’s own number instead of gglib restating it.

pub const fn reasoning_floor() -> Self

The coupled-trio floor for models tagged reasoning.

resolve_layers_with_sources falls back to a floor once it has decided which layer (if any) claims the coupled set and that layer left a field unset. with_hardcoded_defaults’s neutral presence_penalty: 0.0 is the right floor for most models, but wrong for a reasoning-tagged one: those degrade under greedy or near-greedy decoding into repetitive reasoning loops (see reasoning_profile, which pairs presence_penalty: 1.5 with temperature: 1.0 specifically to prevent this). 1.0 keeps a real guard in place at the floor without asserting the full recipe tuned for a different temperature.

min_p is pinned to 0.0 for the same class-specific reason: Qwen3.6’s published guidance is to disable min-p on these models, which reasoning_profile already encodes.

These two are now the only class-specific assertions

The neutral floor used to name min_p: 0.05 and presence_penalty: 0.0, so this function read as “the same seven values, two of them different”. ADR 0003 deferred both of those to llama.cpp, so it now reads as “two values the neutral floor does not name at all”.

The consequence is worth stating because it makes the floor non-uniform in a way it never was: min_p is asserted for reasoning models and deferred for everything else. A reasoning model gets min_p: 0.0 on the wire; every other model gets no min_p key and llama.cpp’s own 0.05. That asymmetry is deliberate — one is a measured divergence from upstream, the other is agreement with it — but it will look like a bug to anyone diffing two requests without this paragraph.

presence_penalty: 1.0 is the same shape: asserted here, deferred elsewhere.

pub const fn agentic_temperature_ceiling(is_reasoning: bool) -> Option<f32>

The highest temperature an agentic turn should decode at, when the model’s class has one.

A turn that carries tools may emit structured output. This is the ceiling that caps its temperature — applied by crate::request_pipeline::sampling after resolution, and only over a value nobody deliberately chose. It never raises a temperature.

Reasoning models have no ceiling — measured, not argued

A reasoning model does not decode its tool call in isolation: the <think> block and the call are one completion under one sampler configuration, so a cap imposed for the sake of structured output lands on the reasoning phase too. This shipped as a 0.6 cap (inside the Qwen3 / DeepSeek-R1 recommended band), and ADR 0004’s addendum named the evidence that would justify changing it. That experiment ran on 2026-08-10 (tune runs #12–#32, Qwen3.5-4B Q8_0, 20 paired runs of the full agentic suite per arm):

• Recipe temperature 1.0 uncapped beat the 0.6 cap on the paired composite 11W–4L–5T, mean +0.067, Wilcoxon one-sided p = 0.0099, bootstrap 95% CI [+0.017, +0.116].
• The cost the cap existed to prevent never materialised: tool-call formatting tasks passed 100% at 1.0 versus 98.6% at 0.6.
• The failure the cap was risking did: loop/stagnation triggers were more frequent under the cap (29/126 vs 22/117) — cooling a thinking model manufactures the repetition its own vendors warn about, which the proxy’s loop guard then acts on.

So a reasoning model’s resolved temperature stands on agentic turns, which in the shipped default means its auto-detected recipe’s 1.0.

0.3 for everything else — unmeasured, unchanged

The non-reasoning cap predates that experiment and no non-reasoning model has been measured against it. It keeps its old rationale (steady structured output without being greedy) and its old value until it earns the same treatment: evidence, not argument.

Why a ceiling and not a floor

The floor this replaced could never fire on the models that most needed it. A reasoning-tagged model carries an auto-detected recipe naming temperature: 1.0, and any layer outranks a floor — so the adjustment was inert on precisely the models used for agentic coding. A ceiling gated on provenance fires there and stays out of the way everywhere a person actually made a choice.

pub const fn reasoning_profile() -> Self

Return a recommended InferenceConfig profile for reasoning / thinking models.

Applied automatically at import time when the "reasoning" capability tag is detected (e.g. Qwen3.6, DeepSeek-R1, QwQ). Values follow the Qwen3.6 upstream guidance for thinking mode — general tasks and are conservative enough to work well across all thinking-capable models.

Parameter	Value	Rationale
temperature	1.0	Recommended thinking-mode baseline
top_p	0.95	Broad nucleus; standard for reasoning
top_k	20	Tighter than the 40 fallback; suppresses low-quality tokens
max_tokens	8192	Safe out-of-the-box ceiling; increase for complex tasks
repeat_penalty	1.0	No penalty; presence_penalty handles anti-repetition
presence_penalty	1.5	Prevents repetitive reasoning loops
min_p	0.0	Explicitly disabled per Qwen3.6 spec

Users can override any parameter with gglib model update <id> --<flag> or the equivalent UI control.

pub fn resolve_with_defaults( self, model: Option<&Self>, global: Option<&Self>, model_ctx: ModelSamplingContext, ) -> Self

Resolve inference parameters using the 4-level hierarchy.

Equivalent to resolve_with_profile with no profile selected — see there for the merge order. This is the entry point for surfaces that have no notion of a named profile (gglib serve, gglib chat, gglib q, the Web UI chat API).

model_ctx carries the two facts about the target model that change how resolution behaves — see ModelSamplingContext, resolve_layers_with_sources and reasoning_floor.

Example

use gglib_core::domain::{InferenceConfig, ModelSamplingContext};

let request = InferenceConfig { temperature: Some(0.9), ..Default::default() };
let model   = InferenceConfig { temperature: Some(0.5), top_p: Some(0.8), ..Default::default() };

let resolved = request.resolve_with_defaults(Some(&model), None, ModelSamplingContext::default());
assert_eq!(resolved.temperature, Some(0.9)); // request wins
assert_eq!(resolved.top_p,       Some(0.8)); // model fills in
assert_eq!(resolved.top_k,       None);      // no layer named it, and
                                             // the floor defers top_k
                                             // to llama.cpp (ADR 0003)

pub fn resolve_with_profile( self, profile: Option<&Self>, model: Option<&Self>, global: Option<&Self>, model_ctx: ModelSamplingContext, ) -> Self

Resolve inference parameters using the full 5-level hierarchy.

Applies fallback layers in order, with each layer filling only None fields from self — explicit values are never overwritten:

1. self — caller-supplied overrides (request params, CLI flags, etc.)
2. profile — the named profile the request selected, if any
3. model — per-model stored defaults, if user-set
4. global — global settings defaults
5. model again, if auto-detected — see below
6. the model-class floor — reasoning_floor when model_ctx.is_reasoning, otherwise with_hardcoded_defaults

This is the single source of truth for inference parameter resolution across every gglib surface that does not need its own layer set; resolve_with_defaults delegates here so there is exactly one merge order to reason about and to test. crate::request_pipeline::sampling needs a sixth rung (the client’s own request, sitting below the CLI override rather than between self and profile) and calls the underlying resolve_layers_with_sources directly for that reason — the merge semantics are identical either way.

Why the profile sits above the model

Selecting model:coding is an explicit act by the caller, so it has to beat the model’s stored defaults or it would appear to do nothing on any model that has them. Because profiles are sparse (see crate::domain::inference_profile), outranking the model layer costs nothing for parameters the profile does not set — those still resolve from the model, which is what keeps one global profile safe to apply across differing architectures.

Why model can rank below global

model is only ever a stand-in for Model.inference_defaults, which gets written two different ways (see DefaultsOrigin): a person tuning it deliberately, or gglib’s own import-time guess for any model tagged reasoning. Those deserve different authority. A deliberate per-model choice should keep outranking the operator’s global defaults — that is what “per-model” means. A guess nobody reviewed should not: it silently shadowed the user’s own configured global settings, which is how #685 happened. model_ctx.defaults_origin decides which rung model occupies for this call — never both at once, since only one of rungs 3 and 5 is ever populated for a given model.

Temperature-tuned parameters do not fall through

See resolve_layers_with_sources for the full rule. In short: once a layer declares a temperature, lower layers may not contribute presence_penalty, repeat_penalty or min_p — those resolve from the claiming layer alone, falling to the class floor if it left them unset.

Example

use gglib_core::domain::{InferenceConfig, ModelSamplingContext};

// A sparse profile: sets temperature, says nothing about anything else.
let profile = InferenceConfig { temperature: Some(0.2), ..Default::default() };
// A thinking model's stored defaults: 1.5 is tuned for temperature 1.0.
let model = InferenceConfig {
    temperature: Some(1.0),
    presence_penalty: Some(1.5),
    top_k: Some(20),
    ..Default::default()
};
let model_ctx = ModelSamplingContext { is_reasoning: true, ..Default::default() };

let resolved = InferenceConfig::default()
    .resolve_with_profile(Some(&profile), Some(&model), None, model_ctx);

assert_eq!(resolved.temperature,      Some(0.2)); // profile beats model
assert_eq!(resolved.presence_penalty, Some(1.0)); // reasoning floor, NOT the model's 1.5
assert_eq!(resolved.top_k,            Some(20));  // untuned: still fills

pub fn resolve_with_profile_explained( self, profile: Option<&Self>, model: Option<&Self>, global: Option<&Self>, model_ctx: ModelSamplingContext, ) -> (Self, FieldSources)

resolve_with_profile plus a record of which rung supplied each field.

This is the implementation; resolve_with_profile delegates here and discards the provenance, so the ladder — including the user/auto rung split and the floor selection — is built exactly once.

Map a ParamSource::Layer index back to a rung with SamplingLayer::from_index, which is kept beside this ladder for that purpose.

pub fn extract_client_sampling(value: &Value) -> (Self, Vec<FieldIssue>)

Parse inference parameters from an OpenAI-format JSON body (snake_case keys), plus what the read had to reject or normalise.

Missing keys, explicit nulls and keys this type does not model all yield None for that field and leave the rest untouched. This is the inverse of Self::to_openai_json_patch; a caller with nothing to report on the rejections takes .0.

One bad field must not cost the other ten

This read used to camel-case the whole body and hand it to serde_json::from_value(..).unwrap_or_default(). Serde parses an object as a unit, so a single wrongly-typed key failed the whole deserialise and unwrap_or_default() returned an all-None config — silently discarding every sampling value the client sent, with no log and no test covering the failure path.

Reading field by field means a bad max_tokens costs max_tokens and nothing else.

The coercion policy is upstream’s, not ours

ADR 0003 finding 6 measured what llama.cpp itself accepts on the pinned build, so this does not have to invent a policy:

sent	llama.cpp	here
max_tokens: -1	200	Normalised to None — omission already means “no limit”
top_k: 40.0	200	accepted as 40; a fractional float is rejected
temperature: "0.7"	400	Rejected — a numeric string is a client bug, and quietly parsing it teaches nobody

The principle is to accept what upstream accepts and reject what upstream rejects, so gglib never becomes the stricter of the two on a value that would have worked. Before this change it was: llama.cpp takes max_tokens: -1 and gglib threw away the entire layer over it.

One field departs from it, and says so

reasoning_effort is read against a closed enum, which llama-server is not: it validates that field not at all and renders "banana" into the prompt. The departure is argued on the field itself rather than here, because the argument is about that field’s measured wire behaviour and not about coercion in general — its twin reasoning_budget_tokens follows the principle exactly, reproducing upstream’s own 400 boundary and nothing narrower.

pub fn to_openai_json_patch(&self) -> Map<String, Value>

Serialise as an OpenAI-format JSON patch (snake_case keys, Some fields only).

Uses serde to produce the camelCase form, then renames each key to snake_case via camel_to_snake. Only Some fields are emitted — None values are filtered out. The returned map can be merged directly into an OpenAI-compatible request body with body_obj.insert(k, v).

This is the inverse of extract_client_sampling.

Trait Implementations

impl Clone for InferenceConfig

fn clone(&self) -> InferenceConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for InferenceConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for InferenceConfig

fn default() -> InferenceConfig

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for InferenceConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for InferenceConfig

fn eq(&self, other: &InferenceConfig) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for InferenceConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for InferenceConfig