Struct ServerConfig 

pub struct ServerConfig {

    pub model_id: i64,
    pub model_name: String,
    pub model_path: PathBuf,
    pub port: Option<u16>,
    pub base_port: u16,
    pub context_size: Option<u64>,
    pub gpu_layers: Option<i32>,
    pub jinja: JinjaMode,
    pub reasoning_format: Option<String>,
    pub spec_draft_n_max: Option<u32>,
    pub spec_draft_p_min: Option<f32>,
    pub inference_config: Option<InferenceConfig>,
    pub extra_args: Vec<String>,
    pub slot_save_path: Option<PathBuf>,
    pub cache_ram_mb: Option<u64>,
    pub cache_reuse: Option<u32>,
    pub cache_type_k: Option<KvCacheType>,
    pub cache_type_v: Option<KvCacheType>,
    pub mlock: bool,
    pub embeddings: bool,
}

Configuration for starting a model server.

This is an intent-based configuration — it expresses what the caller wants, not how the server should be started. All typed fields are handled by build_and_spawn(); extra_args is an escape hatch for flags not yet promoted to first-class fields.

Fields

model_id: i64

Database ID of the model to serve.

model_name: String

Human-readable model name.

model_path: PathBuf

Path to the model file.

port: Option<u16>

Port to listen on (if None, a free port will be assigned).

base_port: u16

Base port for allocation when port is None.

context_size: Option<u64>

Context size to use (if None, use model default).

gpu_layers: Option<i32>

Number of GPU layers to offload (if None, use default).

jinja: JinjaMode

What this launch says about Jinja templating for chat formats.

See JinjaMode — JinjaMode::Defer emits nothing, which leaves llama-server’s own (on) default in place rather than turning jinja off.

reasoning_format: Option<String>

Reasoning format override (e.g., "deepseek", "none").

spec_draft_n_max: Option<u32>

Number of MTP draft tokens to speculate ahead (--spec-draft-n-max).

None means MTP speculative decoding is disabled. When Some(n), --spec-type draft-mtp and --spec-draft-n-max n are passed to llama-server. Recommended value: 2 (Unsloth default).

spec_draft_p_min: Option<f32>

Minimum acceptance probability for MTP draft tokens (--spec-draft-p-min).

Only meaningful when spec_draft_n_max is Some. Skipping low-confidence draft tokens is especially important on Apple Silicon (Metal) to avoid throughput regression. Recommended value: 0.75.

inference_config: Option<InferenceConfig>

Inference sampling parameters (temperature, top_p, etc.).

Nothing reads this. ADR 0003 deleted to_cli_args and its one caller, so no sampler value becomes a command-line argument any more, and the launch narration reports sampling from llama::args::sampling’s constants rather than from here. The field is written by build_server_config and read by nobody.

Kept for now rather than removed because the plumbing that fills it (ServerConfigOptions::inference_params, threaded from four call sites) is a larger removal than it looks and belongs in its own change. Said out loud so the next reader does not wire something to it on the assumption that it already does something.

extra_args: Vec<String>

Additional server-specific options (escape hatch).

slot_save_path: Option<PathBuf>

Directory for llama-server KV cache slot persistence (--slot-save-path).

None means the disk slot-persistence feature is disabled — no --slot-save-path flag is passed. Independent of Self::cache_ram_mb / Self::cache_reuse: llama-server’s own host-RAM prompt cache can be tuned (or left at its built-in default) regardless of whether disk persistence is on.

cache_ram_mb: Option<u64>

RAM budget in MiB for llama-server’s own host-RAM prompt cache (--cache-ram).

None means no explicit flag is passed — llama-server’s own built-in default (8192 MiB) applies. Some(n) passes --cache-ram n directly; Some(0) disables the cache.

cache_reuse: Option<u32>

Minimum chunk size in tokens for KV-shift cache reuse past the first prefix divergence point (--cache-reuse).

None means no flag is passed (--cache-reuse off, llama-server default 0). Some(n) passes --cache-reuse n, letting llama-server salvage matching KV chunks after an edited/summarized earlier message instead of only reusing an unbroken prefix from token 0.

cache_type_k: Option<KvCacheType>

K cache element type (--cache-type-k). None means no flag is passed — llama-server’s own f16 default applies.

cache_type_v: Option<KvCacheType>

V cache element type (--cache-type-v). Same semantics as Self::cache_type_k.

mlock: bool

Whether to lock the model in RAM (--mlock). Default: false.

embeddings: bool

Whether to serve this model in embedding mode (--embeddings).

This is not an additive flag: llama-server reads it as restrict to only the embedding use case, so a server started with it refuses /v1/chat/completions, and one started without it answers /v1/embeddings with a 501. Resolved from the model’s "embedding" tag, which makes the mode a property of which model is loaded rather than of any individual request.

Implementations

impl ServerConfig

pub const fn new( model_id: i64, model_name: String, model_path: PathBuf, base_port: u16, ) -> Self

Create a new server configuration with required fields.

pub const fn with_port(self, port: u16) -> Self

Set the port to listen on.

pub const fn with_context_size(self, size: u64) -> Self

Set the context size.

pub const fn with_gpu_layers(self, layers: i32) -> Self

Set the number of GPU layers.

pub const fn with_jinja_mode(self, mode: JinjaMode) -> Self

State this launch’s position on Jinja templating.

Takes the mode rather than defaulting to “on” because the caller that has resolved it is the only one that knows which of the two falsy answers it holds — see JinjaMode.

pub const fn with_embeddings(self) -> Self

Serve this model in embedding mode (--embeddings).

See Self::embeddings — this makes the server embeddings-only.

pub fn with_reasoning_format(self, format: String) -> Self

Set the reasoning format (e.g., "deepseek", "none").

pub const fn with_spec_draft_n_max(self, n: u32) -> Self

Enable MTP speculative decoding with the given draft token count.

This causes --spec-type draft-mtp and --spec-draft-n-max n to be passed to llama-server. Call Self::with_spec_draft_p_min to also set the acceptance probability threshold (defaults to 0.75).

pub const fn with_spec_draft_p_min(self, p: f32) -> Self

Set the minimum acceptance probability for MTP draft tokens.

Has no effect unless spec_draft_n_max is also set. Recommended value is 0.75; lower values trade quality for speed.

pub const fn with_inference_config(self, config: InferenceConfig) -> Self

Set inference sampling parameters.

pub fn with_slot_save_path(self, path: Option<PathBuf>) -> Self

Set the KV cache slot-save directory (--slot-save-path).

None disables the disk slot-persistence feature (no --slot-save-path flag emitted). Independent of Self::with_cache_ram_mb / Self::with_cache_reuse.

pub const fn with_cache_ram_mb(self, mb: u64) -> Self

Set the RAM budget (in MiB) for llama-server’s own host-RAM prompt cache (--cache-ram). None leaves llama-server’s built-in default.

pub const fn with_cache_reuse(self, n: u32) -> Self

Set the minimum chunk size (in tokens) for KV-shift cache reuse (--cache-reuse). None leaves the feature off.

pub const fn with_cache_type_k(self, t: KvCacheType) -> Self

Set the K cache element type (--cache-type-k).

pub const fn with_cache_type_v(self, t: KvCacheType) -> Self

Set the V cache element type (--cache-type-v).

pub const fn with_mlock(self) -> Self

Enable memory lock (--mlock).

Trait Implementations

impl Clone for ServerConfig

fn clone(&self) -> ServerConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ServerConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for ServerConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for ServerConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.