Struct ServerConfig 

pub struct ServerConfig {

    pub model_id: i64,
    pub model_name: String,
    pub model_path: PathBuf,
    pub port: Option<u16>,
    pub base_port: u16,
    pub context_size: Option<u64>,
    pub gpu_layers: Option<i32>,
    pub jinja: bool,
    pub reasoning_format: Option<String>,
    pub spec_draft_n_max: Option<u32>,
    pub spec_draft_p_min: Option<f32>,
    pub inference_config: Option<InferenceConfig>,
    pub extra_args: Vec<String>,
}

Configuration for starting a model server.

This is an intent-based configuration — it expresses what the caller wants, not how the server should be started. All typed fields are handled by build_and_spawn(); extra_args is an escape hatch for flags not yet promoted to first-class fields.

Fields

model_id: i64

Database ID of the model to serve.

model_name: String

Human-readable model name.

model_path: PathBuf

Path to the model file.

port: Option<u16>

Port to listen on (if None, a free port will be assigned).

base_port: u16

Base port for allocation when port is None.

context_size: Option<u64>

Context size to use (if None, use model default).

gpu_layers: Option<i32>

Number of GPU layers to offload (if None, use default).

jinja: bool

Enable Jinja templating for chat formats.

reasoning_format: Option<String>

Reasoning format override (e.g., "deepseek", "none").

spec_draft_n_max: Option<u32>

Number of MTP draft tokens to speculate ahead (--spec-draft-n-max).

None means MTP speculative decoding is disabled. When Some(n), --spec-type draft-mtp and --spec-draft-n-max n are passed to llama-server. Recommended value: 2 (Unsloth default).

spec_draft_p_min: Option<f32>

Minimum acceptance probability for MTP draft tokens (--spec-draft-p-min).

Only meaningful when spec_draft_n_max is Some. Skipping low-confidence draft tokens is especially important on Apple Silicon (Metal) to avoid throughput regression. Recommended value: 0.75.

inference_config: Option<InferenceConfig>

Inference sampling parameters (temperature, top_p, etc.).

extra_args: Vec<String>

Additional server-specific options (escape hatch).

Implementations

impl ServerConfig

pub const fn new( model_id: i64, model_name: String, model_path: PathBuf, base_port: u16, ) -> Self

Create a new server configuration with required fields.

pub const fn with_port(self, port: u16) -> Self

Set the port to listen on.

pub const fn with_context_size(self, size: u64) -> Self

Set the context size.

pub const fn with_gpu_layers(self, layers: i32) -> Self

Set the number of GPU layers.

pub const fn with_jinja(self) -> Self

Enable Jinja templating.

pub fn with_reasoning_format(self, format: String) -> Self

Set the reasoning format (e.g., "deepseek", "none").

pub const fn with_spec_draft_n_max(self, n: u32) -> Self

Enable MTP speculative decoding with the given draft token count.

This causes --spec-type draft-mtp and --spec-draft-n-max n to be passed to llama-server. Call Self::with_spec_draft_p_min to also set the acceptance probability threshold (defaults to 0.75).

pub const fn with_spec_draft_p_min(self, p: f32) -> Self

Set the minimum acceptance probability for MTP draft tokens.

Has no effect unless spec_draft_n_max is also set. Recommended value is 0.75; lower values trade quality for speed.

pub const fn with_inference_config(self, config: InferenceConfig) -> Self

Set inference sampling parameters.

pub fn with_extra_args(self, args: Vec<String>) -> Self

Add extra arguments to pass to the server.

Trait Implementations

impl Clone for ServerConfig

fn clone(&self) -> ServerConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ServerConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for ServerConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for ServerConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.