Enum CacheRamSetting 

pub enum CacheRamSetting {
    Auto,
    ExplicitMb(u64),
}

How to determine the host-RAM prompt cache budget (--cache-ram).

Deliberately a two-state enum rather than Option<u64>: benchmark launches (which must never gain a prompt cache — it would perturb throughput measurements and RAM footprint) pass ExplicitMb(0), which unambiguously disables the cache rather than leaving it to an implicit “no value” state.

Variants

Auto

Compute a budget from system RAM, model size, and the KV estimate. The default variant — every launch surface auto-sizes unless it opts out.

ExplicitMb(u64)

Use exactly this MiB value. 0 disables the cache.

Trait Implementations

impl Clone for CacheRamSetting

fn clone(&self) -> CacheRamSetting

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CacheRamSetting

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CacheRamSetting

fn default() -> CacheRamSetting

Returns the “default value” for a type.

impl PartialEq for CacheRamSetting

fn eq(&self, other: &CacheRamSetting) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for CacheRamSetting

impl Eq for CacheRamSetting

impl StructuralPartialEq for CacheRamSetting