Struct ResidentSlotSnapshot 

pub struct ResidentSlotSnapshot {
    pub slot: usize,
    pub model_name: String,
    pub model_id: u32,
    pub port: u16,
    pub inflight: u32,
    pub is_primary: bool,
    pub resident_for_secs: u64,
}

One model resident in VRAM.

Fields

slot: usize

Slot index. 0 is the primary.

model_name: String

Name of the resident model.

model_id: u32

Database id of the resident model.

port: u16

Port its llama-server is listening on.

inflight: u32

Requests currently holding a lease on this slot.

A slot serving anything at all can never be evicted — a swap must not preempt a live generation.

is_primary: bool

Whether this is the primary slot, i.e. the one chat traffic and the llama.cpp /slots poller follow.

resident_for_secs: u64

Seconds this model has been resident.

Trait Implementations

impl Clone for ResidentSlotSnapshot

fn clone(&self) -> ResidentSlotSnapshot

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ResidentSlotSnapshot

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for ResidentSlotSnapshot

fn eq(&self, other: &ResidentSlotSnapshot) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for ResidentSlotSnapshot

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Eq for ResidentSlotSnapshot

impl StructuralPartialEq for ResidentSlotSnapshot