Struct NoopModelRuntime 

pub struct NoopModelRuntime;

A ModelRuntimePort that never has anything running.

For callers with no shared ProcessManager to point at — the CLI’s single-shot commands, whose is_serving checks against a runtime scoped to that one process invocation would report “nothing running” regardless, since nothing was started in it. Making that explicit here is more honest than wiring in a real runner that can only ever agree.

Trait Implementations

impl Debug for NoopModelRuntime

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for NoopModelRuntime

fn default() -> NoopModelRuntime

Returns the “default value” for a type.

impl ModelRuntimePort for NoopModelRuntime

fn admit<'life0, 'life1, 'async_trait>( &'life0 self, _model_name: &'life1 str, _num_ctx: Option<u64>, _default_ctx: Option<u64>, _overrides: LaunchOverrides, ) -> Pin<Box<dyn Future<Output = Result<Admission, ModelRuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Admit a request to a running model, launching or swapping if needed.

fn current_model<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Option<RunningTarget>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Get information about the currently running model, if any.

fn stop_current<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<(), ModelRuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Stop the currently running model.

fn admission_snapshot(&self) -> AdmissionSnapshot

What the admission queue and the VRAM resident set look like right now.

fn list_running<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Vec<ProcessHandle>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Every llama-server process this runtime currently owns.

fn pinned_model(&self) -> Option<String>

The one model this runtime is pinned to, if any.

fn set_pin(&self, pin: Option<PinnedSpec>) -> Result<(), ModelRuntimeError>

Pin this runtime to a single model, or clear the pin.