Skip to main content

resolve_context_size

gglib_core::server_config

Function resolve_context_size 

Source 
pub const fn resolve_context_size(opts: &ServerConfigOptions) -> u64
Expand description

Resolve context size using the 5-level fallback chain.

  1. Runtime request / CLI flag (opts.context_size) — highest priority
  2. Per-model server defaults (opts.model_server_ctx) — from DB
  3. Global app setting (opts.global_default_ctx) — only when the user set one
  4. Fitted to this machine (opts.fitted_ctx)
  5. Hardcoded default (DEFAULT_CONTEXT_SIZE = 4096) — lowest priority

The fitted value sits below the global default deliberately: a number the user typed outranks one gglib computed. It sits above the built-in floor for the same reason — 4096 is what you serve when you know nothing, and by this rung something is known.