Skip to main content

CACHE_RAM_UNKNOWN_KV_ALLOWANCE_BYTES

Constant CACHE_RAM_UNKNOWN_KV_ALLOWANCE_BYTES 

Source
pub const CACHE_RAM_UNKNOWN_KV_ALLOWANCE_BYTES: u64 = _; // 8_589_934_592u64
Expand description

KV allowance assumed when the model’s metadata doesn’t permit an estimate. Deliberately generous: over-reserving shrinks the cache (safe), whereas under-reserving risks memory pressure.