Module cache_budget 

Auto-sizing math for llama-server’s host-RAM prompt cache (--cache-ram).

Extracted from server_config so the pure budget arithmetic lives alongside the rest of the domain’s pure calculations, with its own focused test suite.

Enums

CacheRamHealth

How healthy a resolved --cache-ram budget is, for user-facing display.

Constants

CACHE_RAM_FLOOR_BYTES

Below this, a prompt cache holds too little to be worth the memory pressure, so the budget collapses to 0 (explicitly disabled).

CACHE_RAM_HEADROOM_BYTES

RAM reserved for the OS, other applications, and llama.cpp’s own compute/scratch buffers — never handed to the prompt cache.

CACHE_RAM_LOW_WATERMARK_BYTES

Threshold below which a working prompt cache counts as cramped.

CACHE_RAM_UNKNOWN_KV_ALLOWANCE_BYTES

KV allowance assumed when the model’s metadata doesn’t permit an estimate. Deliberately generous: over-reserving shrinks the cache (safe), whereas under-reserving risks memory pressure.

Functions

classify_cache_ram

Classify a resolved --cache-ram budget.

compute_auto_cache_ram_mb

Compute the auto --cache-ram budget, in MiB.