Module cache_config 

KV-cache configuration types: quantized cache types and the host-RAM prompt cache setting.

Kept as a standalone, low-complexity module (not folded into server_config) so cache-related config resolution has one home.

Enums

CacheRamSetting

How to determine the host-RAM prompt cache budget (--cache-ram).

KvCacheType

A llama.cpp KV-cache element type, as accepted by --cache-type-k / --cache-type-v.

Constants

DEFAULT_CACHE_TYPE_K

Default K cache type: quantized to roughly halve the KV footprint versus f16, doubling how much conversation history the RAM/disk prompt caches can hold.

DEFAULT_CACHE_TYPE_V

Default V cache type. Same rationale as DEFAULT_CACHE_TYPE_K; see the module docs on the Flash Attention requirement for quantized V.