Struct CacheMetricsStore 

pub struct CacheMetricsStore {
    reporting_requests: AtomicU64,
    unreported_requests: AtomicU64,
    prompt_tokens: AtomicU64,
    cached_tokens: AtomicU64,
    last: AtomicU64,
    has_last: AtomicBool,
}

Running totals of prompt-cache reuse.

Lock-free: every field is an independent atomic, updated with Relaxed ordering. The counters are a display aid, not a consistency boundary — a dashboard tick that reads mid-update sees one request’s figures land slightly out of step, which is invisible at a one-second refresh and not worth a mutex on the request path.

Fields

reporting_requests: AtomicU64

unreported_requests: AtomicU64

prompt_tokens: AtomicU64

cached_tokens: AtomicU64

last: AtomicU64

Last reporting request’s figures, packed as (prompt << 32) | cached so the pair is written in one store and can never be read half-updated (e.g. a new prompt count beside the previous cached count).

Validity is tracked by Self::has_last rather than by an in-band sentinel: u64::MAX looks unreachable but is exactly what a (u32::MAX, u32::MAX) request packs to, which would then read back as “nothing recorded”.

has_last: AtomicBool

Whether Self::last holds a real measurement. Stored with Release after last, and loaded with Acquire before it, so observing true guarantees the packed pair beside it is fully written.

Implementations

impl CacheMetricsStore

pub fn new() -> Self

Create an empty store.

pub fn record(&self, prompt_tokens: u32, cached_tokens: Option<u32>)

Record a completed request’s usage figures.

cached_tokens is None when the upstream didn’t report the field; such a request bumps only CacheUsage::unreported_requests and leaves the token totals untouched.

pub fn snapshot(&self) -> CacheUsage

Snapshot the current totals.

Trait Implementations

impl CacheMetricsSink for CacheMetricsStore

The in-process store is the default CacheMetricsSink: the adapter records straight into the atomics the dashboard snapshots.

fn record(&self, prompt_tokens: u32, cached_tokens: Option<u32>)

Record one completed request’s prompt-token count and how many of those tokens the upstream served from its KV cache.

impl Debug for CacheMetricsStore

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CacheMetricsStore

fn default() -> CacheMetricsStore

Returns the “default value” for a type.