gglib_core/ports/
usage_sink.rs

1//! Outbound port for recording per-request token usage.
2
3/// A sink for one completed request's token usage.
4///
5/// The recording site — the in-process LLM adapter's response stream — writes
6/// each completed request's usage here without knowing where it lands. The
7/// proxy's dashboard implements it with an in-memory
8/// [`CacheMetricsStore`](crate::cache_metrics::CacheMetricsStore), which keeps
9/// only the prompt-cache figures; the benchmark harness implements it with a
10/// completion-token accumulator, so a task whose agent loop aborts still
11/// reports the tokens it burned. A process that wants neither passes no sink at
12/// all, so recording becomes a no-op. A future cross-process reporter — a
13/// `gglib chat` run posting its usage to a running proxy — would be another
14/// implementation behind this same seam, needing no change to the adapter.
15///
16/// A request whose upstream reported no usage at all records nothing — the
17/// recording site simply does not call this — so "never called" is how absence
18/// is expressed, and every reported figure is a real measurement.
19///
20/// `cached_tokens` keeps the `Option<u32>` absent-vs-zero distinction: `None`
21/// means the upstream didn't report the field, `Some(0)` means a real full
22/// re-prefill. Implementations must not collapse the two.
23pub trait UsageSink: Send + Sync {
24    /// Record one completed request's token usage: the prompt-token count, how
25    /// many tokens the model generated, and how many of the prompt tokens the
26    /// upstream served from its KV cache.
27    fn record(&self, prompt_tokens: u32, completion_tokens: u32, cached_tokens: Option<u32>);
28}
gglib_core/ports/usage_sink.rs

gglib_core/ports/
usage_sink.rs
