Skip to main content

Module defects

Module defects 

Source
Expand description

Per-model defect counters — the Tier C signals the closed loop steers by.

The proxy records defect events (a loop-guard trip, a tool-call repair, a turn that died mid-stream) as they happen; a reader turns those into rates over whatever window it cares about. Writers never interpret and readers never guess: a trip is a fact about one request, a rate is a claim about a model, and the split keeps both honest.

Counters are cumulative and process-lifetime (they live on the proxy supervisor, like the agent cache metrics, so a proxy restart does not zero them). There is no windowing here, and no delta helper: that pair existed for the tune scheduler, which kept per-model baselines and rated the difference. Since ADR 0006 nothing acts on these automatically, and the one reader left — gglib proxy dashboard — shows the run’s totals, which is the honest shape for a counter that resets with the process.

They are diagnosis: what actually fails, per model, for a person to read and act on.

Deliberately not persisted: a defect rate is a claim about recent traffic on this build of everything, and yesterday’s rate answering today’s question is exactly the staleness ADR 0001 warns about. The loop reacts to what is happening, not to what once happened.

That was tried the other way and reverted, so it does not need trying again. Persistence — a defect_windows table, exponential decay by wall-clock age, and outright discard of evidence recorded against a different llama.cpp release — was built to let the idle-time tune scheduler carry evidence across restarts. Decay and build scoping existed only to answer the staleness objection above; they were the price of persisting at all, not features in their own right.

With the scheduler removed, nothing acts on these counts automatically, and sampling defaults now come from the model’s own metadata rather than from measured rates. Nobody was left who needed yesterday’s numbers, so the whole apparatus went rather than sit dormant. These counters are diagnostic, per-process, and reset on restart — which is the correct lifetime for a claim about what is happening now.

One reading has since needed yesterday’s numbers: ADR 0011’s kill criterion asks whether the guard’s trips reach zero across more traffic than one run sees. So the loop guard’s decisions — not these counters — are also written to a log that outlives the process, super::loop_guard_log. It answers the objection above in part: every row is dated, so a person chooses the window; nothing automatic reads it; and every row carries the gglib version and the guard’s mode. It records neither the llama.cpp build nor the model file, which defect_windows scoped by, so a reading that spans either has to be split by date.

Re-exports§

pub use super::defect_counts::LoopGuardTrip;
pub use super::defect_counts::ModelDefectCounts;

Structs§

ModelDefectLedger
Process-lifetime per-model defect counters.