pub struct ModelDefectLedger {
counts: Mutex<HashMap<String, ModelDefectCounts>>,
}Expand description
Process-lifetime per-model defect counters.
A synchronous mutex over a small map: every operation is a couple of integer bumps under the lock, on paths that already do far heavier work.
Fields§
§counts: Mutex<HashMap<String, ModelDefectCounts>>Implementations§
Source§impl ModelDefectLedger
impl ModelDefectLedger
Sourcepub fn record_request(&self, model: &str)
pub fn record_request(&self, model: &str)
Count one request for model.
Sourcepub fn record_loop_guard_trip(&self, model: &str, which: LoopGuardTrip)
pub fn record_loop_guard_trip(&self, model: &str, which: LoopGuardTrip)
Count one loop-guard intervention for model, under the detector that
raised it.
Since #1052 an intervention is a note or a refusal — the default forwards the request with a note rather than rejecting it.
Bumps the detector’s own count and loop_guard_trips, which stays the
sum of the two. Also counts the request itself: a trip outside its own
denominator would overstate every rate computed from these numbers.
Scoped to the snapshot, not the client request. Exactly one of this
and Self::record_request runs per snapshot recorded, because the
caller branches on whether the snapshot names a detector. A client
request that is noted and then retried after an upstream death records
two snapshots — the second deliberately carries no trip — so it bumps
requests twice and loop_guard_trips once. That double count of
requests predates this and is the retry path’s, not the guard’s.
Sourcepub fn record_repair(&self, model: &str, succeeded: bool)
pub fn record_repair(&self, model: &str, succeeded: bool)
Count one tool-call repair attempt for model.
Sourcepub fn record_stream_error(&self, model: &str)
pub fn record_stream_error(&self, model: &str)
Count one upstream mid-stream failure for model.
Deliberately does not bump requests, unlike
Self::record_loop_guard_trip, which counts a request the guard
acted on — refused instead of forwarding, or forwarded with a note —
and so has to count its own denominator either way. A stream error
happens after the request was forwarded and already counted; bumping
here would count the same request twice and deflate every rate.
Sourcepub fn record_truncated_generation(&self, model: &str)
pub fn record_truncated_generation(&self, model: &str)
Count one generation cut off at the token ceiling for model.
Sourcepub fn record_empty_response(&self, model: &str, reasoning_only: bool)
pub fn record_empty_response(&self, model: &str, reasoning_only: bool)
Count one turn that produced nothing client-renderable for model.
reasoning_only says whether the model produced reasoning and nothing
else. It is counted within the empty total, not beside it — the turn
was empty either way, and this records why.
Sourcepub fn record_dialect_residue(&self, model: &str)
pub fn record_dialect_residue(&self, model: &str)
Count one turn where dialect markup reached client-visible output.
Sourcepub fn record_unvalidatable_schema(&self, model: &str)
pub fn record_unvalidatable_schema(&self, model: &str)
Count one turn whose tool call could not be validated at all.
Sourcepub fn record_normalization_error(&self, model: &str)
pub fn record_normalization_error(&self, model: &str)
Count one turn whose normalization discarded a malformed tool call.
Sourcepub fn record_identical_result_repeat(&self, model: &str)
pub fn record_identical_result_repeat(&self, model: &str)
Count one turn that repeated the call before it and got an equal result back.
Sourcepub fn record_repeat_rescued(&self, model: &str)
pub fn record_repeat_rescued(&self, model: &str)
Count one turn the guard would have acted on for repeating and did not, because the answer had moved. A repeat still inside the allowance is not.
Sourcepub fn record_repeat_not_evaluated(&self, model: &str)
pub fn record_repeat_not_evaluated(&self, model: &str)
Record that one turn repeated a batch whose results could not be compared.
Sourcepub fn snapshot(&self) -> HashMap<String, ModelDefectCounts>
pub fn snapshot(&self) -> HashMap<String, ModelDefectCounts>
The current counts for every model that has any.