Skip to main content

ModelDefectCounts

Struct ModelDefectCounts 

Source
pub struct ModelDefectCounts {
Show 16 fields pub requests: u64, pub loop_guard_trips: u64, pub loop_guard_loops: u64, pub loop_guard_stagnations: u64, pub repairs_attempted: u64, pub repairs_succeeded: u64, pub stream_errors: u64, pub truncated_generations: u64, pub empty_responses: u64, pub reasoning_only: u64, pub dialect_residue: u64, pub unvalidatable_schemas: u64, pub normalization_errors: u64, pub identical_result_repeats: u64, pub repeats_not_evaluated: u64, pub repeats_rescued: u64,
}
Expand description

Cumulative defect counts for one model.

Fields§

§requests: u64

Requests the proxy forwarded (or would have, but for a guard) for this model — every rate’s denominator.

§loop_guard_trips: u64

Requests the loop/stagnation guard acted on.

Since #1052 that is not the same as rejected: the guard’s default forwards a tripped request with a note, and only --loop-guard-mode refuse rejects it before dispatch. Both count here, so this number is a count of interventions — per process, reset when the daemon restarts. ADR 0011’s kill criterion reads the loop guard’s log instead (gglib proxy trips), which outlives the process and counts decisions rather than snapshots: a noted request the embedding check or admission then refuses is a decision there and no snapshot here, so the log can count more than this for the same traffic.

The sum of the two counts below, kept because it is the row people already read and the one an older dashboard knows. Adding all three double-counts.

§loop_guard_loops: u64

Of those, the ones LoopGuardTrip::Loop raised.

§loop_guard_stagnations: u64

Of those, the ones LoopGuardTrip::Stagnation raised.

§repairs_attempted: u64

Turns whose tool call failed schema validation and was re-issued, with tool_choice: "required" or as a second draw under gglib’s grammar.

§repairs_succeeded: u64

Of those, the re-issues that produced a conformant call.

§stream_errors: u64

Streaming turns that died on an upstream mid-stream failure — an error event the model server emitted mid-generation, or the byte stream itself breaking.

The catastrophic sibling of the repair signal. Both of the counters above require a model coherent enough to produce structured output: one counts verbatim repetition, the other a tool call that was attempted and malformed. A model whose sampling has collapsed produces neither — it emits output so far outside the expected shape that the model server kills the stream, and the person’s turn simply fails, invisibly to every other counter here.

Client disconnects are deliberately not in here: hanging up is a person’s action, not a model defect.

§truncated_generations: u64

Turns the model server cut off at the token ceiling (finish_reason == "length").

Not a model defect in the same sense as the others — a long answer is allowed to be long — but a rising rate is how a runaway generation looks before anything else notices, and it is the cheapest evidence that a context budget is mis-sized.

§empty_responses: u64

Turns that produced nothing a client can render.

§reasoning_only: u64

Of those, the ones that produced reasoning and nothing else.

Counted inside Self::empty_responses rather than beside it: the turn was empty from the client’s point of view either way, and the distinction is why. A model stranding its whole answer in reasoning_content is a prompt/template problem; one producing nothing at all is not.

§dialect_residue: u64

Turns where dialect markup survived normalization into client-visible output — the drift alarm, per model rather than fleet-wide.

§unvalidatable_schemas: u64

Turns whose tool call could not be validated at all, so repair never had an opinion to act on.

The blind spot this makes visible: a client whose tools all use anyOf gets zero repair coverage and, until now, zero evidence of that fact. A high rate here means the repair rate below it is measuring a much smaller slice of traffic than it appears to.

§normalization_errors: u64

Turns whose normalization discarded a malformed dialect tool call and surfaced the raw body as visible text instead.

§identical_result_repeats: u64

Turns whose newest tool-call batch repeated the batch before it and got an equal result back.

The comparison is against the preceding occurrence of that signature, not any earlier one: a call that returned A, then B, then A again is not counted, because the model did get a different answer last time.

The odd one out, deliberately. Every counter above measures a gglib organ firing or a defect in the shape of the model’s own output. This one measures a condition in the conversation: the model asked for the same thing twice and the environment answered the same way twice, which is the only evidence available that a repeat was genuinely stuck rather than progress that happens to look alike.

One increment per turn, like every counter above it — not a tally over the replayed history. A client resends the whole conversation each turn, so counting history-wide would re-count the same event on every later request and grow with the square of session length.

“Equal” means equal after hashing the result’s content as it arrived, per turn. Bounded to the calls the batch actually made, and only when every one of them was answered.

Counted whether or not the guard trips — a repeat under the threshold is exactly the case a verdict cannot see. Nothing acts on it: it exists to answer whether a corrective arm on the input plane would ever have a trigger, before one is built.

§repeats_not_evaluated: u64

Turns whose newest tool-call batch repeated the batch before it but whose results could not be compared.

The denominator for the counter above, and the reason a zero there can be read at all. A repeat gglib could not evaluate is not a repeat that did not happen: without this, an instrument that never managed to join a single result would look exactly like a fleet with nothing wrong.

Bumps when a client omits id on replayed tool calls, when results are not contiguous after the assistant turn, or when a parallel batch went partly unanswered.

§repeats_rescued: u64

Turns the loop guard would have acted on for repeating, and did not, because the answer had moved. A repeat inside the allowance is not one.

Unlike the two above, this is not a fact about the conversation — it is a fact about gglib’s own reflex, which is what the ledger was chartered for before ADR 0006 had to widen it. It reads the detector’s run-scoped outcome, not the session-wide map those two are computed from, so it is a third instrument rather than a third view of one.

It exists because ADR 0010 promoted the results join from an observation to a policy input, and a kill criterion nobody can read is not a kill criterion. If this dwarfs identical_result_repeats in real use, the join is being defeated by output that carries a clock rather than measuring progress, and the rescue wants narrowing or removing.

Trait Implementations§

Source§

impl Clone for ModelDefectCounts

Source§

fn clone(&self) -> ModelDefectCounts

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for ModelDefectCounts

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Default for ModelDefectCounts

Source§

fn default() -> ModelDefectCounts

Returns the “default value” for a type. Read more
Source§

impl PartialEq for ModelDefectCounts

Source§

fn eq(&self, other: &ModelDefectCounts) -> bool

Tests for self and other values to be equal, and is used by ==.
1.0.0 (const: unstable) · Source§

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.
Source§

impl Serialize for ModelDefectCounts

Source§

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>
where __S: Serializer,

Serialize this value into the given Serde serializer. Read more
Source§

impl Copy for ModelDefectCounts

Source§

impl Eq for ModelDefectCounts

Source§

impl StructuralPartialEq for ModelDefectCounts

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T> Instrument for T

§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more
§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<T> WithSubscriber for T

§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more