pub struct ArmDelta {
pub tool_accuracy: Option<f64>,
pub loop_avoidance: Option<f64>,
pub task_completion: Option<f64>,
pub composite: Option<f64>,
pub withheld: Option<DeltaWithheld>,
pub wall_time_speedup: Option<f64>,
pub completion_token_ratio: Option<f64>,
}Expand description
Per-axis difference, gglib − raw. Positive means gglib scored higher.
Every axis is Option: an arm-level difference is only meaningful when both
arms were measured over the same runs and the same axes. See
ArmDelta::withheld, and prefer PairedEffect — which drops the
affected pairs instead of diluting everything — whenever it is available.
Fields§
§tool_accuracy: Option<f64>Tool-accuracy difference. None when Self::withheld is set.
loop_avoidance: Option<f64>Loop-avoidance difference.
None unless both arms measured the axis — a difference against an
arm that never risked a loop would be arithmetic on a number that was
never observed.
task_completion: Option<f64>Task-completion difference. None when Self::withheld is set.
composite: Option<f64>Composite-score difference, taken over the axes both arms measured.
Each arm’s own ArmScores::composite is renormalized over whichever
axes that arm measured, so two arms can carry composites on different
scales — an arm with no loop-eligible run divides by 0.6 where an arm
with one divides by 0.9. Subtracting those directly measures the scale.
The 2026-08-28 eval did exactly that: the raw arm’s free 1.0 on an
axis the gglib arm could not be scored on was worth about half the
reported gap.
None when Self::withheld is set.
withheld: Option<DeltaWithheld>Why the axis differences above are absent, when they are.
wall_time_speedup: Option<f64>Per-run wall-time speedup, raw ÷ gglib. Above 1.0 means gglib got
through the same work faster.
A ratio rather than a difference, for two reasons: lower is better
here, so a plain subtraction would invert this struct’s “positive means
gglib did better” convention; and the magnitudes are multiplicative —
a 230× gap reads as 230.0, not as -1099737 milliseconds. None
when the gglib arm recorded no measured wall time to divide by.
Taken per measured run on both sides. Summed totals put the two arms
on different denominators the moment either loses a run, and a run lost
to a timeout contributes the timeout rather than nothing: the
2026-08-28 eval reported 0.2× — 84% of which was five stalled runs
waiting out a ten-minute deadline — for an arm that was in fact about
1.2× faster on the work it actually did.
completion_token_ratio: Option<f64>Per-run completion-token ratio, raw ÷ gglib. Above 1.0 means gglib
reached the same outcome on fewer generated tokens. None when either
arm generated nothing measurable.
Per measured run for the same reason as Self::wall_time_speedup: the
summed form divided one arm’s 63-run total by the other’s 58-run total
and reported 1.48× where the per-run figure is 1.36×.