Struct PairedEffect 

pub struct PairedEffect {
    pub pairs: usize,
    pub unmeasured_pairs: usize,
    pub wins: usize,
    pub losses: usize,
    pub ties: usize,
    pub mean_delta: f64,
    pub p_value: Option<f64>,
}

The paired view of the raw-versus-gglib comparison.

The two real arms run the same seeds on the same tasks, so every (task, seed) cell is a matched pair — and pairing is what removes the eval’s identical-arm spread from the comparison. The ceiling experiment (tune runs #12–#32, ADR 0004’s postscript) resolved a +0.067 effect through noise wider than that only because it paired per run; the same data has been sitting in AgenticEvalReport::tasks all along, compared only as arm means.

Pairs are on TuneTaskResult::tool_match_score — the one graded per-run quality scalar. Pass/fail flips remain visible per task in AgenticTaskComparison::pass_counts; folding them in here would double count, since the match score is most of what decides passed.

Fields

pairs: usize

Matched (task, seed) pairs in which both arms produced a real observation.

unmeasured_pairs: usize

Pairs both arms ran but at least one side never reached the model — dropped from every number here, and reported so the drop is visible.

wins: usize

Pairs the gglib arm scored strictly higher.

losses: usize

Pairs the raw arm scored strictly higher.

ties: usize

Pairs with identical scores. On a suite where most tasks pass cleanly under both arms this is the largest bucket, and that is information: the arms mostly agree.

mean_delta: f64

Mean of gglib − raw over the measured pairs.

p_value: Option<f64>

One-sided Wilcoxon signed-rank p for “gglib scores higher”, by normal approximation with tie correction.

None below WILCOXON_MIN_PAIRS non-tied pairs — the approximation is not trustworthy there, and rendering a statistic the design cannot support is worse than rendering none (the EffectVerdict rule). At small counts, read Self::wins against Self::losses instead.

Implementations

impl PairedEffect

pub fn from_tasks(tasks: &[AgenticTaskComparison]) -> Option<Self>

Compute the paired comparison from the per-task drill-down.

None when no (task, seed) pair has both sides measured — a paired analysis of nothing is not a zero effect.

pub fn from_paired_runs( a: &[TuneTaskResult], b: &[TuneTaskResult], ) -> Option<Self>

The paired comparison between two runs of the same task list, paired by task_id — the first argument’s score minus the second’s, so wins counts pairs the first run took.

Built for the tune apply gate (winner versus incumbent), where the two sides are candidates rather than eval arms. A task present in one run and absent from the other is skipped, not counted: an unpaired task has nothing to compare.

fn from_deltas(deltas: &[f64], unmeasured_pairs: usize) -> Option<Self>

Aggregate a delta list into the paired record. None on no deltas — a paired analysis of nothing is not a zero effect.

Trait Implementations

impl Clone for PairedEffect

fn clone(&self) -> PairedEffect

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PairedEffect

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for PairedEffect

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for PairedEffect

fn eq(&self, other: &PairedEffect) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for PairedEffect

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Copy for PairedEffect

impl StructuralPartialEq for PairedEffect