Struct AgenticEvalReport 

pub struct AgenticEvalReport {

    pub model_name: String,
    pub quantization: Option<String>,
    pub param_count_b: f64,
    pub ctx_size: u64,
    pub raw: ArmScores,
    pub gglib: ArmScores,
    pub delta: ArmDelta,
    pub tasks: Vec<AgenticTaskComparison>,
    pub seeds: Vec<u32>,
    pub control: Option<ArmScores>,
    pub raw_replicate: Option<ArmScores>,
    pub replicate_seeds: Vec<u32>,
    pub raw_replicates: Vec<ArmScores>,
    pub replicate_seed_sets: Vec<Vec<u32>>,
    pub paired: Option<PairedEffect>,
}

The complete A/B report — the leaderboard interchange format.

Fields

model_name: String

Model name as stored in the catalog.

quantization: Option<String>

Quantization label (e.g. Q4_K_M), when known.

param_count_b: f64

Parameter count in billions.

ctx_size: u64

Context size both arms ran at, in tokens.

raw: ArmScores

Aggregate scores under the raw arm.

gglib: ArmScores

Aggregate scores under the gglib arm.

delta: ArmDelta

Per-axis gglib − raw differences.

tasks: Vec<AgenticTaskComparison>

Per-task drill-down, one entry per suite task.

seeds: Vec<u32>

The seeds every task ran under, in order. Empty on a legacy row and on an explicitly unseeded run.

control: Option<ArmScores>

Scores under the positive control arm, when it ran.

Read Self::control_verdict rather than these numbers directly: what matters is not the control’s score but whether it differs. Its ArmScores::seeds is usually smaller than the real arms’ — see AgenticEvalConfig::control_seeds — so its composite is a coarser number than the ones it sits beside.

raw_replicate: Option<ArmScores>

Scores under the A/A arm — the raw pipeline again, different seeds.

Read Self::effect_verdict rather than this directly: the number that matters is its distance from Self::raw, not its own value.

replicate_seeds: Vec<u32>

The seeds the A/A arm ran under. Empty when it did not run, and when it ran unseeded.

Recorded rather than derived at read time so an overlap with Self::seeds is visible in the report instead of having to be recomputed from replicate_seeds.

raw_replicates: Vec<ArmScores>

Every A/A pair’s scores, in pair order, when more than one ran.

Self::raw_replicate stays populated with the first pair so a single-pair report — and every report written before this field — reads exactly as it always did. A legacy row deserializes this empty, and Self::noise_floor falls back to the single pair.

replicate_seed_sets: Vec<Vec<u32>>

The seed set behind each entry of Self::raw_replicates.

paired: Option<PairedEffect>

The paired per-(task, seed) comparison, computed at assembly.

Stored rather than derived-only, unlike the verdicts: those re-derive from two floats in any language, while this one carries a rank test nobody should maintain twice. Self::paired_effect re-derives it from the drill-down for reports written before the field existed.

Implementations

impl AgenticEvalReport

pub fn control_verdict(&self) -> Option<ControlVerdict>

What the positive control demonstrated, or None when it did not run.

None is distinct from any failure for the same reason Blind is distinct from zero divergences: nothing was claimed either way.

pub fn noise_floor(&self) -> Option<f64>

The eval’s own drift: the mean pairwise composite gap over every run of the identical raw configuration — the primary plus each A/A pair.

With one A/A pair this is exactly the old single-gap number. With K pairs it averages the C(K+1, 2) pairwise gaps among K + 1 runs of the same arm, which estimates the same quantity from more than one degree of freedom. A mean absolute gap, not a standard deviation: EFFECT_NOISE_RATIO was calibrated against a gap, and changing the estimator and the threshold at once would make old and new verdicts incomparable.

None when no A/A arm ran, which is distinct from a measured zero for the same reason Blind is distinct from zero divergences.

pub fn noise_pairs(&self) -> usize

How many pairwise gaps stand behind Self::noise_floor — the degrees of freedom a reader should weigh the verdict by.

fn drift_gaps(&self) -> Vec<f64>

Pairwise absolute composite gaps among every run of the raw configuration. Empty when no A/A arm ran.

pub fn effect_verdict(&self) -> Option<EffectVerdict>

Whether the measured effect is larger than the eval’s own drift.

None when no A/A arm ran, or when the composite delta was withheld — in either case the report contains no basis for the judgement. The second guard matters as much as the first: comparing a diluted effect against a drift figure produces a confident ratio out of two numbers that are not about the same thing, which is how a contaminated −0.058 came to be reported as “8.3× the drift”.

pub fn unstable_tasks(&self) -> Vec<&AgenticTaskComparison>

Tasks whose outcome was not stable across seeds under either arm.

The direct read of run-to-run variance, and the first thing to look at when two arms differ by less than they ought to.

pub fn paired_effect(&self) -> Option<PairedEffect>

The paired per-(task, seed) comparison, derived from the drill-down.

Derived rather than stored, like the verdicts above it — which also means a legacy report’s stored per-seed detail yields it retroactively. None when no pair has both sides measured.

pub fn delta_of( raw: &ArmScores, gglib: &ArmScores, weights: &ScoreWeights, ) -> ArmDelta

Compute the per-axis delta from the two arms’ scores.

weights is needed because the compared composite is not the difference of the two stored composites: each of those is renormalized over the axes its own arm measured, so subtracting them across a mismatch measures the renormalization. The axes both arms share are re-weighted here instead.

Trait Implementations

impl Clone for AgenticEvalReport

fn clone(&self) -> AgenticEvalReport

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AgenticEvalReport

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for AgenticEvalReport

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for AgenticEvalReport

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.