pub fn replicate_seeds(seeds: &[u32]) -> Vec<u32>Expand description
The seed set the A/A arm runs, derived from the primary one.
§Why the seeds must differ
This is the whole design of the arm. Re-running the same seeds would
measure how reproducible a fixed seed is — which, given a deterministic
decode, is approximately “perfectly”, and would report a noise floor near
zero. That number is true and useless: the primary comparison’s precision
is not limited by whether seed 12345 replays, it is limited by which five
seeds happened to be drawn. So the replicate draws five different ones and
measures exactly that.
A pathological seed list can still overlap — [1, 1 + OFFSET] maps onto
itself by one element — so the seeds the replicate actually used are
recorded in AgenticEvalReport::replicate_seeds rather than left implicit.