Module benchmark 

benchmark

Benchmark domain types.

Two benchmark modes today (a third, tuning, is being added incrementally):

• Compare (compare): send the same prompt to N models sequentially; capture live streamed text and real-world timing data from llama-server’s timings response field.
• Perf (perf): run llama-bench for raw prompt-processing (pp) and token-generation (tg) throughput in tokens/sec.
• Tune (tune): sweep sampling parameters for one model against an agentic tool-calling task suite, scoring each candidate for tool-call accuracy and loop/stagnation avoidance to find the best-scoring settings.

All timing fields are Option<f64> because llama-server may omit the timings object (e.g. older builds, stream errors). Missing timing data is gracefully represented as None — never causes a panic or parse error.

Modules

Module	Contents
run	BenchmarkRun, BenchmarkRunType, BenchmarkRunStatus
summary	ModelBenchmarkSummary — denormalised per-model aggregate
compare	CompareConfig, ModelCompareResult
perf	PerfConfig, ModelPerfResult
tune	TuneConfig, task-suite schema, scoring result types
events	BenchmarkEvent (SSE units), BenchmarkModelResult

Modules

Module	LOC	Complexity	Coverage
compare.rs
events.rs
perf.rs
run.rs
summary.rs
tune/

Re-exports

pub use compare::CompareConfig;

pub use compare::ModelCompareResult;

pub use events::BenchmarkEvent;

pub use events::BenchmarkModelResult;

pub use perf::ModelPerfResult;

pub use perf::PerfConfig;

pub use run::BenchmarkRun;

pub use run::BenchmarkRunStatus;

pub use run::BenchmarkRunType;

pub use summary::ModelBenchmarkSummary;

pub use tune::CandidateSource;

pub use tune::ScoreWeights;

pub use tune::SweepSpec;

pub use tune::TaskCategory;

pub use tune::TaskSuite;

pub use tune::TuneCandidateResult;

pub use tune::TuneConfig;

pub use tune::TuneTask;

pub use tune::TuneTaskResult;

Modules

compare

Compare-mode benchmark types: configuration and per-model results.

events

SSE / channel event enum shared by all benchmark run types.

perf

Perf-mode benchmark types: llama-bench configuration and results.

run

Benchmark run metadata: type, lifecycle status, and the run record itself.

summary

Denormalised per-model benchmark summary.

tune

tune