Struct KvElemsPerToken 

pub struct KvElemsPerToken {
    pub k: u64,
    pub v: u64,
}

Per-token K and V element counts, type-agnostic.

K and V element counts are tracked separately (not just summed) because callers may quantize K and V to different types (e.g. q8_0 K with f16 V to sidestep the Flash Attention requirement on quantized V), so the byte cost of each side must be computed independently.

Fields

k: u64

v: u64

Trait Implementations

impl Clone for KvElemsPerToken

fn clone(&self) -> KvElemsPerToken

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for KvElemsPerToken

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for KvElemsPerToken

fn eq(&self, other: &KvElemsPerToken) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for KvElemsPerToken

impl Eq for KvElemsPerToken

impl StructuralPartialEq for KvElemsPerToken