Skip to main content

recommend

Function recommend 

Source
pub fn recommend(mem: &SystemMemoryInfo) -> Option<Recommendation>
Expand description

Recommend the largest shortlisted model that fits this machine.

Returns None when even the smallest candidate would not fit. That is a real answer, not a failure: suggesting a model that overflows would produce a working-but-unusably-slow endpoint, and the user is better served by being told their budget and left to choose.