infer_from_chat_template

Function infer_from_chat_template 

Source
pub fn infer_from_chat_template(
    template: Option<&str>,
    model_name: Option<&str>,
) -> ModelCapabilities
Expand description

Infer model capabilities from chat template Jinja source and model name.

Uses string heuristics to detect template constraints. Returns safe defaults if template is missing or unparseable.

§Detection Strategy

Two-layer approach:

  • Layer 1 (Metadata): Check chat template for reliable signals (preferred)
  • Layer 2 (Name Heuristics): Use model name patterns as fallback when metadata is missing

§Capabilities Detected

  • System role: Looks for explicit rejection messages in template
  • Strict turns: Looks for alternation enforcement logic
  • Tool calling: Checks for <tool_call>, if tools, function_call patterns (metadata); falls back to model name patterns like “hermes”, “functionary” (heuristic)
  • Reasoning: Checks for <think>, <reasoning>, enable_thinking (metadata); falls back to model name patterns like “deepseek-r1”, “qwq”, “o1” (heuristic)

§Fallback Behavior

Missing or unparseable templates default to empty capabilities (unknown state).