Skip to main content

fetch_published_sampling

Function fetch_published_sampling 

Source
pub async fn fetch_published_sampling(
    client: &dyn HfClientPort,
    repo_id: &str,
    tags: &[String],
) -> Option<InferenceConfig>
Expand description

Fetch the model author’s published sampling recipe, if one can be found.

Tries each repo name generation_config_candidates yields, in order, and stops at the first that gives back a usable recipe.

§Every failure is the same failure

This returns None and never an error, because there is exactly one response to any negative answer: carry on with the import and let the reasoning tag guess apply as it always has. A sampling recipe is a nicety; failing an import over one would be absurd. The distinct causes are logged rather than propagated:

  • 404 — the repo publishes no generation_config.json. The ordinary case for a quant repo, and the reason the candidate list exists.
  • Gated or private (401/403) — the base repo needs a token this installation does not have. Common for Llama and Gemma.
  • Offline, rate-limited, malformed — nothing to do but proceed.
  • Published nothing usable — a file carrying only token ids. Treated as a miss so it cannot displace the tag guess with an all-None recipe, and the search continues to the next candidate.

§Bounded work

At most MAX_GENERATION_CONFIG_LOOKUPS requests, and it stops at the first hit. An import is already dominated by downloading gigabytes of weights, but this runs on the local-add path too, where it must not turn a fast operation into a network-bound one.