Skip to main content

parse_generation_config

Function parse_generation_config 

Source
pub fn parse_generation_config(body: &str) -> Option<PublishedGenerationConfig>
Expand description

Read a model author’s generation_config.json.

Returns None only when the body is not a JSON object. Anything else — an empty file, one carrying nothing but token ids, values of the wrong type — yields a PublishedGenerationConfig that reports what it found, because “the author published nothing usable” and “gglib could not read the file” are different answers and only the second is a fault.

§What is deliberately not read

  • max_new_tokens / max_length. max_tokens is unset by design in gglib (see InferenceConfig), so that a client’s own limit is the only thing that bounds a response. Importing an author’s default would quietly cap every request that named none, which is a much larger decision than this module is making.
  • do_sample: false. Reported via PublishedGenerationConfig::requests_greedy, never applied.
  • Everything else in the filebos_token_id, eos_token_id, pad_token_id, transformers_version and friends. Naming only the fields gglib models keeps this from becoming an obligation to track the whole transformers generation schema, the same rule SlotParams states about /props’s 42 fields.