pub const CHARS_PER_TOKEN_APPROX: usize = 4;Expand description
Character-to-token conversion factor used to translate a model’s token
context size into the character budget truncate_history measures.
This is not an attempt at precise real-world tokenization — it deliberately
matches the GitHub Copilot LLM Gateway extension’s own
TOKEN_CONSTANTS.CHARS_PER_TOKEN = 4 (see its tokenBudget.ts), so that
gglib’s advertised context window and the extension’s own char-to-token
budget estimate agree on the same conversion factor. What matters is
consistency between the two sides, not tokenizer accuracy.