max tokens
A hyperparameter on the large-model invocation side that limits the maximum number of tokens the model generates in a single call, thereby controlling output length and cost.
Timeline (last 90 days)
Common hyperparameters include temperature (controls output randomness, the lower the more deterministic), max tokens (limits generation length), top_p, and so on.
One of the most common sources of LLMs outputting incomplete JSON is that the model has a maximum output length (max tokens), and once exceeded it is truncated, leaving brackets and quotes unclosed.
All Facts (2)
Common hyperparameters include temperature (controls output randomness, the lower the more deterministic), max tokens (limits generation length), top_p, and so on.
50%UnverifiedOne of the most common sources of LLMs outputting incomplete JSON is that the model has a maximum output length (max tokens), and once exceeded it is truncated, leaving brackets and quotes unclosed.
50%