Skip to content
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/source/en/main_classes/text_generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,8 @@ like token streaming.
[[autodoc]] GenerationMixin
- generate
- compute_transition_scores


> **Note**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be removed now

> `max_new_tokens` is now the recommended argument to control how many tokens the model generates.
> `max_length` remains for backward compatibility because it includes the length of the input prompt, which can be less intuitive.
7 changes: 7 additions & 0 deletions src/transformers/generation/configuration_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,10 @@ class GenerationConfig(PushToHubMixin):
max_length (`int`, *optional*, defaults to 20):
The maximum length the generated tokens can have. Corresponds to the length of the input prompt +
`max_new_tokens`. Its effect is overridden by `max_new_tokens`, if also set.

`max_new_tokens` is now the recommended argument to control how many tokens the model generates.
`max_length` remains for backward compatibility because it includes the length of the input prompt, which can be less intuitive.

max_new_tokens (`int`, *optional*):
The maximum numbers of tokens to generate, ignoring the number of tokens in the prompt.
min_length (`int`, *optional*, defaults to 0):
Expand Down Expand Up @@ -330,6 +334,9 @@ class GenerationConfig(PushToHubMixin):
Whether to disable the automatic compilation of the forward pass. Automatic compilation happens when
specific criteria are met, including using a compilable cache. Please open an issue if you find the
need to use this flag.



"""

extra_output_flags = ("output_attentions", "output_hidden_states", "output_scores", "output_logits")
Expand Down