We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
prepare_inputs_for_generation
1 parent 496c283 commit f30c225Copy full SHA for f30c225
src/transformers/generation/utils.py
@@ -608,7 +608,7 @@ def prepare_inputs_for_generation(
608
use_cache = kwargs.get("use_cache")
609
if use_cache is None:
610
use_cache = getattr(self.config, "use_cache", False)
611
- if past_key_values is None or use_cache:
+ if past_key_values is not None or use_cache:
612
# TODO (joao): handle the case where cache length == input_ids length. The function below results in an
613
# exception because we get empty input_ids after slicing. In essence, we need to roll back the cache 1
614
# token to recompute the logits for the first token to be generated (but not all caches support roll backs)
0 commit comments