Skip to content

Commit 3f06a3a

Browse files
committed
fix following huggingface#27907
1 parent 4721c36 commit 3f06a3a

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/transformers/models/llama/modeling_llama.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -699,7 +699,7 @@ def forward(
699699

700700
kv_seq_len = key_states.shape[-2]
701701
if past_key_value is not None:
702-
kv_seq_len += past_key_value.get_seq_length(self.layer_idx)
702+
kv_seq_len += past_key_value.get_usable_length(kv_seq_len, self.layer_idx)
703703
cos, sin = self.rotary_emb(value_states, seq_len=kv_seq_len)
704704
query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin, position_ids)
705705

0 commit comments

Comments
 (0)