Skip to content

Commit 5997515

Browse files
committed
Updated modeling_auto.py to handle num_kv_blocks=None case gracefully
Signed-off-by: Vaibhav Verma <vaibverm@qti.qualcomm.com>
1 parent 4703696 commit 5997515

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

QEfficient/transformers/models/modeling_auto.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2308,7 +2308,11 @@ def __init__(
23082308
if self.is_tlm:
23092309
self.model.qaic_config["return_pdfs"] = True
23102310

2311-
num_kv_blocks = self.model.qaic_config.get("num_kv_blocks", None)
2311+
if self.model.qaic_config is not None:
2312+
num_kv_blocks = self.model.qaic_config.get("num_kv_blocks", None)
2313+
else:
2314+
num_kv_blocks = None
2315+
23122316
if num_kv_blocks is not None:
23132317
BlockedKVAttentionTransform.apply(model, num_kv_blocks=num_kv_blocks)
23142318

0 commit comments

Comments
 (0)