symbolic_shape_infer.py crashes on a Range node with non-scalar inputs when fixing dynamic shapes on a dynamo-exported LoRA-adapter graph (--fixed_param_dict)

[chkrishna2001]

Title

symbolic_shape_infer.py crashes on a Range node with non-scalar inputs when fixing dynamic shapes on a dynamo-exported LoRA-adapter graph (--fixed_param_dict)

Environment

• onnxruntime 1.27.0 (onnxruntime/python/tools/symbolic_shape_infer.py / onnxruntime.tools.symbolic_shape_infer)
• Reached via olive-ai 0.13.0's capture-onnx-graph --fixed_param_dict → dynamic_to_fixed_shape pass, but the crash is inside onnxruntime's own shape-inference code, not Olive's.
• Base model: Qwen/Qwen2.5-0.5B-Instruct + a PEFT LoRA adapter, exported with torch.export/dynamo (--use_dynamo_exporter) specifically to keep the LoRA weights unmerged/swappable (ExtractAdapters pass), then attempting to convert dynamic dims to fixed via --fixed_param_dict "batch_size=1,max_length=4096".

Repro

python -m olive capture-onnx-graph \
  -m Qwen/Qwen2.5-0.5B-Instruct \
  -a <path-to-peft-lora-checkpoint> \
  --use_dynamo_exporter \
  --fixed_param_dict "batch_size=1,max_length=4096" \
  -o <output-dir>

Bug 1 — _infer_Range crashes on non-scalar inputs (symbolic_shape_infer.py)

File ".../onnxruntime/tools/symbolic_shape_infer.py", line 1598, in _infer_Range
    start = as_scalar(input_data[0])
  File ".../onnxruntime/tools/symbolic_shape_infer.py", line 101, in as_scalar
    assert len(x) == 1
AssertionError

Added a debug print just before the crash and captured the actual data for the failing node:

node='node_arange' inputs=['sym_size_int_423', 'add_12202', 'val_104']
input_data=[[1, 2, past_sequence_length, 64], [2, sequence_length + 2, past_sequence_length + 1, sequence_length + 64], 1]

input_data[0] and input_data[1] are 4-element shape tuples (looks like [batch, num_heads, seq_len, head_dim]), not scalars — _get_int_or_float_values appears to resolve these two Range inputs to the wrong cached value (a full shape tuple instead of the single symbolic dimension the Range op actually consumes as start/limit). Only delta=1 is a genuine scalar. This looks like it may be triggered specifically by the mix of newly-concrete dims (from --fixed_param_dict) and still-symbolic ones (past_sequence_length, sequence_length) confusing the value cache for this particular access pattern — I did not trace _get_int_or_float_values/sympy_data_ far enough to find the true root cause of the mis-resolution.

_infer_Range already has a graceful fallback for "inputs not fully known" (falls back to a symbolic output dim rather than crashing); it just doesn't cover "inputs known but not real scalars." Minimal, targeted patch that preserves existing behavior for the normal case:

def _infer_Range(self, node):  # noqa: N802
    vi = self.known_vi_[node.output[0]]
    input_data = self._get_int_or_float_values(node)
    inputs_are_scalar = all(i is not None and not isinstance(i, list) for i in input_data)
    if inputs_are_scalar:
        start = as_scalar(input_data[0])
        limit = as_scalar(input_data[1])
        delta = as_scalar(input_data[2])
        new_sympy_shape = [sympy.Max(sympy.ceiling((limit - start) / delta), 0)]
    else:
        new_sympy_shape = [self._new_symbolic_dim_from_output(node)]
    ...

Bug 2 — a latent, pre-existing bug this then reaches: unguarded vi.type assumed always set (broadcasting dim-merge check)

After patching Bug 1, a different crash surfaces a few nodes later — same file, same function area (_infer_impl's Add/Sub/Mul/Div/MatMul/etc. dim-merge check):

File ".../onnxruntime/tools/symbolic_shape_infer.py", line 2821, in _infer_impl
    out_rank = len(get_shape_from_type_proto(vi.type))
File ".../onnxruntime/tools/symbolic_shape_infer.py", line 42, in get_shape_from_type_proto
    assert not is_sequence(type_proto)
File ".../onnxruntime/tools/symbolic_shape_infer.py", line 37, in is_sequence
    assert cls_type in ["tensor_type", "sequence_type"]
AssertionError

vi.type.WhichOneof("value") is None here (type not yet set at all) — a knock-on effect of an upstream node's output now legitimately having an unresolved/symbolic shape (e.g. Bug 1's fallback). This dim-merge check is a broadcasting-aware refinement, not essential to correctness; suggest guarding it:

if vi.type.WhichOneof("value") is not None:
    out_rank = len(get_shape_from_type_proto(vi.type))
    ...

Bug 3 — same pattern, different call site: out_shape assumed non-None where it's already documented as possibly-None a few lines up

File ".../onnxruntime/tools/symbolic_shape_infer.py", line 2932, in _infer_impl
    for idx in range(len(out_shape))
TypeError: object of type 'NoneType' has no len()

This is inside the same auto-merge block. A few lines earlier in the same function, out_shape is explicitly checked with out_shape is not None and (...) before entering this whole block — but that possibility isn't re-checked at the for idx in range(len(out_shape)) line. Suggest:

if shapes and out_shape is not None:
    for idx in range(len(out_shape)):
        ...

Bug 4 — unresolved: after patching 1–3, infer_shapes still raises Exception("Incomplete symbolic shape inference")

With all three patches applied, the crash is gone but SymbolicShapeInference.infer_shapes(..., guess_output_rank=True) still ends with:

raise Exception("Incomplete symbolic shape inference")

i.e. after Bug 1's fallback introduces a fresh, unconnected symbolic dimension for the affected Range node, nothing downstream ever resolves it, so shape inference converges without fully resolving all shapes even with guess_output_rank=True. I believe patches 1–3 are defensible (they preserve existing behavior for normal graphs and just avoid crashing on cases the code already anticipated as "unknown"), but a complete fix likely requires understanding why _get_int_or_float_values mis-resolves this Range node's inputs in the first place, so it can compute the correct concrete value instead of giving up — that's beyond what I was able to root-cause in the time I had. Filing this now in case it's useful signal or a starting point for someone more familiar with the value-cache/sympy_data_ internals.

Happy to open a PR with patches 1–3 if useful, though I want to flag that they don't fully resolve fixed-shape export for this specific graph shape (dynamo-exported + LoRA-adapter-swappable + --fixed_param_dict) — only bugs 1–3 are things I'm confident are correct, narrow fixes; bug 4 needs someone who knows this code better.

[xadupre]

cc @justinchuby