[IR][fix] Save value info for initializers #1552

justinchuby · 2024-05-16T22:38:53Z

Previously initializers are not included in the graph value_info because they are not easily accessible from the Graph object. Now what we store all the Values for initializers, we can serialize the value information into the graph.

Updated test models to include the value info protos for initializers so the round tripping tests can pass.

Fix #1501

codecov · 2024-05-16T22:43:32Z

❌ 9 Tests Failed:

Tests completed	Failed	Passed	Skipped
14678	9	14669	2385

View the top 3 failed test(s) by shortest run time

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0410_test_gemm_default_zero_bias

Stack Traces | 0.004s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_gemm_default_zero_bias'

The above exception was the direct cause of the following exception:
.nox\test_ort_nightly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_gemm_default_zero_bias' (e=No module named 'tests.onnx_backend_test_code.test_gemm_default_zero_bias') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_gemm_default_zero_bias.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_gemm_default_zero_bias.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT
E   from onnxscript.onnx_opset import opset13
E   
E   @script()
E   def bck_test_gemm_default_zero_bias(a: FLOAT[3,5], b: FLOAT[5,4], c: FLOAT[1,4]) -> (FLOAT[3,4]):
E       y = opset13.Gemm(a, b, c)
E       return y

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_1161_test_spacetodepth_example

Stack Traces | 0.004s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_spacetodepth_example'

The above exception was the direct cause of the following exception:
.nox\test_ort_nightly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_spacetodepth_example' (e=No module named 'tests.onnx_backend_test_code.test_spacetodepth_example') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_spacetodepth_example.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_spacetodepth_example.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT
E   from onnxscript.onnx_opset import opset13
E   
E   @script()
E   def bck_test_spacetodepth_example(x: FLOAT[1,1,4,6]) -> (FLOAT[1,4,2,3]):
E       y = opset13.SpaceToDepth(x, blocksize=2)
E       return y

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_1308_test_xor_bcast3v1d

Stack Traces | 0.004s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_xor_bcast3v1d'

The above exception was the direct cause of the following exception:
.nox\test_onnx_weekly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_xor_bcast3v1d' (e=No module named 'tests.onnx_backend_test_code.test_xor_bcast3v1d') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_xor_bcast3v1d.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_xor_bcast3v1d.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import BOOL
E   from onnxscript.onnx_opset import opset7
E   
E   @script()
E   def bck_test_xor_bcast3v1d(x: BOOL[3,4,5], y: BOOL[5]) -> (BOOL[3,4,5]):
E       xor = opset7.Xor(x, y)
E       return xor

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

github-actions · 2024-05-16T22:59:18Z

Test Results

24 files ± 0 24 suites ±0 42m 4s ⏱️ - 2h 46m 54s
11 859 tests - 3 830 7 857 ✅ - 5 827 3 989 💤 + 2 012 13 ❌ - 15
101 688 runs - 362 142 43 228 ✅ - 54 264 58 267 💤 - 307 842 193 ❌ - 36

For more details on these failures, see this check.

Results for commit 64e740a. ± Comparison against base commit 9bae2b5.

This pull request removes 9868 and adds 6038 tests. Note that renamed tests count towards both.

docs.test.test_documentation_examples.TestDocumentationExample ‑ test_documentation_examples
tests.eager_mode_test.EagerModeTest_0_reference_runtime ‑ test_sequence_input
tests.eager_mode_test.EagerModeTest_1_onnxruntime ‑ test_sequence_input
tests.eager_mode_test.TestEagerModeArguments_0_reference_runtime ‑ test_function_all_input_by_kwargs
tests.eager_mode_test.TestEagerModeArguments_0_reference_runtime ‑ test_function_attribute_by_positional_args
tests.eager_mode_test.TestEagerModeArguments_0_reference_runtime ‑ test_function_input_and_attribute_by_kwargs_out_of_order
tests.eager_mode_test.TestEagerModeArguments_0_reference_runtime ‑ test_function_some_input_by_kwargs
tests.eager_mode_test.TestEagerModeArguments_0_reference_runtime ‑ test_op_all_input_by_kwargs
tests.eager_mode_test.TestEagerModeArguments_0_reference_runtime ‑ test_op_attribute_by_positional_args
tests.eager_mode_test.TestEagerModeArguments_0_reference_runtime ‑ test_op_input_and_attribute_by_kwargs_out_of_order
…

onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0000_test_convinteger_without_padding
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0000_test_lrn_default
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0001_test_sce_NCd1d2d3_sum_weight_high_ii_expanded
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0001_test_sqrt
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0002_test_min_int32
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0002_test_nonmaxsuppression_suppress_by_IOU_and_scores
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0003_test_cast_no_saturate_FLOAT_to_FLOAT8E4M3FNUZ
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0003_test_col2im
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0004_test_reduce_l2_negative_axes_keep_dims_example
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0004_test_reduce_sum_square_do_not_keepdims_example_expanded
…

This pull request removes 14 skipped tests and adds 2018 skipped tests. Note that renamed tests count towards both.

tests.eager_test.TestOnnxSignal ‑ test_dft_rstft_04_A2
tests.eager_test.TestOnnxSignal ‑ test_dft_rstft_07_B2
tests.eager_test.TestOnnxSignal ‑ test_dft_rstft_10_C2
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__native_layer_norm_cpu_float16
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__nn_functional_avg_pool1d_cpu_float16
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__nn_functional_avg_pool2d_cpu_float16
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__nn_functional_cross_entropy_cpu_float16
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyFullGraphCPU ‑ test_output_match_opinfo__native_layer_norm_cpu_float16
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyFullGraphCPU ‑ test_output_match_opinfo__nn_functional_avg_pool1d_cpu_float16
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyFullGraphCPU ‑ test_output_match_opinfo__nn_functional_avg_pool2d_cpu_float16
…

onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0003_test_cast_no_saturate_FLOAT_to_FLOAT8E4M3FNUZ
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0006_test_cast_FLOAT16_to_FLOAT8E4M3FN
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0007_test_sequence_insert_at_front
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0012_test_qlinearmatmul_2D_int8_float16
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0015_test_loop11
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0016_test_split_variable_parts_1d_opset13
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0017_test_if_seq
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0020_test_castlike_FLOAT_to_STRING
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0021_test_qlinearmatmul_2D_uint8_float32
onnxscript.backend.onnx_export_test.TestOnnxBackEnd ‑ test_export2python_produces_correct_onnx_script_model_0022_test_unsqueeze_two_axes
…

This pull request skips 16 and un-skips 8 tests.

tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__div_mode_floor_rounding_cpu_float16
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__floor_divide_cpu_float16
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__index_put_bool_cpu_float16
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__index_put_bool_cpu_float32
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__index_put_bool_cpu_int32
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__index_put_bool_cpu_int64
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__matmul_cpu_float16
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__var_correction_cpu_float16
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__var_dim_cpu_float16
tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__var_mean_correction_cpu_float16
…

onnxscript.function_libs.torch_lib.graph_building.graph_building_test.TestModelSaving ‑ test_experimental_function_value_info_are_stored_in_graph_value_info
onnxscript.function_libs.torch_lib.graph_building.graph_building_test.TestModelSaving ‑ test_input_output_and_initializer_are_not_stored_in_value_info
onnxscript.function_libs.torch_lib.graph_building.graph_building_test.TestModelSaving ‑ test_save_initializer_to_files_for_large_model
onnxscript.ir.serde_test.TensorProtoTensorTest ‑ test_tensor_proto_tensor_bfloat16
onnxscript.tools.memory_peak_test.TestMemoryPeak ‑ test_memory
onnxscript.tools.memory_peak_test.TestMemoryPeak ‑ test_spy
onnxscript.tools.transformers_models.llama_test.TestExportLlama ‑ test_llama_dort_static
onnxscript.tools.transformers_models.phi_test.TestExportPhi ‑ test_phi_dort_static

♻️ This comment has been updated with latest results.

onnxscript/ir/serde.py

titaiwangms

Ci fails

onnxscript/ir/serde.py

gramalingam · 2024-09-05T22:42:01Z

Can this be closed, or is this still relevant?

justinchuby · 2024-09-05T23:28:57Z

This is still relavent. We need to update our test data first.

onnxscript/ir/serde.py

Copilot

Pull Request Overview

This PR fixes the issue where initializers were not included in the graph’s value_info by ensuring that all initializer tensors are processed and their associated values are updated or created accordingly. Key changes include:

Using an enumerate loop in deserialization to warn and skip initializers with an empty name.
Consistently using a local variable (initializer_name) to reference initializer names when looking up or inserting values.
Updating the serialization loop to add value_info for initializers that are not also graph inputs.

Files not reviewed (4)

testdata/e2e_models/Speech2Text2ForCausalLM/dynamo/Speech2Text2ForCausalLM_dynamo.onnx: Language not supported
testdata/e2e_models/mobilenetv2_100/dynamo/mobilenetv2_100_dynamo.onnx: Language not supported
testdata/e2e_models/resnet18/dynamo/resnet18_dynamo.onnx: Language not supported
testdata/e2e_models/torchscript_model/torchscript_model.onnx: Language not supported

Comments suppressed due to low confidence (1)

onnxscript/ir/serde.py:635

[nitpick] Consider renaming the loop variable 'i' to 'index' for improved clarity and consistency in the code.

for i, tensor in enumerate(initializer_tensors):

titaiwangms · 2025-04-21T16:32:58Z

onnxscript/ir/serde.py

-        if initializer.const_value is None:
+    for value in from_.initializers.values():
+        _maybe_add_quantization_annotation(graph_proto, value)
+        if _should_create_value_info_for_value(value) and value.name not in input_names:


What are the cases that initializers are model inputs? Does that mean the inputs are constants?

It's like a parameter having a default value. Any input can be initialized if an initializer of the same name is in the graph. Users can choose to overwrite the initializer by providing their own input.

titaiwangms

Even the model is updated, should we assert valueinfo in the tests?

justinchuby · 2025-04-21T17:00:54Z

Even the model is updated, should we assert valueinfo in the tests?

Good point. I will create a follow up to update some of the test behaviors, thanks.

[IR][fix] Save value info for initializers

5e6d729

justinchuby requested review from gramalingam and titaiwangms May 16, 2024 22:39

justinchuby added the module: IR Intermediate representation label May 16, 2024

Update deser

3533a65

justinchuby commented May 17, 2024

View reviewed changes

onnxscript/ir/serde.py Show resolved Hide resolved

justinchuby commented May 17, 2024

View reviewed changes

onnxscript/ir/serde.py Show resolved Hide resolved

Update onnxscript/ir/serde.py

ee38b6d

titaiwangms reviewed May 17, 2024

View reviewed changes

onnxscript/ir/serde.py Show resolved Hide resolved

justinchuby and others added 6 commits May 20, 2024 07:52

Merge branch 'main' into justinchu/ser-initializer

05dd283

Merge branch 'main' into justinchu/ser-initializer

feea637

snap

5156668

Merge branch 'main' into justinchu/ser-initializer

ae831e6

Merge branch 'main' into justinchu/ser-initializer

6388d95

Merge branch 'main' into justinchu/ser-initializer

64e740a

justinchuby added 3 commits September 9, 2024 18:52

Merge branch 'main' into justinchu/ser-initializer

30401da

Merge branch 'main' into justinchu/ser-initializer

5c0865b

revert

a554517