Issue with multiple input quantization(tf savedmodel)

[Akshaysharma29]

Hi Team, thanks for the wonderful tool. I am trying to quantize savedmodel with multiple inputs but facing below issue.

Code:

import tensorflow as tf
from neural_compressor.experimental import Quantization, common

quantizer = Quantization()
quantizer.model = '../model/'
dataset = quantizer.dataset('dummy', shape=[(1,256),(1,256)])
quantizer.calib_dataloader = common.DataLoader(dataset)
quantized_model = quantizer.fit()
output_path = '../model/'
quantized_model.save(output_path)

2022-07-15 10:10:03 [WARNING] Force convert framework model to neural_compressor model.
2022-07-15 10:10:03.488839: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-07-15 10:10:03.490012: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting:
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:05 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:08 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:10 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:10 [WARNING] Output tensor names should not be empty.
2022-07-15 10:10:10 [WARNING] Input tensor names should not be empty.
2022-07-15 10:10:10 [INFO] Generate a fake evaluation function.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:13 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:13.470210: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:10:13.470453: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-07-15 10:10:14.615203: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:10:14.615386: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:18 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:20 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:20 [INFO] ConvertLayoutOptimizer elapsed time: 0.32 ms
2022-07-15 10:10:23.822868: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:10:23.823242: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-07-15 10:10:24 [INFO] Pass GrapplerOptimizer elapsed time: 3781.35 ms
2022-07-15 10:10:24 [INFO] Pass SwitchOptimizer elapsed time: 277.86 ms
2022-07-15 10:10:24 [INFO] Pass RemoveTrainingNodesOptimizer elapsed time: 275.21 ms
2022-07-15 10:10:24 [INFO] Pass SplitSharedInputOptimizer elapsed time: 5.65 ms
2022-07-15 10:10:25 [INFO] Pass GraphFoldConstantOptimizer elapsed time: 276.08 ms
2022-07-15 10:10:25 [INFO] Pass FuseColumnWiseMulOptimizer elapsed time: 275.88 ms
WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/util.py:322: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.extract_sub_graph
2022-07-15 10:10:25 [WARNING] From /opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/util.py:322: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.extract_sub_graph
2022-07-15 10:10:26 [INFO] Pass StripUnusedNodesOptimizer elapsed time: 1121.14 ms
2022-07-15 10:10:26 [INFO] Pass GraphCseOptimizer elapsed time: 279.52 ms
2022-07-15 10:10:27 [INFO] Pass FuseDecomposedBNOptimizer elapsed time: 539.8 ms
2022-07-15 10:10:27 [INFO] Pass FoldBatchNormNodesOptimizer elapsed time: 277.56 ms
2022-07-15 10:10:27 [INFO] Pass UpdateEnterOptimizer elapsed time: 277.27 ms
2022-07-15 10:10:28 [INFO] Pass ConvertLeakyReluOptimizer elapsed time: 279.11 ms
2022-07-15 10:10:28 [INFO] Pass ConvertAddToBiasAddOptimizer elapsed time: 275.88 ms
2022-07-15 10:10:28 [INFO] Pass FuseTransposeReshapeOptimizer elapsed time: 277.57 ms
2022-07-15 10:10:29 [INFO] Pass FuseConvWithMathOptimizer elapsed time: 276.92 ms
2022-07-15 10:10:29 [INFO] Pass ExpandDimsOptimizer elapsed time: 275.92 ms
2022-07-15 10:10:29 [INFO] Pass FetchWeightFromReshapeOptimizer elapsed time: 279.25 ms
2022-07-15 10:10:29 [INFO] Pass InjectDummyBiasAddOptimizer elapsed time: 282.46 ms
2022-07-15 10:10:30 [INFO] Pass MoveSqueezeAfterReluOptimizer elapsed time: 273.58 ms
2022-07-15 10:10:31 [INFO] Pass Pre Optimization elapsed time: 15803.25 ms
2022-07-15 10:10:32 [INFO] Get FP32 model baseline.
2022-07-15 10:10:32 [INFO] Save tuning history to /home/jupyter/model-compressor/conversion_notebooks/nc_workspace/2022-07-15_10-10-02/./history.snapshot.
2022-07-15 10:10:32 [INFO] FP32 baseline is: [Accuracy: 1.0000, Duration (seconds): 0.0000]
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:34 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:36 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:39 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:41 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:41 [WARNING] Found possible input node names: ['attention_mask', 'input_ids'], output node names: ['global_average_pooling1d'].
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:44 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:46 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:48 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:51 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:51 [WARNING] Found possible input node names: ['attention_mask', 'input_ids'], output node names: ['global_average_pooling1d'].
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:54 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:54.913870: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:10:54.914104: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-07-15 10:10:56.035642: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:10:56.035843: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-07-15 10:11:04 [INFO] Pass Quantization elapsed time: 5884.0 ms
2022-07-15 10:11:07 [INFO] Start sampling on calibration dataset.
2022-07-15 10:11:07.605083: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1377, in _do_call
return fn(*args)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1361, in _run_fn
target_list, run_metadata)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1455, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,0] = is not in [0, )
[[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 573, in quantize
self._inference(self._sampling_model)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 181, in _inference
_ = model.sess.run(output_tensor, feed_dict) if model.iter_op==[]
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 968, in run
run_metadata_ptr)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1191, in _run
feed_dict_tensor, options, run_metadata)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1371, in _do_run
run_metadata)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1396, in _do_call
raise type(e)(node_def, op, message) # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:

Detected at node 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup' defined at (most recent call last):
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 536, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
Node: 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup'
indices[0,0] = is not in [0, )
[[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]

Original stack trace for 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup':
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 536, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 408, in import_graph_def
producer_op_list=producer_op_list)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 516, in _import_graph_def_internal
_ProcessNewOps(graph)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 247, in _ProcessNewOps
for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in _add_new_tf_operations
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3787, in _create_op_from_tf_operation
ret = Operation(c_op, self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2133, in init
self._traceback = tf_stack.extract_stack_for_node(self._c_op)

2022-07-15 10:11:08 [ERROR] Fail to quantize graph due to Graph execution error:

Detected at node 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup' defined at (most recent call last):
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 536, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
Node: 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup'
indices[0,0] = is not in [0, )
[[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]

Original stack trace for 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup':
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 536, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 408, in import_graph_def
producer_op_list=producer_op_list)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 516, in _import_graph_def_internal
_ProcessNewOps(graph)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 247, in _ProcessNewOps
for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in _add_new_tf_operations
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3787, in _create_op_from_tf_operation
ret = Operation(c_op, self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2133, in init
self._traceback = tf_stack.extract_stack_for_node(self._c_op)
.
2022-07-15 10:11:08 [WARNING] Fail to forward with batch size=1, set to 1 now.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:10 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:13 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:15 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:18 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:18 [WARNING] Found possible input node names: ['attention_mask', 'input_ids'], output node names: ['global_average_pooling1d'].
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:21 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:23 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:25 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:27 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:28 [WARNING] Found possible input node names: ['attention_mask', 'input_ids'], output node names: ['global_average_pooling1d'].
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:31 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:31.485104: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:11:31.485309: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-07-15 10:11:32.564946: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:11:32.565120: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-07-15 10:11:40 [INFO] Pass Quantization elapsed time: 5611.09 ms
2022-07-15 10:11:43 [INFO] Start sampling on calibration dataset.
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 536, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 275, in convert
post_cse_graph_def = PostCseOptimizer(model.graph_def).do_transformation()
AttributeError: 'NoneType' object has no attribute 'graph_def'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1377, in _do_call
return fn(*args)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1361, in _run_fn
target_list, run_metadata)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1455, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,0] = is not in [0, )
[[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 573, in quantize
self._inference(self._sampling_model)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 181, in _inference
_ = model.sess.run(output_tensor, feed_dict) if model.iter_op==[]
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 968, in run
run_metadata_ptr)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1191, in _run
feed_dict_tensor, options, run_metadata)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1371, in _do_run
run_metadata)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1396, in _do_call
raise type(e)(node_def, op, message) # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:

Detected at node 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup' defined at (most recent call last):
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 551, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
Node: 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup'
indices[0,0] = is not in [0, )
[[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]

Original stack trace for 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup':
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 551, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 408, in import_graph_def
producer_op_list=producer_op_list)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 516, in _import_graph_def_internal
_ProcessNewOps(graph)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 247, in _ProcessNewOps
for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in _add_new_tf_operations
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3787, in _create_op_from_tf_operation
ret = Operation(c_op, self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2133, in init
self._traceback = tf_stack.extract_stack_for_node(self._c_op)

2022-07-15 10:11:43 [ERROR] Fail to quantize graph due to Graph execution error:

Detected at node 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup' defined at (most recent call last):
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 551, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
Node: 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup'
indices[0,0] = is not in [0, )
[[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]

Original stack trace for 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup':
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 551, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 408, in import_graph_def
producer_op_list=producer_op_list)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 516, in _import_graph_def_internal
_ProcessNewOps(graph)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 247, in _ProcessNewOps
for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in _add_new_tf_operations
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3787, in _create_op_from_tf_operation
ret = Operation(c_op, self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2133, in init
self._traceback = tf_stack.extract_stack_for_node(self._c_op)
.
2022-07-15 10:11:43 [ERROR] Unexpected exception AttributeError("'NoneType' object has no attribute 'graph_def'") happened during tuning.
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 536, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 275, in convert
post_cse_graph_def = PostCseOptimizer(model.graph_def).do_transformation()
AttributeError: 'NoneType' object has no attribute 'graph_def'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 551, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 275, in convert
post_cse_graph_def = PostCseOptimizer(model.graph_def).do_transformation()
AttributeError: 'NoneType' object has no attribute 'graph_def'
2022-07-15 10:11:43 [ERROR] Specified timeout or max trials is reached! Not found any quantized model which meet accuracy goal. Exit.

AttributeError Traceback (most recent call last)
/tmp/ipykernel_28346/3554570389.py in
5 quantized_model = quantizer.fit()
6 output_path = '../models/gev8_image_encoder_quant/'
----> 7 quantized_model.save(output_path)

AttributeError: 'NoneType' object has no attribute 'save'

Any suggestion?

[chensuyue]

Thanks for your feedback, is this model available to be shared for debug?

[Akshaysharma29]

Hi @chensuyue sorry I can't share the model

[chensuyue]

Ok, we may find some similar model for debug.

[Akshaysharma29]

Hi @chensuyue is there any update?

One piece of information which I can share is that it has a transformer layer in it.

[chensuyue]

Not yet, pls wait for 2~3 days.

[lvliang-intel]

Hi @Akshaysharma29,
Could you please try to convert your savedmodel to pb file and then use session run to check if your fp32 graph is good?
You didn't evaluate the fp32 model to get the accuracy data. Maybe the fp32 graph itself has problem.

2022-07-15 10:10:32 [INFO] FP32 baseline is: [Accuracy: 1.0000, Duration (seconds): 0.0000]

It‘s hard for us to identify the root cause without the model file. INC can handle the multiple input quantization well.
You can take the 'bert_large_squad_model_zoo' as an example, please make sure the dataset compatible with your model.

https://github.com/intel/neural-compressor/blob/master/examples/tensorflow/nlp/bert_large_squad_model_zoo/quantization/ptq/bert.yaml#L19

[Akshaysharma29]

Hi @lvliang-intel actually I am trying using dataset = quantizer.dataset('dummy', shape=[(1,256),(1,256)]) dummy data without using yaml file.

Have you tried that or YAML file is compulsory for bert type of models?

[lvliang-intel]

Hi @Akshaysharma29,
There are built-in datasets 'bert' and 'mzbert' which are for the bert type of models. They use the tf_record format.
The 'dummy' dataset is not suitable for bert type of models. I tried to apply the 'dummy' dataset for the bert example
and got a similar error.

I0725 14:11:07.101656 140584788830016 utility.py:264] Pass Quantization elapsed time: 25517.31 ms
Traceback (most recent call last):
File "/home2/lvl/anaconda3/envs/tf-default-qdq-py3.8/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1377, in _do_call
return fn(*args)
File "/home2/lvl/anaconda3/envs/tf-default-qdq-py3.8/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1360, in _run_fn
return self._call_tf_sessionrun(options, feed_dict, fetch_list,
File "/home2/lvl/anaconda3/envs/tf-default-qdq-py3.8/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1453, in _call_tf_sessionrun
return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[337] = -23 is not in [0, 30522)
[[{{node bert/embeddings/GatherV2}}]]

You can also register your own datasets in INC.
https://github.com/intel/neural-compressor/blob/master/docs/dataset.md

[chensuyue]

Closed due to no feedback for a long time, please re-open the issue if it still there.