Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] tests/micro/zephyr/test_zephyr.py::test_onnx fails with CONFIG_SPEED_OPTIMIZATIONS=y on qemu_riscv64 #12721

Closed
mehrdadh opened this issue Sep 7, 2022 · 0 comments
Labels
type: bug type: micro microTVM-related issues vert:micro MicroTVM: src/runtime/micro, src/runtime/crt, apps/microtvm

Comments

@mehrdadh
Copy link
Member

mehrdadh commented Sep 7, 2022

Testing this optimization flag on qemu_riscv64 causes timeout in transport:

BUG    tvm.micro.project_api.client:client.py:111 send -> {"jsonrpc": "2.0", "method": "read_transport", "params": {"n": 2, "timeout_sec": 9.992954}, "id": 368}
DEBUG    tvm.micro.project_api.client:client.py:116 recv <- {"jsonrpc": "2.0", "id": 368, "result": {"data": "6U_"}}

DEBUG    tvm.micro.transport:transport.py:226 micro-rpc: read { 9.99s}    2 B -> [  2 B]: 13 cd                                            ..
DEBUG    tvm.micro.project_api.client:client.py:111 send -> {"jsonrpc": "2.0", "method": "write_transport", "params": {"data": "|NQ", "timeout_sec": 10.0}, "id": 369}
DEBUG    tvm.micro.project_api.client:client.py:116 recv <- {"jsonrpc": "2.0", "id": 369, "result": null}

DEBUG    tvm.micro.transport:transport.py:274 micro-rpc: write {10.00s}        <- [  2 B]: ff fd                                            ..
DEBUG    tvm.micro.project_api.client:client.py:111 send -> {"jsonrpc": "2.0", "method": "write_transport", "params": {"data": "vj6}9", "timeout_sec": 10.0}, "id": 370}
DEBUG    tvm.micro.project_api.client:client.py:116 recv <- {"jsonrpc": "2.0", "id": 370, "result": null}

DEBUG    tvm.micro.transport:transport.py:274 micro-rpc: write {10.00s}        <- [  4 B]: b3 00 00 00                                      ....
DEBUG    tvm.micro.project_api.client:client.py:111 send -> {"jsonrpc": "2.0", "method": "write_transport", "params": {"data": "V#W{", "timeout_sec": 10.0}, "id": 371}
DEBUG    tvm.micro.project_api.client:client.py:116 recv <- {"jsonrpc": "2.0", "id": 371, "result": null}

DEBUG    tvm.micro.transport:transport.py:274 micro-rpc: write {10.00s}        <- [  3 B]: 62 c6 10                                         b..
DEBUG    tvm.micro.project_api.client:client.py:111 send -> {"jsonrpc": "2.0", "method": "write_transport", "params": {"data": "r~m)}000000{{R31b_g5000000ssI22LJ#72LJ#7$W#J=000000RR91000001poj50w4hZ0RR91000000ssI2000008~^|S000008~^|S000001ONa4000000000000000z#{^H000000RR91000001poj50w4hZ", "timeout_sec": 10.0}, "id": 372}
DEBUG    tvm.micro.project_api.client:client.py:116 recv <- {"jsonrpc": "2.0", "id": 372, "result": null}

DEBUG    tvm.micro.transport:transport.py:265 micro-rpc: write {10.00s}        <- [128 B]:
0000  a8 00 00 00 00 00 00 00 03 00 00 00 04 80 00 80  ................
0010  00 00 00 00 02 00 00 00 07 00 00 00 07 00 00 00  ................
0020  c8 54 02 80 00 00 00 00 01 00 00 00 00 00 00 00  .T..............
0030  05 00 00 00 02 20 01 00 01 00 00 00 00 00 00 00  ................
0040  02 00 00 00 00 00 00 00 1c 00 00 00 00 00 00 00  ................
0050  1c 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00  ................
0060  00 00 00 00 00 00 00 00 c0 23 02 80 00 00 00 00  .........#......
0070  01 00 00 00 00 00 00 00 05 00 00 00 02 20 01 00  ................
DEBUG    tvm.micro.project_api.client:client.py:111 send -> {"jsonrpc": "2.0", "method": "write_transport", "params": {"data": "0RR91000000ssI2000004gdfE000004gdfE000001ONa4000000000000000", "timeout_sec": 10.0}, "id": 373}
DEBUG    tvm.micro.project_api.client:client.py:116 recv <- {"jsonrpc": "2.0", "id": 373, "result": null}

DEBUG    tvm.micro.transport:transport.py:265 micro-rpc: write {10.00s}        <- [ 48 B]:
0000  01 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00  ................
0010  0e 00 00 00 00 00 00 00 0e 00 00 00 00 00 00 00  ................
0020  04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
DEBUG    tvm.micro.project_api.client:client.py:111 send -> {"jsonrpc": "2.0", "method": "write_transport", "params": {"data": "<EQ", "timeout_sec": 10.0}, "id": 374}
DEBUG    tvm.micro.project_api.client:client.py:116 recv <- {"jsonrpc": "2.0", "id": 374, "result": null}

DEBUG    tvm.micro.transport:transport.py:274 micro-rpc: write {10.00s}        <- [  2 B]: e3 a8                                            ..
DEBUG    tvm.micro.project_api.client:client.py:111 send -> {"jsonrpc": "2.0", "method": "read_transport", "params": {"n": 1, "timeout_sec": 9.999999}, "id": 375}
DEBUG    tvm.micro.project_api.client:client.py:116 recv <- {"jsonrpc": "2.0", "id": 375, "error": {"code": -32000, "message": "IoTimeoutError", "data": {"traceback": "Traceback (most recent call last):\n  File \"/home/mhessar/work/tvm/python/tvm/micro/project_api/server.py\", line 481, in serve_one_request  # <--- Outermost server-side stack frame\n    self._dispatch_request(request)\n  File \"/home/mhessar/work/tvm/python/tvm/micro/project_api/server.py\", line 593, in _dispatch_request\n    return_value = dispatch_method(**params)\n  File \"/home/mhessar/work/tvm/python/tvm/micro/project_api/server.py\", line 642, in _dispatch_read_transport\n    reply_data = self._handler.read_transport(n, timeout_sec)\n  File \"/home/mhessar/work/tvm/tests/micro/zephyr/workspace_qemu_riscv64/2022-09-07T04-06-00/project/microtvm_api_server.py\", line 774, in read_transport\n    return self._transport.read(n, timeout_sec)\n  File \"/home/mhessar/work/tvm/tests/micro/zephyr/workspace_qemu_riscv64/2022-09-07T04-06-00/project/microtvm_api_server.py\", line 998, in read\n    return server.read_with_timeout(self.read_fd, n, timeout_sec)\n  File \"/home/mhessar/work/tvm/python/tvm/micro/project_api/server.py\", line 694, in read_with_timeout\n    _await_nonblocking_ready([fd], [], end_time=end_time)\n  File \"/home/mhessar/work/tvm/python/tvm/micro/project_api/server.py\", line 657, in _await_nonblocking_ready\n    raise IoTimeoutError()\ntvm.micro.project_api.server.IoTimeoutError: JSON-RPC error # -32000: IoTimeoutError\n"}}}

DEBUG    tvm.micro.transport:transport.py:192 micro-rpc: read {10.00s}    1 B -> [IoTimeoutError 10.00s]
DEBUG    tvm.micro.transport:transport.py:178 micro-rpc: closing transport
DEBUG    tvm.micro.project_api.client:client.py:111 send -> {"jsonrpc": "2.0", "method": "close_transport", "params": {}, "id": 376}
DEBUG    tvm.micro.project_api.client:client.py:116 recv <- {"jsonrpc": "2.0", "id": 376, "result": null}
============================================================================================= warnings summary =============================================================================================
tests/micro/zephyr/test_zephyr.py::test_onnx[qemu_riscv64]
  /home/mhessar/work/tvm/python/tvm/driver/build_module.py:267: UserWarning: target_host parameter is going to be deprecated. Please pass in tvm.target.Target(target, host=target_host) instead.
    "target_host parameter is going to be deprecated. "

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================================= short test summary info ==========================================================================================
FAILED ../tests/micro/zephyr/test_zephyr.py::test_onnx[qemu_riscv64] - tvm._ffi.base.TVMError: MicroSessionTimeoutError: failed to read reply message after timeout 10s

@areusch areusch added the needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it label Oct 19, 2022
@janetsc janetsc added type: micro microTVM-related issues vert:micro MicroTVM: src/runtime/micro, src/runtime/crt, apps/microtvm and removed needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it labels Oct 19, 2022
@tqchen tqchen closed this as completed Sep 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug type: micro microTVM-related issues vert:micro MicroTVM: src/runtime/micro, src/runtime/crt, apps/microtvm
Projects
None yet
Development

No branches or pull requests

4 participants