Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Tune fails on macOS #15771

Closed
gmeeker opened this issue Sep 17, 2023 · 1 comment
Closed

[Bug] Tune fails on macOS #15771

gmeeker opened this issue Sep 17, 2023 · 1 comment
Labels
needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it type: bug

Comments

@gmeeker
Copy link
Contributor

gmeeker commented Sep 17, 2023

tvmc tune appears to have broken between 0.12.0 and 0.13.0.

Expected behavior

Produce an autotuner json file (which worked in 0.12.0).

Actual behavior

[Task  1/25]  Current/Best:    0.00/   0.00 GFLOPS | Progress: (0/40) | 0.00 sException in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.8/3.8.18/Frameworks/Python.framework/Versions/3.8/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/Cellar/python@3.8/3.8.18/Frameworks/Python.framework/Versions/3.8/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/gmeeker/src/tvm_0.13.0/python/tvm/rpc/server.py", line 272, in _listen_loop
    _serving(conn, addr, opts, load_library)
  File "/Users/gmeeker/src/tvm_0.13.0/python/tvm/rpc/server.py", line 143, in _serving
    server_proc.start()
  File "/usr/local/Cellar/python@3.8/3.8.18/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/usr/local/Cellar/python@3.8/3.8.18/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/usr/local/Cellar/python@3.8/3.8.18/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/usr/local/Cellar/python@3.8/3.8.18/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/local/Cellar/python@3.8/3.8.18/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/local/Cellar/python@3.8/3.8.18/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/usr/local/Cellar/python@3.8/3.8.18/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object '_serving.<locals>._serve_loop'

This looks similar to #8363 and #8347 although those are older than 0.12.0 which is working on macOS.

Environment

macOS 13.5.2
Python 3.8.18
TVM 0.13.0 or current head of main branch

Steps to reproduce

Compiled using brew per install from source instructions:
https://tvm.apache.org/docs/install/from_source.html

Run tvmc tune --target "llvm" --output resnet50-v2-7-autotuner_records.json resnet50-v2-7.onnx as described here:
https://tvm.apache.org/docs/tutorial/tvmc_command_line_driver.html

Triage

  • needs-triage
  • tune:autotvm
@gmeeker gmeeker added needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it type: bug labels Sep 17, 2023
gmeeker added a commit to gmeeker/tvm that referenced this issue Jan 6, 2024
Fix regression in (apache#15187) when multiprocessing start method is not 'fork',
which prevented tuning from working. This affects macOS and Windows.
Also in python 3.14 the default start method will be 'spawn'.
gmeeker added a commit to gmeeker/tvm that referenced this issue Jan 6, 2024
Fix regression in (apache#15187) when multiprocessing start method is not 'fork',
which prevented tuning from working. This affects macOS and Windows.
Also in python 3.14 the default start method will be 'spawn'.
gmeeker added a commit to gmeeker/tvm that referenced this issue Jan 6, 2024
Fix regression in (apache#15187) when multiprocessing start method is not 'fork',
which prevented tuning from working. This affects macOS and Windows.
Also in python 3.14 the default start method will be 'spawn'.
Johnson9009 pushed a commit that referenced this issue Jan 12, 2024
* [RPC] Fix tuning on macOS and Windows (#15771)

Fix regression in (#15187) when multiprocessing start method is not 'fork',
which prevented tuning from working. This affects macOS and Windows.
Also in python 3.14 the default start method will be 'spawn'.

* [RPC] clean up _serve_loop function
@Johnson9009
Copy link
Contributor

Thanks @gmeeker for the fix, because the PR #16357 is merged, so this bug will be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it type: bug
Projects
None yet
Development

No branches or pull requests

2 participants