Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Serve] Only install dataclasses on Python 3.6 #10936

Merged

Conversation

architkulkarni
Copy link
Contributor

Why are these changes needed?

Before this PR, running the following script

import ray
from ray import serve
ray.init()
client = serve.start() 

on the product (a linux machine running python 3.7) would cause the following error:

2020-09-21 14:07:19,632	WARNING worker.py:1070 -- Failed to unpickle actor class 'ServeController' for actor ID d327fad408000000. Traceback:
Traceback (most recent call last):
  File "/root/anaconda3/lib/python3.7/site-packages/ray/function_manager.py", line 493, in _load_actor_class_from_gcs
    actor_class = pickle.loads(pickled_class)
  File "/root/anaconda3/lib/python3.7/site-packages/ray/serve/__init__.py", line 1, in <module>
    from ray.serve.api import (accept_batch, Client, connect, start)  # noqa: F401
  File "/root/anaconda3/lib/python3.7/site-packages/ray/serve/api.py", line 7, in <module>
    from ray.serve.controller import ServeController
  File "/root/anaconda3/lib/python3.7/site-packages/ray/serve/controller.py", line 12, in <module>
    from ray.serve.backend_worker import create_backend_worker
  File "/root/anaconda3/lib/python3.7/site-packages/ray/serve/backend_worker.py", line 16, in <module>
    from ray.serve.config import BackendConfig
  File "/root/anaconda3/lib/python3.7/site-packages/ray/serve/config.py", line 23, in <module>
    @dataclass
  File "/root/anaconda3/lib/python3.7/site-packages/dataclasses.py", line 958, in dataclass
    return wrap(_cls)
  File "/root/anaconda3/lib/python3.7/site-packages/dataclasses.py", line 950, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash, frozen)
  File "/root/anaconda3/lib/python3.7/site-packages/dataclasses.py", line 801, in _process_class
    for name, type in cls_annotations.items()]
  File "/root/anaconda3/lib/python3.7/site-packages/dataclasses.py", line 801, in <listcomp>
    for name, type in cls_annotations.items()]
  File "/root/anaconda3/lib/python3.7/site-packages/dataclasses.py", line 659, in _get_field
    if (_is_classvar(a_type, typing)
  File "/root/anaconda3/lib/python3.7/site-packages/dataclasses.py", line 550, in _is_classvar
    return type(a_type) is typing._ClassVar
AttributeError: module 'typing' has no attribute '_ClassVar'

A couple sources (HumanCellAtlas/metadata-api#95, konradhalas/dacite#48) suggest this error comes from having the dataclasses package (which is a backport for python 3.6) installed alongside python 3.7 (or presumably 3.8). Indeed, running pip uninstall dataclasses fixes the error, and reinstalling dataclasses reintroduces the error.

The solution is to only include dataclasses as a dependency if the python version is less than 3.7.

Related issue number

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@architkulkarni architkulkarni added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Sep 22, 2020
@richardliaw richardliaw merged commit 67c653c into ray-project:master Sep 22, 2020
@architkulkarni architkulkarni deleted the dataclasses-conditional-install branch September 22, 2020 21:15
Leemoonsoo added a commit to open-datastudio/ray that referenced this pull request Sep 25, 2020
dup image hash check

build ray-ml

uninstall dataclasses in the last docker image

build whl

remove GPU tag and dataclasses uninstall patch
ray-project#10909
ray-project#10936
Leemoonsoo added a commit to open-datastudio/ray that referenced this pull request Oct 3, 2020
dup image hash check

build ray-ml

uninstall dataclasses in the last docker image

build whl

remove GPU tag and dataclasses uninstall patch
ray-project#10909
ray-project#10936
@aced125
Copy link

aced125 commented Nov 18, 2020

Would say this is urgent - currently the landing page tune example will not work because of this (on a distributed cluster).

@jhayes-py
Copy link

I am still encountering this issue as of dacite 1.6.0, with python 3.10 and using the mamba package manager. The issue occurs with regular conda as well. Abridged output of the install command:

(myenv) C:\Users\jhayes>mamba install dacite

Looking for: ['dacite']

...

  Updating specs:

   - dacite
   - ca-certificates
   - certifi
   - openssl


  Package        Version  Build         Channel                Size
---------------------------------------------------------------------
  Install:
---------------------------------------------------------------------

  + dacite         1.6.0  pyhd3deb0d_0  conda-forge/noarch     14kB
  + dataclasses      0.8  pyh6d0b6a4_7  pkgs/main/noarch        8kB

  Summary:

  Install: 2 packages

  Total download: 22kB

---------------------------------------------------------------------

Confirm changes: [Y/n]
...

@architkulkarni
Copy link
Contributor Author

@jhayes-py It sounds like your issue is unrelated to Ray. The command conda install dacite in Python 3.10 is installing dataclasses, but it shouldn't be, because dataclasses is part of the python standard library starting with Python 3.7.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tests-ok The tagger certifies test failures are unrelated and assumes personal liability.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants