Fix support for actor classmethods #2146

ericl · 2018-05-27T21:38:39Z

What do these changes do?

Currently, you cannot convert a class into a remote class if it has class methods defined (it will crash incorrectly with Methods must take a self argument). This fixes the crash by supporting methods properly.

robertnishihara

Looks good to me! I'm surprised this hasn't come up before.

AmplabJenkins · 2018-05-27T22:06:17Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5650/
Test FAILed.

robertnishihara · 2018-05-27T22:32:31Z

jenkins, retest this please

AmplabJenkins · 2018-05-27T23:02:22Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5651/
Test FAILed.

robertnishihara · 2018-05-27T23:35:09Z

Hmm, the jenkins test failure looks related

STDERR:
Traceback (most recent call last):
  File "/ray/test/jenkins_tests/multi_node_tests/remove_driver_test.py", line 272, in <module>
    driver_0(redis_address, driver_index)
  File "/ray/test/jenkins_tests/multi_node_tests/remove_driver_test.py", line 113, in driver_0
    Actor1.remote(driver_index, i, redis_address) for i in range(5)
  File "/ray/python/ray/actor.py", line 539, in remote
    return self._submit(args=args, kwargs=kwargs)
  File "/ray/python/ray/actor.py", line 615, in _submit
    actor_handle.__init__.remote(*args, **kwargs)
  File "/ray/python/ray/actor.py", line 437, in remote
    return self._submit(args, kwargs)
  File "/ray/python/ray/actor.py", line 448, in _submit
    dependency=self._actor._ray_actor_cursor)
  File "/ray/python/ray/actor.py", line 751, in _actor_method_call
    args = signature.extend_args(function_signature, args, kwargs)
  File "/ray/python/ray/signature.py", line 209, in extend_args
    keyword_name, function_name))
Exception: No value was provided for the argument 'redis_address' for the function '__init__'.

ericl · 2018-05-28T19:53:56Z

This appears to be a Python 2.7 issue, investigating.

AmplabJenkins · 2018-05-28T21:31:20Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5664/
Test PASSed.

robertnishihara · 2018-05-28T23:13:35Z

Restarting one of the travis jobs. Almost certainly unrelated, but I saw this error on Ubuntu Python 3.5.

ok
testMultipleRecursive (__main__.ReconstructionTestsMultinode) ... 24065:M 28 May 20:51:24.297 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
24065:M 28 May 20:51:24.297 # Server started, Redis version 3.9.102
24065:M 28 May 20:51:24.297 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
24065:M 28 May 20:51:24.297 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
Waiting for redis server at 127.0.0.1:37909 to respond...
24069:M 28 May 20:51:24.405 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
24069:M 28 May 20:51:24.405 # Server started, Redis version 3.9.102
24069:M 28 May 20:51:24.405 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
24069:M 28 May 20:51:24.405 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
Waiting for redis server at 127.0.0.1:41867 to respond...
test/stress_tests.py:181: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_store_0-2018-05-28_20-51-24-09074.out' mode='a' encoding='UTF-8'>
  "plasma_store_{}".format(i), True)
test/stress_tests.py:181: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_store_0-2018-05-28_20-51-24-09074.err' mode='a' encoding='UTF-8'>
  "plasma_store_{}".format(i), True)
test/stress_tests.py:184: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_manager_0-2018-05-28_20-51-24-00914.out' mode='a' encoding='UTF-8'>
  True))
test/stress_tests.py:184: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_manager_0-2018-05-28_20-51-24-00914.err' mode='a' encoding='UTF-8'>
  True))
test/stress_tests.py:181: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_store_1-2018-05-28_20-51-24-02619.out' mode='a' encoding='UTF-8'>
  "plasma_store_{}".format(i), True)
test/stress_tests.py:181: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_store_1-2018-05-28_20-51-24-02619.err' mode='a' encoding='UTF-8'>
  "plasma_store_{}".format(i), True)
test/stress_tests.py:184: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_manager_1-2018-05-28_20-51-24-04339.out' mode='a' encoding='UTF-8'>
  True))
test/stress_tests.py:184: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_manager_1-2018-05-28_20-51-24-04339.err' mode='a' encoding='UTF-8'>
  True))
test/stress_tests.py:181: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_store_2-2018-05-28_20-51-25-02954.out' mode='a' encoding='UTF-8'>
  "plasma_store_{}".format(i), True)
test/stress_tests.py:181: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_store_2-2018-05-28_20-51-25-02954.err' mode='a' encoding='UTF-8'>
  "plasma_store_{}".format(i), True)
test/stress_tests.py:184: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_manager_2-2018-05-28_20-51-25-05926.out' mode='a' encoding='UTF-8'>
  True))
test/stress_tests.py:184: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_manager_2-2018-05-28_20-51-25-05926.err' mode='a' encoding='UTF-8'>
  True))
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/worker.py:1476: ResourceWarning: unclosed <socket.socket fd=12, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('10.20.0.208', 40404), raddr=('8.8.8.8', 53)>
  node_ip_address = ray.services.get_node_ip_address()
Process STDOUT and STDERR is being redirected to /tmp/raylogs/.
Starting local scheduler with the following resources: {'CPU': 1, 'GPU': 0}.
Starting local scheduler with the following resources: {'CPU': 1, 'GPU': 0}.
Starting local scheduler with the following resources: {'CPU': 1, 'GPU': 0}.
Starting local scheduler with the following resources: {'CPU': 1, 'GPU': 0}.
Failed to start the UI, you may need to run 'pip install jupyter'.
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/services.py:1680: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/log_monitor-2018-05-28_20-51-25-01665.out' mode='a' encoding='UTF-8'>
  use_raylet=use_raylet)
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/services.py:1680: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/log_monitor-2018-05-28_20-51-25-01665.err' mode='a' encoding='UTF-8'>
  use_raylet=use_raylet)
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/services.py:1680: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/global_scheduler-2018-05-28_20-51-25-09812.out' mode='a' encoding='UTF-8'>
  use_raylet=use_raylet)
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/services.py:1680: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/global_scheduler-2018-05-28_20-51-25-09812.err' mode='a' encoding='UTF-8'>
  use_raylet=use_raylet)
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/services.py:1680: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/webui-2018-05-28_20-51-26-06912.out' mode='a' encoding='UTF-8'>
  use_raylet=use_raylet)
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/services.py:1680: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/webui-2018-05-28_20-51-26-06912.err' mode='a' encoding='UTF-8'>
  use_raylet=use_raylet)
/home/travis/miniconda/lib/python3.6/unittest/case.py:601: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_store_3-2018-05-28_20-51-25-06154.out' mode='a' encoding='UTF-8'>
  self.setUp()
/home/travis/miniconda/lib/python3.6/unittest/case.py:601: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_store_3-2018-05-28_20-51-25-06154.err' mode='a' encoding='UTF-8'>
  self.setUp()
/home/travis/miniconda/lib/python3.6/unittest/case.py:601: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_manager_3-2018-05-28_20-51-25-07138.out' mode='a' encoding='UTF-8'>
  self.setUp()
/home/travis/miniconda/lib/python3.6/unittest/case.py:601: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/raylogs/plasma_manager_3-2018-05-28_20-51-25-07138.err' mode='a' encoding='UTF-8'>
  self.setUp()
The object with ID ObjectID(c8b42b14882c85cd2936bd92f3d08c9d27bd0bf3) already exists in the object store.
The object with ID ObjectID(200db01425842c20627664cc214c58055a5e3d68) already exists in the object store.
The object with ID ObjectID(c86145ca92a5f27d39a7966005ee5d7d99d7db2c) already exists in the object store.
The object with ID ObjectID(52188d67db7937c2e04009d4a4f22d08fd838012) already exists in the object store.
The object with ID ObjectID(200db01425842c20627664cc214c58055a5e3d68) already exists in the object store.
The object with ID ObjectID(be8b6f1c68a7db2934c832d5b8f0b42c86333c42) already exists in the object store.
The object with ID ObjectID(9fed328cb06959e6efafdd36cbf0e2c6660795f9) already exists in the object store.
/home/travis/build/ray-project/ray/src/local_scheduler/local_scheduler_algorithm.cc:728: Lost connection to the plasma manager, local scheduler is exiting. Error: IOError: Broken pipe
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/local_scheduler/../core/src/local_scheduler/local_scheduler[0x43a085]
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/local_scheduler/../core/src/local_scheduler/local_scheduler(_ZN3ray8internal7CerrLogD1Ev+0x8e)[0x44609e]
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/local_scheduler/../core/src/local_scheduler/local_scheduler(_Z28fetch_object_timeout_handlerP11aeEventLoopxPv+0x261)[0x44b3f1]
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/local_scheduler/../core/src/local_scheduler/local_scheduler(aeProcessEvents+0x2b4)[0x47a944]
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/local_scheduler/../core/src/local_scheduler/local_scheduler(aeMain+0x2b)[0x47ab5b]
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/local_scheduler/../core/src/local_scheduler/local_scheduler(main+0x5c0)[0x43aa20]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7f490a2cbf45]
/home/travis/.local/lib/python3.6/site-packages/ray-0.4.0-py3.6-linux-x86_64.egg/ray/local_scheduler/../core/src/local_scheduler/local_scheduler[0x43c866]
The object with ID ObjectID(bc8589a48c6f2ac59632037583fb80d9eea041bf) already exists in the object store.


No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received

The build has been terminated

* master: [autoscaler] GCP node provider (ray-project#2061) [xray] Evict tasks from the lineage cache (ray-project#2152) [ASV] Add ray.init and simple Ray benchmarks (ray-project#2166) Re-encrypt key for uploading to S3 from travis to use travis-ci.com. (ray-project#2169) [rllib] Fix A3C PyTorch implementation (ray-project#2036) [JavaWorker] Do not kill local-scheduler-forked workers in RunManager.cleanup (ray-project#2151) Update Travis CI badge from travis-ci.org to travis-ci.com. (ray-project#2155) Implement Python global state API for xray. (ray-project#2125) [xray] Improve flush algorithm for the lineage cache (ray-project#2130) Fix support for actor classmethods (ray-project#2146) Add empty df test (ray-project#1879) [JavaWorker] Enable java worker support (ray-project#2094) [DataFrame] Fixing the code formatting of the tests (ray-project#2123) Update resource documentation (remove outdated limitations). (ray-project#2022) bugfix: use array redis_primary_addr out of its scope (ray-project#2139) Fix infinite retry in Push function. (ray-project#2133) [JavaWorker] Changes to the directory under src for support java worker (ray-project#2093) Integrate credis with Ray & route task table entries into credis. (ray-project#1841)

ericl added 2 commits May 27, 2018 14:36

cls

846a3a6

re

a5e1416

ericl requested a review from robertnishihara May 27, 2018 21:38

robertnishihara approved these changes May 27, 2018

View reviewed changes

fix python 2.7

56429fb

robertnishihara merged commit bc2a83e into ray-project:master May 29, 2018

robertnishihara deleted the fix-classmethod branch May 29, 2018 00:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix support for actor classmethods #2146

Fix support for actor classmethods #2146

Uh oh!

ericl commented May 27, 2018

Uh oh!

robertnishihara left a comment

Uh oh!

AmplabJenkins commented May 27, 2018

Uh oh!

robertnishihara commented May 27, 2018

Uh oh!

AmplabJenkins commented May 27, 2018

Uh oh!

robertnishihara commented May 27, 2018

Uh oh!

ericl commented May 28, 2018

Uh oh!

AmplabJenkins commented May 28, 2018

Uh oh!

robertnishihara commented May 28, 2018

Uh oh!

Uh oh!

Fix support for actor classmethods #2146

Fix support for actor classmethods #2146

Uh oh!

Conversation

ericl commented May 27, 2018

What do these changes do?

Uh oh!

robertnishihara left a comment

Choose a reason for hiding this comment

Uh oh!

AmplabJenkins commented May 27, 2018

Uh oh!

robertnishihara commented May 27, 2018

Uh oh!

AmplabJenkins commented May 27, 2018

Uh oh!

robertnishihara commented May 27, 2018

Uh oh!

ericl commented May 28, 2018

Uh oh!

AmplabJenkins commented May 28, 2018

Uh oh!

robertnishihara commented May 28, 2018

Uh oh!

Uh oh!