Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release gil when doing ray.wait. #4190

Merged
merged 1 commit into from
Feb 28, 2019

Conversation

robertnishihara
Copy link
Collaborator

This fixes #4082. Some logs weren't getting printed because the driver was blocked in ray.wait which didn't release the GIL so the error printing thread didn't get to print. Note that some prints can still be missed if the driver exits before the prints get streamed from the log monitor to the driver.

Copy link
Contributor

@ericl ericl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, nice catch.

@raulchen
Copy link
Contributor

Can you also help fix prepare_actor_checkpoint? this method also waits for raylet's reply and better release the GIL while waiting.

@pcmoritz
Copy link
Contributor

@raulchen Can you do that in a separate PR? This PR should be merged ASAP because it is a release blocker (we are in the process of preparing the 0.6.4 release)

@raulchen
Copy link
Contributor

ok, will do

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12375/
Test PASSed.

@pcmoritz pcmoritz merged commit 9c5fdbb into ray-project:master Feb 28, 2019
@robertnishihara robertnishihara deleted the vanishinglogs branch February 28, 2019 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Worker print output vanishes if not flushed (or if worker unexpectedly dies)
5 participants