distributed==1.24.0 dask==0.20.0 crashes

[psthiagu]

I am trying to install PyBNF (python3.7) which uses the distributed package (1.24.0 dask==0.20.0).
It throws up an error message generated by the call to distributed. I am attaching the error message generated when I run python with:
import distributed
distributed.Client()
It will be great if you can help.
dist_error.txt

Best,
Thiagu
psthiagu@gmail.com

[emitra17]

PyBNF developer here. I should add from our issue discussion that the user is trying to run distributed on ubuntu on windows10 (windows subsystem for linux).

[mrocklin]

It looks like there is some oddity with forkserver, which isn't too surprising given the Windows connection.

I recommend setting the multiprocessing context to "spawn", which should make Windows happier. You can do this in Python code with the following lines

import dask
dask.config.set({'distributed.worker.multiprocessing-method': 'spawn'})

Or by including the following value in a dask config file, probably living in ~/.config/dask/...

distributed:
  worker:
    multiprocessing-method: spawn

You can learn more about setting configuration at http://docs.dask.org/en/latest/configuration.html

[psthiagu]

Unfortunately neither fix works. But thanks for giving it a go. I have contacted the WSL team. They are looking into it.

…

On Thu, Jan 10, 2019 at 10:44 PM Matthew Rocklin ***@***.***> wrote: It looks like there is some oddity with forkserver, which isn't too surprising given the Windows connection. I recommend setting the multiprocessing context to "spawn", which should make Windows happier. You can do this in Python code with the following lines import dask dask.config.set({'distributed.worker.multiprocessing-method': 'spawn'}) Or by including the following value in a dask config file, probably living in ~/.config/dask/... distributed: worker: multiprocessing-method: spawn You can learn more about setting configuration at http://docs.dask.org/en/latest/configuration.html — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2455 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQCvXvjmgwhoEHLXUnTEoorr0_-ZGsSeks5vB3T4gaJpZM4Z4uHu> .

[mrocklin]

There are several other ways to create a Dask client/cluster. For example you might ...

• Use only the local scheduler
• Use the local distributed scheduler, but with processes=False
• Set up your cluster by hand with the dask-scheduler and dask-worker commands

I would be surprised if one of those options didn't work.

See http://docs.dask.org/en/latest/scheduling.html for more information about different schedulers

[psthiagu]

Will try them out, thanks.

…

On Thu, 10 Jan 2019, 23:15 Matthew Rocklin ***@***.*** wrote: There are several other ways to create a Dask client/cluster. For example you might ... - Use only the local scheduler - Use the local distributed scheduler, but with processes=False - Set up your cluster by hand with the dask-scheduler and dask-worker commands I would be surprised if one of those options didn't work. See http://docs.dask.org/en/latest/scheduling.html for more information about different schedulers — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2455 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQCvXuZRn6Wo7qFGEpmpUd-gFgRqDNkxks5vB3wcgaJpZM4Z4uHu> .

[mrocklin]

Any update here?

[psthiagu]

Sorry. Should have closed this. The Microsoft team couldn't help. I now have dual boot system with Ubuntu and windows 10. Dask works fine under Ubuntu.

…

On Wed, 30 Jan 2019, 05:21 Matthew Rocklin ***@***.*** wrote: Any update here? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2455 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQCvXpVjRjMOiyv-HFPFmFKQ1yaY6LVUks5vIN5ogaJpZM4Z4uHu> .

[CJ-Wright]

xrefs:
microsoft/WSL#3935
microsoft/WSL#3764
microsoft/WSL#4604

[matthewgson]

It looks like there is some oddity with forkserver, which isn't too surprising given the Windows connection.

I recommend setting the multiprocessing context to "spawn", which should make Windows happier. You can do this in Python code with the following lines

import dask
dask.config.set({'distributed.worker.multiprocessing-method': 'spawn'})

Or by including the following value in a dask config file, probably living in ~/.config/dask/...

distributed:
  worker:
    multiprocessing-method: spawn

You can learn more about setting configuration at http://docs.dask.org/en/latest/configuration.html

I was having the same issue with my windows workstation with WSL(ubuntu), and I confirm that this worked for me. However it generates some warning messages like below:

distributed.comm.tcp - WARNING - Could not set timeout on TCP stream: [Errno 92] Protocol not available

[mrocklin]

We're moving towards making spawn the default in #3461 so the default experience may become a bit smoother.

I don't have any particular information to give about the protocol not available message. That seems concerning. Perhaps TCP sockets in WSL don't support timeouts?