asyncio.run() causes crashes

[bryaan]

PythonCall

I have to use the following function otherwise I get crashes that interestingly enough don't crash the julia program. But it does print a python stacktrace to console.

In addition, if using this in concurrent for loop with @sync and @async it looks like the code runs sequentially.

function asyncio_run(coro)
    loop = asyncio.get_event_loop()
    loop.run_until_complete(coro)

    # This is the recommended way but keeps crashing 
    # Interestingly it doesnt error in julia, it just prints a python error to console.
    # asyncio.run(coro)
end

Fatal error on SSL transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x29da04d30>
transport: <_SelectorSocketTransport closing fd=24>
Traceback (most recent call last):
  File "/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/selector_events.py", line 924, in write
    n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/sslproto.py", line 690, in _process_write_backlog
    self._transport.write(chunk)
  File "/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/selector_events.py", line 930, in write
    self._fatal_error(exc, 'Fatal write error on socket transport')
  File "/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/selector_events.py", line 725, in _fatal_error
    self._force_close(exc)
  File "/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/selector_events.py", line 737, in _force_close
    self._loop.call_soon(self._call_connection_lost, exc)
  File "/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 753, in call_soon
    self._check_closed()
  File "/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

[cjdoris]

For the first issue, please provide a full MWE of how to produce the error, and include the full error output too.

For the second issue, interleaving Julia tasks and Python coroutines asynchronously is not easy. Simply doing asyncio.run(...) or whatever just blocks Julia until Python completes the given coroutine. I've never tried but you may be able to create a Python coroutine which simply keeps yielding to the Julia scheduler, then run that asynchronously (in Python) with the other coroutine you want to run.

[bryaan]

It sounds more of a hack where perf will suffer. Since this lib doesn't work with threads well its not really worth it to pursue this.

For future updates to PythonCall: would it be possible to somehow use python's libuv with julia's libuv?

[cjdoris]

OK, I'll close the issue then.

TBH I don't know much about libuv, what it's place is in Julia, and I didn't know Python used it too. If you have any more info/suggestions about this, please open a separate issue, thanks!