event_loop teardown prevents cleanup tasks from finishing

[amit-traiana]

Hello everyone!

I'm not sure if that's even a pytest-asyncio bug, it might go down up to asyncio, but Im sharing the details in case it is. I'm trying to use pytest-asyncio with pyppeteer - a package the wrap Google's Puppetter API and interacts with Chrome. Here's and example test:

from pyppeteer import launch
import pytest

@pytest.mark.asyncio
async def test_some_asyncio_code():
    browser = await launch()
    page = await browser.newPage()
    raise Exception('Unexpected Exception')
    await browser.close()

The following makes pytest freeze, or to be exact - the process is sleeping forever while the C method epoll_wait is waiting for any interrupt.

The following code seems to work-around it:

from pyppeteer import launch
import pytest

@pytest.yield_fixture
def loop():
     loop = asyncio.new_event_loop()
     asyncio.set_event_loop(loop)
     yield loop

def test_some_asyncio_code(loop):
    async def inner():
        browser = await launch()
        page = await browser.newPage()
        raise Exception('Unexpected Exception')
        await browser.close()

    loop.run_until_complete(inner())

The difference here is that I do not close the loop, and Python does it for me when the run ends. Adding 'loop.close()to the fixture, prints an error thatThe event loop is already closed`, which might be a hint of what's happening here?

[aparamon]

Yury Selivanov mentions that asyncio.run() is the correct way to execute async code in Python 3.7.
Does it eliminate the problem reported here?

[amit-traiana]

Well, the above was tested in Python 3.6, but I'm not sure how pytest-asyncio is being implements and if it's using asyncio.run at all. Maybe one of the maintainers will have a better answer.

[rmorshea]

I'm experiencing this as well. The event_loop fixture appears to close the loop without awaiting pending tasks:

https://github.com/pytest-dev/pytest-asyncio/blob/master/pytest_asyncio/plugin.py#L186

It seems like the fixture should implement something like this before closing:

tasks = loop.all_tasks()
loop.run_until_complete(gather(*tasks))
loop.close()

[seifertm]

@amit-traiana thanks for the report and thanks @rmorshea for debugging the issue :)

There were a couple of issues related to this behaviour. #235 proposes a solution for this. I'm closing this issue to direct the discussion to #235.

[rmorshea]

Thanks! @seifertm