pytest now skips async tests

[red-eight]

Here's a simple test case:

import asyncio
import asynctest


async def my_async_function(a, b):
    asyncio.sleep(1)
    return a + b


class TestMyFunctions(asynctest.TestCase):
    async def test_my_async_function(self):
        expected_value = 8
        value = await my_async_function(4, 4)
        self.assertEqual(expected_value, value)

Everything works using an older version of pytest (pytest-5.3.5).

(virtenv) me@me-Latitude-7490:~/src/quicktests/pytest_regression$ python -m pytest -vx
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.6.9, pytest-5.3.5, py-1.8.1, pluggy-0.13.1 -- /home/me/.virtualenvs/virtenv/bin/python
cachedir: .pytest_cache
rootdir: /home/me/src/quicktests/tests_pytest_regression
collected 1 item                                                                                                                                                                                           

test_simple.py::TestMyFunctions::test_my_async_function PASSED                                                                                                                                       [100%]

============================================================================================ 1 passed in 0.04s =============================================================================================

For some reason, the test is skipped in newer versions of pytest.

(virtenv) me@me-Latitude-7490:~/src/quicktests/pytest_regression$ pip install pytest --upgrade
(virtenv) me@me-Latitude-7490:~/src/quicktests/pytest_regression$ python -m pytest -vx
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.6.9, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- /home/me/.virtualenvs/virtenv/bin/python
cachedir: .pytest_cache
rootdir: /home/me/src/quicktests/tests_pytest_regression
collected 1 item                                                                                                                                                                                           

test_simple.py::TestMyFunctions::test_my_async_function SKIPPED                                                                                                                                      [100%]

============================================================================================= warnings summary =============================================================================================
test_simple.py::TestMyFunctions::test_my_async_function
  /home/me/.virtualenvs/virtenv/lib/python3.6/site-packages/_pytest/python.py:171: PytestUnhandledCoroutineWarning: async def functions are not natively supported and have been skipped.
  You need to install a suitable plugin for your async framework, for example:
    - pytest-asyncio
    - pytest-trio
    - pytest-tornasync
    - pytest-twisted
    warnings.warn(PytestUnhandledCoroutineWarning(msg.format(nodeid)))

-- Docs: https://docs.pytest.org/en/latest/warnings.html

Also, it seems like AioHTTPTestCase tests are NOT skipped.

[nicoddemus]

Hi @red-eight,

Thanks for the report!

Indeed we should not try to skip async functions from TestCase subclasses. I can verify that indeed on 5.3.5 things work as expected (added a failure to make sure):

λ pytest .tmp\test-async.py
======================== test session starts ========================
platform win32 -- Python 3.6.6, pytest-5.3.5.dev422+ga03e076e8.d20200303, py-1.8.1, collected 1 item

.tmp\test-async.py F                                           [100%]

============================= FAILURES ==============================
______________ TestMyFunctions.test_my_async_function _______________

self = <test-async.TestMyFunctions testMethod=test_my_async_function>

    async def test_my_async_function(self):
        expected_value = 8
        value = await my_async_function(4, 4)
>       assert 0
E       AssertionError: assert 0

.tmp\test-async.py:14: AssertionError
========================= 1 failed in 0.10s =========================

[RonnyPfannschmidt]

@nicoddemus i believe its related to #6927 but not entirely sure

there is an issue in our codebase wrt monkeypatching to work with unittest

[nicoddemus]

It is related: #6927 explicitly adds support for IsolatedAsyncioTestCase (py38+), but asynctest doesn't have the same privilege so we issue a warning instead.

I'm currently investigating this; I think I have a bugfix for 5.4.X, and an enhancement for async functions in general for 6.0.0, should open a PR in a bit

[nicoddemus]

PR open: #7144