Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testbed has async issues on Python 3.11 #1982

Closed
mhsmith opened this issue Jun 13, 2023 · 3 comments · Fixed by #2917
Closed

Testbed has async issues on Python 3.11 #1982

mhsmith opened this issue Jun 13, 2023 · 3 comments · Fixed by #2917
Labels
bug A crash or error in behavior.

Comments

@mhsmith
Copy link
Member

mhsmith commented Jun 13, 2023

After experiencing some 6-hour testbed hangs in #1949, I added timeouts to the relevant await statements using wait_for. When the timeout is triggered on Python 3.8-3.10, it gives a stack trace like this:

Traceback (most recent call last):
  File "/Users/msmith/git/beeware/toga/testbed/tests/widgets/test_webview.py", line 90, in test_set_url
    await assert_content_change(
  File "/Users/msmith/git/beeware/toga/testbed/tests/widgets/test_webview.py", line 46, in assert_content_change
    new_content = await get_content(widget, timer)
  File "/Users/msmith/git/beeware/toga/testbed/tests/widgets/test_webview.py", line 23, in get_content
    return await wait_for(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/tasks.py", line 501, in wait_for
    raise exceptions.TimeoutError()
asyncio.exceptions.TimeoutError

But on Python 3.11, it gives the following useless stack trace:

Traceback (most recent call last):
  File "/Users/msmith/.venv/bw-311/lib/python3.11/site-packages/pytest_asyncio/plugin.py", line 478, in inner
    _loop.run_until_complete(task)
  File "/Users/msmith/git/beeware/toga/testbed/tests/conftest.py", line 66, in run_until_complete
    return asyncio.run_coroutine_threadsafe(coro, self.loop).result()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
TimeoutError
@mhsmith mhsmith added the bug A crash or error in behavior. label Jun 13, 2023
@mhsmith
Copy link
Member Author

mhsmith commented Jun 13, 2023

EDIT: this comment is wrong: see below.

This issue is probably caused by limitations of the ProxyEventLoop in testbed/tests/conftest.py. This only implements the methods which are used by the pytest-asyncio plugin, but now that we're testing async code in Toga itself, it may need to be expanded.

In particular, I'm not sure how futures are working at all, since ProxyEventLoop should inherit create_future from AbstractEventLoop, which simply raises NotImplementedError.

@mhsmith
Copy link
Member Author

mhsmith commented Jun 13, 2023

#1949 works around this by making CI run the testbed on Python 3.10, which was also done by #1956 for a different reason.

@mhsmith
Copy link
Member Author

mhsmith commented Jun 15, 2023

On further thought, the ProxyEventLoop is only used on the pytest thread, and it's only used to dispatch the test function to the main thread, so it's fine for it to implement only those methods which are used by pytest-asyncio. The test body, and all the Toga code it exercises, run on the main thread and interact directly with the main thread's loop, just as a normal app would.

So the only thing that's actually going wrong here is that the TimeoutError stack trace is being lost on its way from the main thread to the pytest thread.

Note that in Python 3.11 asyncio.exceptions.TimeoutError became an alias for the global TimeoutError, but I don't see how that could have caused this problem, because none of the relevant code was catching it by name.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A crash or error in behavior.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant