Skip to content

Implement loop.shutdown_default_executor() #353

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 10, 2020
Merged

Conversation

aeros
Copy link
Contributor

@aeros aeros commented Jul 9, 2020

Closes #349.

Since I don't have too much experience with Cython, I wasn't entirely certain about the purpose of uvloop/includes/stdlib.pxi, but I assumed that threading.Thread should be included there since it was also done for concurrent.futures.ThreadPoolExecutor. At a glance, it seems to be a means of importing the classes from stdlib modules at compile time rather than at run time? I'd appreciate clarification on this if anyone has the time to explain.

Otherwise, it's mostly similar to the reference implementation in asyncio.

@aeros
Copy link
Contributor Author

aeros commented Jul 9, 2020

/cc @1st1

@aeros aeros changed the title Implement shutdown_default_executor() Implement loop.shutdown_default_executor() Jul 9, 2020
@aeros
Copy link
Contributor Author

aeros commented Jul 9, 2020

The MacOS CI failures seem to be build-related and not due to the PR. I'm seeing similar CI failures in the most recent commit.

@aeros
Copy link
Contributor Author

aeros commented Jul 10, 2020

I'll wait on squashing into a single commit until the review process is complete, since it tends to affect previous review comments after force-pushing to the PR branch.

@aeros aeros requested a review from 1st1 July 10, 2020 04:07

self.loop.run_until_complete(foo())
self.loop.run_until_complete(
self.loop.shutdown_default_executor())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the only test we have in cpython/asyncio/tests?

Copy link
Contributor Author

@aeros aeros Jul 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it looks like this is the only test that uses it in CPython:

    def test_run_in_executor_cancel(self):
        called = False

        def patched_call_soon(*args):
            nonlocal called
            called = True

        def run():
            time.sleep(0.05)

        f2 = self.loop.run_in_executor(None, run)
        f2.cancel()
        self.loop.run_until_complete(
                self.loop.shutdown_default_executor())
        self.loop.close()
        self.loop.call_soon = patched_call_soon
        self.loop.call_soon_threadsafe = patched_call_soon
        time.sleep(0.4)
        self.assertFalse(called)

In retrospect, I probably should have added some dedicated test(s) for shutdown_default_executor() in the PR that added it in the first place, but at the time, I didn't have a decent understanding of asyncio unit tests. Should I add this one to the CPython regression tests and maybe a couple more if I can think of some that could be useful?

Note: It's also used in test_threads.py for a tearDown() and utils.py for close_loop(). But those are hardly dedicated tests, so I didn't include it above.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I add this one to the CPython regression tests and maybe a couple more if I can think of some that could be useful?

The more tests the better :) If they are not redundant, of course. I'll merge this PR now, huge thanks!

* Adds loop.shutdown_default_executor(), a new coroutine method added to
  asyncio in 3.9 for safe finalization of the event loop's ThreadPoolExecutor.
@1st1 1st1 merged commit 6ef69a7 into MagicStack:master Jul 10, 2020
@sileht sileht mentioned this pull request Nov 11, 2020
@qeternity
Copy link

Over 4 months since this was merged - is there any hope for a new pypi release?

@1st1
Copy link
Member

1st1 commented Nov 11, 2020

In the next couple of weeks.

@qeternity
Copy link

@1st1 not to be a nag but anything that the community can do in terms of preflight checks?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement shutdown_default_executor()
3 participants