Skip to content

Commit 575a253

Browse files
GH-82448: Add thread timeout for loop.shutdown_default_executor (#97561)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
1 parent 9a404b1 commit 575a253

File tree

6 files changed

+44
-9
lines changed

6 files changed

+44
-9
lines changed

Doc/library/asyncio-eventloop.rst

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -180,18 +180,27 @@ Running and stopping the loop
180180

181181
.. versionadded:: 3.6
182182

183-
.. coroutinemethod:: loop.shutdown_default_executor()
183+
.. coroutinemethod:: loop.shutdown_default_executor(timeout=None)
184184

185185
Schedule the closure of the default executor and wait for it to join all of
186186
the threads in the :class:`ThreadPoolExecutor`. After calling this method, a
187187
:exc:`RuntimeError` will be raised if :meth:`loop.run_in_executor` is called
188188
while using the default executor.
189189

190+
The *timeout* parameter specifies the amount of time the executor will
191+
be given to finish joining. The default value is ``None``, which means the
192+
executor will be given an unlimited amount of time.
193+
194+
If the timeout duration is reached, a warning is emitted and executor is
195+
terminated without waiting for its threads to finish joining.
196+
190197
Note that there is no need to call this function when
191198
:func:`asyncio.run` is used.
192199

193200
.. versionadded:: 3.9
194201

202+
.. versionchanged:: 3.12
203+
Added the *timeout* parameter.
195204

196205
Scheduling callbacks
197206
^^^^^^^^^^^^^^^^^^^^

Doc/library/asyncio-runner.rst

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ Running an asyncio Program
2828

2929
This function runs the passed coroutine, taking care of
3030
managing the asyncio event loop, *finalizing asynchronous
31-
generators*, and closing the threadpool.
31+
generators*, and closing the executor.
3232

3333
This function cannot be called when another asyncio event loop is
3434
running in the same thread.
@@ -41,6 +41,10 @@ Running an asyncio Program
4141
the end. It should be used as a main entry point for asyncio
4242
programs, and should ideally only be called once.
4343

44+
The executor is given a timeout duration of 5 minutes to shutdown.
45+
If the executor hasn't finished within that duration, a warning is
46+
emitted and the executor is closed.
47+
4448
Example::
4549

4650
async def main():

Lib/asyncio/base_events.py

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -561,8 +561,13 @@ async def shutdown_asyncgens(self):
561561
'asyncgen': agen
562562
})
563563

564-
async def shutdown_default_executor(self):
565-
"""Schedule the shutdown of the default executor."""
564+
async def shutdown_default_executor(self, timeout=None):
565+
"""Schedule the shutdown of the default executor.
566+
567+
The timeout parameter specifies the amount of time the executor will
568+
be given to finish joining. The default value is None, which means
569+
that the executor will be given an unlimited amount of time.
570+
"""
566571
self._executor_shutdown_called = True
567572
if self._default_executor is None:
568573
return
@@ -572,7 +577,13 @@ async def shutdown_default_executor(self):
572577
try:
573578
await future
574579
finally:
575-
thread.join()
580+
thread.join(timeout)
581+
582+
if thread.is_alive():
583+
warnings.warn("The executor did not finishing joining "
584+
f"its threads within {timeout} seconds.",
585+
RuntimeWarning, stacklevel=2)
586+
self._default_executor.shutdown(wait=False)
576587

577588
def _do_shutdown(self, future):
578589
try:

Lib/asyncio/constants.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,9 @@
2626
FLOW_CONTROL_HIGH_WATER_SSL_READ = 256 # KiB
2727
FLOW_CONTROL_HIGH_WATER_SSL_WRITE = 512 # KiB
2828

29+
# Default timeout for joining the threads in the threadpool
30+
THREAD_JOIN_TIMEOUT = 300
31+
2932
# The enum should be here to break circular dependencies between
3033
# base_events and sslproto
3134
class _SendfileMode(enum.Enum):

Lib/asyncio/runners.py

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
from . import events
1010
from . import exceptions
1111
from . import tasks
12-
12+
from . import constants
1313

1414
class _State(enum.Enum):
1515
CREATED = "created"
@@ -69,7 +69,8 @@ def close(self):
6969
loop = self._loop
7070
_cancel_all_tasks(loop)
7171
loop.run_until_complete(loop.shutdown_asyncgens())
72-
loop.run_until_complete(loop.shutdown_default_executor())
72+
loop.run_until_complete(
73+
loop.shutdown_default_executor(constants.THREAD_JOIN_TIMEOUT))
7374
finally:
7475
if self._set_event_loop:
7576
events.set_event_loop(None)
@@ -160,8 +161,8 @@ def run(main, *, debug=None):
160161
"""Execute the coroutine and return the result.
161162
162163
This function runs the passed coroutine, taking care of
163-
managing the asyncio event loop and finalizing asynchronous
164-
generators.
164+
managing the asyncio event loop, finalizing asynchronous
165+
generators and closing the default executor.
165166
166167
This function cannot be called when another asyncio event loop is
167168
running in the same thread.
@@ -172,6 +173,10 @@ def run(main, *, debug=None):
172173
It should be used as a main entry point for asyncio programs, and should
173174
ideally only be called once.
174175
176+
The executor is given a timeout duration of 5 minutes to shutdown.
177+
If the executor hasn't finished within that duration, a warning is
178+
emitted and the executor is closed.
179+
175180
Example:
176181
177182
async def main():
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Add *timeout* parameter to :meth:`asyncio.loop.shutdown_default_executor`.
2+
The default value is ``None``, which means the executor will be given an unlimited amount of time.
3+
When called from :class:`asyncio.Runner` or :func:`asyncio.run`, the default timeout is 5 minutes.

0 commit comments

Comments
 (0)