asyncio: nested event loop

[djarb]

BPO	22239
Nosy	@jcea, @njsmith, @jab, @pmp-p, @vadmium, @1st1, @crusaderky, @douglas-raillard-arm, @davidbrochart, @kumaraditya303
Files	nested.py: Decorator allowing a coroutine to run in a nested event loop nested.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-08-25.18:04:32.900>
created_at = <Date 2014-08-20.18:03:50.601>
labels = ['type-feature', 'expert-asyncio']
title = 'asyncio: nested event loop'
updated_at = <Date 2021-12-11.07:40:23.171>
user = 'https://bugs.python.org/djarb'

bugs.python.org fields:

activity = <Date 2021-12-11.07:40:23.171>
actor = 'kumaraditya'
assignee = 'none'
closed = True
closed_date = <Date 2014-08-25.18:04:32.900>
closer = 'gvanrossum'
components = ['asyncio']
creation = <Date 2014-08-20.18:03:50.601>
creator = 'djarb'
dependencies = []
files = ['36422', '36498']
hgrepos = []
issue_num = 22239
keywords = ['patch']
message_count = 23.0
messages = ['225578', '225883', '226037', '298388', '373145', '373177', '373183', '373186', '373192', '373240', '373241', '373244', '373245', '373247', '373249', '373270', '373273', '373280', '373307', '373311', '389487', '398257', '399920']
nosy_count = 14.0
nosy_names = ['jcea', 'djarb', 'zzzeek', 'njs', 'jab', 'pmpp', 'martin.panter', 'yselivanov', 'Rokas K. (rku)', 'crusaderky', 'douglas-raillard-arm', 'davidbrochart', 'kumaraditya', 'rob.moore']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue22239'
versions = []

Linked PRs

• gh-66435: Allow nested event loops #93338

[djarb]

It's occasionally necessary to invoke the asyncio event loop from code that was itself invoked within (although usually not directly by) the event loop.

For example, imagine you are writing a class that serves as a local proxy for a remote data structure. You can not make the __contains__ method of that class into a coroutine, because Python automatically converts the return value into a boolean. However, __contains__ must invoke coroutines in order to communicate over the network, and it must be invokable from within a coroutine to be at all useful.

If the event loop _run_once method were reentrant, addressing this problem would be simple. That primitive could be used to create a loop_until_complete function, which could be applied to the io tasks that __contains__ needs to invoke

So, making _run_once reentrant is one way of addressing this request.

Alternately, I've attached a decorator that sets aside some of the state of _run_once, runs a couroutine to completion in a nested event loop, restores the saved state, and returns the coroutine's result. This is merely a proof of concept, but it does work, at least in my experiments.

[gvanrossum]

While I understand your problem, I really do not want to enable recursive event loops. While they are popular in some event systems (IIRC libevent relies heavily on the concept), I have heard some strong objections from other parts, and I am trying to keep the basic event loop functionality limited to encourage interoperability with other even loop systems (e.g. Tornado, Twisted).

In my own experience, the very programming technique that you are proposing has caused some very hard to debug problems that appeared as very infrequent and hard to predict stack overflows.

I understand this will make your code slightly less elegant in some cases, but I think in the end it is for the best if you are required to define an explicit method (declared to be a coroutine) for membership testing of a remote object. The explicit "yield from" will help the readers of your code understand that global state may change (due to other callbacks running while you are blocked), and potentially help a static analyzer find bugs in your code before they take down your production systems.

[djarb]

All right.

However, for anyone who's interested, here is a patch that enables nested event loops in asyncio, and the accompanying unit tests

[RokasKrku]

I understand rationale for rejection of this issue but i beg to reconsider.

Unlike in traditional coroutines (windows fibers / setjmp|longjmp with stack switching) we can not yield from any point of execution. There must be full async-await chain preserved. This basically divides code into two islands - async and non-async. And there seems to be no way to schedule async call from non-async code and get a response. While suggestion to make custom async def contains() call is a valid one we can not always do that. Consider the case when we have to do some networking calls in a function that is invoked by non-async library. Naturally it would be simple non-awaited call from which we can not call a coroutine and get a response. And since it is a library calling into our code we can not easily change it. It might even be completely unsuitable change for library in question.

I see two solutions to this problem (if i am missing something please point it out):

1. Reentrant loops as suggested in this issue.
2. Allow awaited calls from non-coroutines provided execution is invoked from a coroutine somewhere up the call stack.

First one is certainly easier to implement.

[zzzeek]

hey there,

I seem to have two cents to offer so here it is. An obscure issue in the Python bug tracker is probably not the right place for this so consider this as an early draft of something that maybe I'll talk about more elsewhere.

This basically divides code into two islands - async and non-async

yes, this is the problem, and at the bottom of this apparently somewhat ranty comment is a solution, and the good news is that it does not require Python or asyncio be modified. My concern is kind of around how it is that everyone has been OK with the current state of affairs for so long, why it is that "asyncio is fundamentally incompatible with library X" is considered to be acceptable, and also how easy it was to find a workaround, this is not something I would have expected to come up with. Kind of like you don't expect to invent Velcro or windshield wipers.

asyncio's approach is what those of us in the library/framework community call "explicit async", you have to mark functions that will be doing IO and the points at which IO occurs must also be marked. Long ago it was via callback functions, then asyncio turned it into decorators and yields, and finally PEP-492 turned it into async/await, and it is very nicely done. It is of course a feature of asyncio that writing out async/await means your code can in theory be clearer as to where IO occurs and all that, and while I don't totally buy that myself, I'm of course in favor of that style of coding being available, it definitely has its own kind of self-satisfaction built in when you do it. That's all great.

But as those of us in the library/framework community also know, asyncio's approach essentially means, libraries like Flask, Django, my own SQLAlchemy, etc. are all automatically "non-workable" with the asyncio approach; while these libraries can certainly have asyncio endpoints added to them, the task as designed is not that simple, since to go from an asyncio endpoint all the way through library code that doesn't care about async and then down into a networking library that again has asyncio endpoints, the publishing of "async" and the "await" or yield approach must be wired all the way through every function and method. This is all despite that when you're not at the endpoints, the points at which IO occurs is fully predictable such that libraries like gevent don't need you to write it. So we are told that libraries have to have full end-to-end rewrites of all their code to work this way, or otherwise maintain two codebases, or something like that.

The side effect of this is that a whole bunch of library and framework authors now get to create all new libraries and frameworks, which do exactly the same thing as all the existing libraries and frameworks, except they sprinkle the "async/await" keywords throughout middle tiers as required. Vague claims of "framework X is faster because it's async" appear, impossible to confirm as it is unknown how much of their performance gains come from the "async" aspect and how much of it is that they happened to rewrite a new framework from scratch in a completely different way (hint: it's the latter).

Or in other cases, as if to make it obvious how much the "async/await" keywords come down to being more or less boilerplate for the "middle" parts of libraries, the urllib3 project wrote the "unasync" project [1] so that they can simply maintain two separate codebases, one that has "async/await" and the other which just search-and-replaced them out.

SQLAlchemy has not been "replaced" by this trend as asyncio database libraries have not really taken off in Python, and there are very few actual async drivers. Some folks have written SQLAlchemy-async libraries that use SQLAlchemy's expression system while they have done the tedious, redundant and impossible-to-maintain work of replicating enough of SQLAlchemy's execution internals such that a modest "sqlalchemy-like" experience with asyncio can be reproduced. But these libraries are closed out from all of the fixes and improvements that occur to SQLAlchemy itself, as well as that these systems likely target a smaller subset of SQLAlchemy's behaviors and features in any case. They certainly can't get the ORM working as the ORM runs lots of SQL executions internally, all of which would have to propagate their "asyncness" outwards throughout hundreds of functions.

The asyncpg project, one of the few asyncio database drivers that exists, notes in its FAQ "asyncpg uses asynchronous execution model and API, which is fundamentally incompatible with SQLAlchemy" [2], yet we know this is not true because SQLAlchemy works just fine with gevent and eventlet, with no architectural changes at all. Using libraries like SQLAlchemy or Django with a non-blocking IO, event-based model is commonplace. It's the "explicit" part of it that is hard, which is because of how asyncio is designed, without any mediation for code that doesn't publish "async / await" keywords in the middle.

So I finally just sat down to figure out how to use the underlying greenlet library (which we all know as the portable version of "Stackless Python") to bridge the gap between asyncio and blocking-style code, it's about 30 lines and I have SQLAlchemy working with an async front-end to asyncpg DBAPI as can be seen at [3] based on the proof of concept at [4]. I'm actually running the full py.test suite all inside the asyncio event loop and running asyncpg through SQLAlchemy's whole battery of thousands of tests, all of them written in purely blocking style, and there's not any need to add "async / await / yield / etc" anywhere except the very endpoints, that is, where the top function is called, and then down where we call into asyncpg directly, using a function called await_() that works just like the "await" keyword. Just no "async" function declaration.

A day later, someone took the same idea and got Flask to work in an asyncio event loop at [5]. The general idea of using greenlet in this way is also present at [6], so I won't be patenting this idea today as oremanj can claim prior art.

Using greenlet, there is no need to break out of the asyncio event loop at all, nor does it change the control flow of parallel coroutines within the loop. It uses greenlet's "switch", quite minimally, to bridge the gap between code that does not push out an "async/await" yield and code that does. There are no threadpools, no alternate event loops, no monkeypatching, just a few greenlet.switch() calls in the right spots. A slight performance decrease of about 15%, but in theory one would only be using asyncio if their application is expected to be IO bound in any case (which folks that know me know is another assertion I frequently doubt).

So to sum up, last week, libraries like Flask and SQLAlchemy were "fundamentally incompatible" with asyncio, and this week they are not. What's confusing me is that I'm not that smart and this is something all of the affected libraries should have been doing years ago, and really, while I know this is not going to happen, this should be *part of asyncio itself* or at least a very standard approach so that nobody has to assume asyncio means "rewrite all your library code".

To add an extra bonus, you can use this greenlet approach to have blocking-style functions right in the middle of your otherwise asyncio application. Which means this also is a potential solution to the "lazy-loading" problem. You have an asyncio app that does lots of asyncio to talk to microservices, but some functions are doing database work and they really would like to just work in a transaction, load some objects and access their attributes without worrying that a SQL statement can't be emitted. This approach makes that possible as well. ORM lazy loading with the asyncpg driver: [7] . Indeed, if you have a PostgreSQL SQLAlchemy application already written in blocking style, you can use this new extension and drop the entire application into the event loop and use the asyncpg driver, not too unlike using gevent except nothing is monkeypatched.

The recipe is simple and so far appears to be very effective. Using greenlet to manipulate the stack is of course "spooky" and I would assume Python devs may propose that this would lead to hard-to-debug conditions. I've used gevent and eventlet for many years and while they do produce some new issues, most of them relate to the fact that they use monkeypatching of existing modules and particularly around low level network drivers like pymysql. The actual stack moving around within business logic doesn't seem to produce any difficult new issues. Using plain asyncio has a lot of novel and confusing failure modes too. Using the little bit of "spookyness" of greenlet IMO is a lot less work than rewriting SQLAlchemy, Django ORM, Flask, urllib3, etc. from scratch and maintaining two codebases though.

[1] https://pypi.org/project/unasync/

[2] https://magicstack.github.io/asyncpg/current/faq.html#can-i-use-asyncpg-with-sqlalchemy-orm

[3] https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/2071

[4] https://gist.github.com/zzzeek/4e89ce6226826e7a8df13e1b573ad354

[5] https://twitter.com/miguelgrinberg/status/1279894131976921088

[6] https://github.com/oremanj/greenback

[7] https://gerrit.sqlalchemy.org/plugins/gitiles/sqlalchemy/sqlalchemy/+/refs/changes/71/2071/10/examples/asyncio/greenlet_orm.py

[1st1]

Thanks for posting this, Mike.

Vague claims of "framework X is faster because it's async" appear, impossible to confirm as it is unknown how much of their performance gains come from the "async" aspect and how much of it is that they happened to rewrite a new framework from scratch in a completely different way (hint: it's the latter).

These kind of claims are not specific to async vs. sync. They are all over the place for every two pieces of comparable technologies. While novice users might base their technology choice purely on such benchmarks, it's less of an issue for startups/tech companies.

That said, I agree with most of your points so far.

The asyncpg project, one of the few asyncio database drivers that exists, notes in its FAQ "asyncpg uses asynchronous execution model and API, which is fundamentally incompatible with SQLAlchemy" [2], yet we know this is not true because SQLAlchemy works just fine with gevent and eventlet, with no architectural changes at all.

But it is true. Making asynchronous network requests in asyncio requires async/await or using callbacks and it's not possible to do them, say, from __getattr__ (you mention this yourself). This is what that particular comment is about, nothing more. Using gevent and eventlet as examples in this particular context isn't helping you. Apologies for nitpicking, I know it's not the point of this discussion.

A day later, someone took the same idea and got Flask to work in an asyncio event loop at [5]. The general idea of using greenlet in this way is also present at [6], so I won't be patenting this idea today as oremanj can claim prior art.

Yes, this approach definitely works and I even did that in production myself a few years ago with https://github.com/1st1/greenio (it's terribly outdated now).

The recipe is simple and so far appears to be very effective.

This recipe was one of the reasons why I added loop.set_task_factory method to the spec, so that it's possible to implement this in an existing event loop like uvloop. But ultimately asyncio is flexible enough to let users use their own event loop which can be compatible with greenlets among other improvements.

Ultimately, asyncio will probably never ship with greenlets integration enabled by default, but we should definitely make it possible (if there are some limitations now). It doesn't seem to me that nested event loops are needed for this, right?

[zzzeek]

This recipe was one of the reasons why I added loop.set_task_factory method to the spec, so that it's possible to implement this in an existing event loop like uvloop. But ultimately asyncio is flexible enough to let users use their own event loop which can be compatible with greenlets among other improvements.

Right, when I sought to look at this, I know that my users want to use the regular event loop in asyncio or whatever system they are using.

Ultimately, asyncio will probably never ship with greenlets integration enabled by default, but we should definitely make it possible (if there are some limitations now). It doesn't seem to me that nested event loops are needed for this, right?

So right, the approach I came up with does not need nested event loops and it's my vague understanding that nesting event loops is more difficult to debug, because you have these two separate loops handing off to each other.

What is most striking about my recipe is that it never even leaves the default event loop. Originally my first iteration when I was trying to get it working, I had a separate thread going on, as it seemed intuitive that "of course you need a thread to bridge async and sync code" but then I erased the "Thread()" part around it and then it just worked anyway. Like it's simple enough that shipping this as a third party library is almost not even worth it, you can just drop this in wherever. If different libraries had their own drop-in of this, they would even work together. greenlet is really like totally cheating.

the philosophical thing here is, usually in my many twitter debates on the subject, folks get into how they like the explicit async and await keywords and they like that IO is explicit. So I'm seeking to keep these people happy and give then "async def execute(sql)", and use an async DB driver, but then the library that these folks are using is internally not actually explicit IO. But they don't have to see that, I've done all the work of the "implicit IO" stuff and in a library it's not the same kind of problem anyway. I think this is a really critical technique to have so that libraries that mediate between a user-facing facade and TCP based backends no longer have to make a hard choice about if they are to support sync vs. async (or async with an optional sync facade around it).

[1st1]

I think this is a really critical technique to have so that libraries that mediate between a user-facing facade and TCP based backends no longer have to make a hard choice about if they are to support sync vs. async (or async with an optional sync facade around it).

If this works for such a big and elaborate framework as SQLA, we can definitely highlight this as a valid approach and even add a link to a blog post from the docs. We'll need to add an asyncio specific FAQ page for that or something similar.

Another approach, which would probably be a nonstarter for SQLA, is to use async/await for literally everything internally, and provide a tiny synchronous facade on top. Funny thing you don't even need an event loop for that, just the basic understanding of how coroutines work internally. I used this to create the edgedb-python package which has both sync and async first-class support with one code base. Sync is even faster there in simple throughput benchmarks (as expected).

[zzzeek]

yes so if you have async/await all internal, are you saying you can make that work for synchronous code *without* running the event loop? that is, some kind of container that just does the right thing? my concern with that would still be performance. When asyncio was based on yield and exception throws, that was a lot of overhead to add to functions and that was what my performance testing some years back showed. w/ async/await I'm sure things have been optimized, but in general when i have function a() -> b() -> c(), I am trying to iron as much Python overhead as I possibly can out of that and I'd be concerned that the machinery to work through async/await would add latency. additionally if it was async/await internally but then i need to access the majority of Python DBAPIs that are sync, I need a thread pool anyway, right? which is also another big barrier to jump over.

It seems you were involved with urllib3's approach to use a code rewriter rather than a runtime approach based on the discussion at urllib3/urllib3#1323 , but it's not clear if Python 2 compatibility was the only factor or if the concern of "writing a giant shim" was also.

[njsmith]

Yeah, writing a trivial "event loop" to drive actually-synchronous code is easy. Try it out:

-----

async def f():
    print("hi from f()")
    await g()

async def g():
    print("hi from g()")

# This is our event loop:
coro = f()
try:
    coro.send(None)
except StopIteration:
    pass

---

I guess there's technically some overhead, but it's tiny.

I think dropping 'await' syntax has two major downsides:

Downside 1: 'await' shows you where context switches can happen: As we know, writing correct thread-safe code is mind-bendingly hard, because data can change underneath your feet at any moment. With async/await, things are much easier to reason about, because any span of code that doesn't contain an 'await' is automatically atomic:

---

async def task1():
    # These two assignments happen atomically, so it's impossible for
    # another task to see 'someobj' in an inconsistent state.
    someobj.a = 1
    someobj.b = 2

---

This applies to all basic operations like __getitem__ and __setitem__, arithmetic, etc. -- in the async/await world, any combination of these is automatically atomic.

With greenlets OTOH, it becomes possible for another task to observe someobj.a == 1 without someobj.b == 2, in case someobj.__setattr__ internally invoked an await_(). Any operation can potentially invoke a context switch. So debugging greenlets code is roughly as hard as debugging full-on multithreaded code, and much harder than debugging async/await code.

This first downside has been widely discussed (e.g. Glyph's "unyielding" blog post), but I think the other downside is more important:

• 'await' shows where cancellation can happen: Synchronous libraries don't have a concept of cancellation. OTOH async libraries *are* expected to handle cancellation cleanly and correctly. This is *not* trivial. With your sqlalchemy+greenlets code, you've introduced probably hundreds of extra unwinding paths that you've never tested or probably even thought about. How confident are you that they all unwind correctly (e.g. without corrupting sqlalchemy's internal state)? How do you plan to even find them, given that you can't see the cancellation points in your code? How can your users tell which operations could raise a cancelled exception?

AFAICT you can't reasonably build systems that handle cancellation correctly without some explicit mechanism to track where the cancellation can happen. There's a ton of prior art here and you see this conclusion over and over.

tl;dr: I think switching from async/await -> greenlets would make it much easier to write programs that are 90% correct, and much harder to write programs that are 100% correct. That might be a good tradeoff in some situations, but it's a lot more complicated than it seems.

[1st1]

Yeah, writing a trivial "event loop" to drive actually-synchronous code is easy. Try it out:

This is exactly the approach I used in edgedb-python.

I guess there's technically some overhead, but it's tiny.

Correct, the overhead isn't even detectable in microbenchmarks. In most async programs regular function calls dominate awaits by a few factors of magnitude.

I think dropping 'await' syntax has two major downsides:

For the extra context, in the case of using this approach for something like edgedb-python these downsides don't really apply, because we're adapting async/await implementation to be sync. The async/await code can handle cancellation etc. whereas the sync code only needs to support the general protocol parsing flow. FWIW I don't think it would be possible to apply my approach to SQLA without a very invasive rewrite, which isn't probably worth it.

tl;dr: I think switching from async/await -> greenlets would make it much easier to write programs that are 90% correct, and much harder to write programs that are 100% correct. That might be a good tradeoff in some situations, but it's a lot more complicated than it seems.

Yeah, this sums up my opinion on this topic.

Also, having spent a couple of years writing and debugging big and small greenlet-heavy code bases I wouldn't want to touch them ever again. Your mileage may vary, Mike.

[zzzeek]

With greenlets OTOH, it becomes possible for another task to observe someobj.a == 1 without someobj.b == 2, in case someobj.__setattr__ internally invoked an await_(). Any operation can potentially invoke a context switch. So debugging greenlets code is roughly as hard as debugging full-on multithreaded code, and much harder than debugging async/await code.

I would invite you to look more closely at my approach. The situation you describe above applies to a library like gevent, where IO means a context switch that can go anywhere. My small recipe never breaks out of the asyncio event loop, and it only context switches within the scope of a single coroutine, not to any arbitrary coroutine. So I don't think the above issue applies.

Additionally, we are here talking about *libraries* that are independently developed and tested distinct from end-user code. If there's a bug in SQLAlchemy, the end user isn't the person debugging that. arguments over "is async or sync easier to debug" are IMO pretty subjective and at this point they are not relevant to what sync-based libraries should be doing.