Other event loop scopes?

[2e0byo]

The recent changes deprecated redefining the event_loop fixture, but provided no replacement for scope=package. Presumably the move away from the fixture was taken for good reasons, but I'd like to suggest support for (at least) package scope is worth having.

Package scope is really handy: you can have a directory structure like this:

a/
├─ test_foo.py
├─ test_bar.py
b/
├─ test_baz.py
├─ test_blah.py

with expensive fixtures per-directory, spin them up once per dir and then clean them up (freeing ram) before moving to the next dir.

I know this will work just fine with a session scoped event loop (which is also no longer possible?), but matching the event loop scope to the context has advantages:

• every new testing context (new package) starts in a clean loop
• failing fixtures or leftover state (locks, unawaited coros) throw at the package level, so you get debugging quickly
• certain exotic failure modes where stale state on the loop is inadvertently cleaned up elsewhere (ahem) only show if the loop is explicitly torn down on exiting the context, effectively asserting nothing is running on it

Thus I'd like to suggest support for package scope is added to asyncio_event_loop. For completeness, the scopes are:

• function: default
• class: already available
• module: already available
• package: missing
• session: missing

I also think it is worth implementing session scoping, although I think package is normally the right scope. It's possible I'm missing something here and this is already possible.

[seifertm]

That's a fair point.

I deliberately omitted session scope, but I wasn't aware of package scope.

Do you have suggestion how package-scoped (and session-scoped) loops should be defined?

[val-melev]

what's the rationale for omitting session scope?

[seifertm]

what's the rationale for omitting session scope?

There were essentially two ideas for replacing event loop fixture overrides. The first was to have a config option (similar to asyncio_mode) that defines the scope of the event loop. However, this would force all tests in a project to use the same event loop scope. Since a single project can have a mix of small "unit" tests and larger tests (e.g. involving starting a web server), this approach seemed too limiting.

The other option was to tie the loop scope to a pytest mark. I'm not aware of a way for users to add a mark to the whole session, unless they know about pytest hooks. So in a way, it was an accepted drawback of the current solution.

@val-melev Do you have a use case that requires session-scoped loops? If so, can you share it?

[2e0byo]

I deliberately omitted session scope, but I wasn't aware of package scope.

I think if you add package, you probably end up adding session as well: package scope in the top level package looks like session to me, although I don't know if pytest will create/tear down in a slightly different place.

Do you have suggestion how package-scoped (and session-scoped) loops should be defined?

Do you mean behaviour? In which case I think the old (fixture) behaviour is good: the loop gets created when the first test in the package requests it, which happens when you run the first async test, and is destroyed when the last test in the package releases it.

If you mean implementation I don't know. I need to look at the new code and its motivation before I can suggest something.

As an aside, I wonder if the current behaviour is more breaking than it might appear. It's quite common in the db/backend world to have things like engines or session factories which are bound to an event loop when first accessed and throw if that loop goes away. Whilst you can handle this all in fixtures, it makes sense in integration testing (in my opinion) to let the code do it it's normal way and just override the configuration. I agree that long-lived sessions of any kind don't normally belong in unit testing.

[seifertm]

Do you have suggestion how package-scoped (and session-scoped) loops should be defined?

Do you mean behaviour? In which case I think the old (fixture) behaviour is good: the loop gets created when the first test in the package requests it, which happens when you run the first async test, and is destroyed when the last test in the package releases it.

I was rather wondering how you'd write your test code so it gives you a package or session scoped loop.
Given that event loop fixture overrides are going to be deprecated and you cannot @pytest.mark packages or sessions, I'm looking for ideas :)

[2e0byo]

hmm, I can think of a few options, though I'm just brainstorming:

1. don't deprecate fixtures, at least in the package/session case (I really need to go and read the thread to understand if this is a stupid suggestion, but if it's just about stopping users cutting themselves this could be a way to go: overlapping fixtures would go away, and exactly one of session_event_loop or package_event_loop could be overridden, implemented by default as yield None. I don't know if this would work with the new code.)

2. use a fixture with the right scope to set a mark on the test, via pytestconfig. Off the top of my head I don't know whether that would work in terms of lifetimes, but if it would I'd personally consider it an acceptable workaround. I don't mind using hooks, I just don't want to reimplement collection. Edit sorry, this is nonsense isn't it, as the fixture will only get evaluated once, so it only has access to one of the function names.

3. use a hook and do it at collection by inspecting the module. I'd really like to avoid this. Most python devs don't use pytest hooks but do use marks/fixtures. If I do hook magic inside a fixture called register_custom_event_loop anyone can grok what it does from the name and where from the fixture scope.

If you have it handy, could we link this issue to the discussion which motivated this change?

[kusaku]

@val-melev Do you have a use case that requires session-scoped loops? If so, can you share it?

For example, the presence of other async fixtures that have a scope wider than function will result in You tried to access the function scoped fixture event_loop with a module scoped request object, involved factories: ...

This is why we "extend" the standard scope of the event_loop fixture.

[mthpower]

Hi, I hope it's helpful to this discussion if I can share my use case for an async fixture which we scope to the session.

# conftest.py

@pytest.fixture(scope="session")
async def test_database(database_url: URL) -> URL:
    await create_database(database_url)
    return database_url

We use a fixture like this to create a database to test a web application. We want the database to be created at the point when we start the test session, and reuse that test database for the whole session. This fixture requests the event loop fixture, with session scope.

So far, we've been using a fixture like so:

# conftest.py

@pytest.fixture(scope="session")
def event_loop() -> Iterator[asyncio.AbstractEventLoop]:
    loop = asyncio.new_event_loop()
    yield loop
    loop.close()

From what I've read so far, I don't see a way to make this work with version 0.22.0.

[2e0byo]

@seifertm (and everyone else) how would you feel about this interface?

@pytest.fixture(scope="package", autouse=True)
def custom_event_loop_scope() -> str:
    return "package"

Or, IMHO clearer:

from pytest_asyncio import SCOPE

@pytest.fixture(scope="package", autouse=True)
def custom_event_loop_scope() -> SCOPE:
    return SCOPE.package

? This could be inspected in the collection code just like a mark and appropriate action taken.

Arguments in favour:

• other packages use 'config fixtures' a lot, e.g. pytest-recording, pytest itself (arguably, with e.g. pytest_addoption)
• fixtures are trivial to broadcast over scopes, but marks are neater in the function/class/module case, so we can have both
• from a glance through the code now I don't think this would be very hard to implement, and it doesn't really add two logic flows like having both the mark and overriding the loop fixture does.

Alternatively the fixture could supply an eventloop policy directly. That might be a fix for #646, since fixture parametrisation would push the problem out of pytest-asyncio's flow?

@pytest.fixture(scope="package")
def event_loop_policy() -> AbstractEventLoopPolicy:
    return DefaultEventLoopPolicy()

This form I think is more powerful, but it would make contradictions less obvious between e.g. a function marked for module scope event loop but requesting a package level event loop policy. We'd still have the scope, so we could just raise at collection.

If this is something you'd be interested in I can try to prototype it at the weekend.

[lindycoder]

Sharing use case from the other issue

We have session scoped async fixtures so we have this central fixture

@pytest.fixture(scope="session")
def event_loop() -> Generator[AbstractEventLoop, None, None]:
    """Make the loop session scope to use session async fixtures."""
    policy = asyncio.get_event_loop_policy()
    loop = policy.new_event_loop()
    yield loop
    loop.close()

And every single AsyncGenerator fixtures we have actually require event_loop to make sure the teardown is done before the loop is closed.

The main use case for us is we run black box tests using containers we launch part of session scoped fixtures and we return an httpx client configured to talk to the container which we async with AsyncClient(...) . Of course it would be possible to split the client in a lesser scoped fixture... but keeping it there is handy. We probably have other use-cases that i don't think of.

[2e0byo]

Incidentally I've found a few cases in my code where I deliberately requested the event loop fixture in a sync test (either because I needed to interact with a running event loop to test e.g. inspecting/getting loops, but all the methods were sync, or because something deep under the hood would need an event loop but the interface was sync). I don't think it's a problem to deprecate this pattern as you can get exactly the same effect by declaring the test as async def, but it might be worth mentioning in the release notes. I've no idea how common a usage it is.