Importing app has loop already running

[wrnrlr]

This problem could very well be caused by me being confused about the correct usage of testing an existing Sanic application, but when I try using my app as a fixture like so

from myapp import server

@pytest.yield_fixture
def app():
    yield server.app

@pytest.fixture
def test_cli(loop, app, test_client):
    return loop.run_until_complete(test_client(app, protocol=WebSocketProtocol))

I get this error:

test setup failed
loop = <_UnixSelectorEventLoop running=False closed=False debug=False>
app = <sanic.app.Sanic object at 0x10a350ac8>
test_client = <function test_client.<locals>.create_client at 0x10a8330d0>

    @pytest.fixture
    def test_cli(loop, app, test_client):
>       return loop.run_until_complete(test_client(app, protocol=WebSocketProtocol))

tests/test_registration.py:18: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py:466: in run_until_complete
    return future.result()
/usr/local/lib/python3.6/site-packages/pytest_sanic/plugin.py:129: in create_client
    await client.start_server()
/usr/local/lib/python3.6/site-packages/pytest_sanic/utils.py:185: in start_server
    await self._server.start_server(loop=self._loop)
/usr/local/lib/python3.6/site-packages/pytest_sanic/utils.py:88: in start_server
    trigger_events(self.after_server_start, self.loop)
/usr/local/lib/python3.6/site-packages/sanic/server.py:354: in trigger_events
    loop.run_until_complete(result)
/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py:454: in run_until_complete
    self.run_forever()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_UnixSelectorEventLoop running=False closed=False debug=False>

    def run_forever(self):
        """Run until stop() is called."""
        self._check_closed()
        if self.is_running():
>           raise RuntimeError('This event loop is already running')
E           RuntimeError: This event loop is already running

Why is it behaving this way, If i simply define a test app in de closure of the app function everything works as documented, when i import it from somewhere else I get this error. BTW the server file gaurds the app.run(...) statement with an if __name__ == "__main__":. Any hints are more then welcome, slightly pulling on my own hair here.

[yunstanford]

i tried to reproduce your issue here, but it works fine for me.

1. server.py

from sanic import Sanic
from sanic.response import json

app = Sanic()

@app.route("/")
async def test(request):
    return json({"hello": "world"})

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8000)

2. conftest.py

from my_module.server import app as my_app


@pytest.yield_fixture
def app():
    yield my_app


@pytest.fixture
def test_cli(loop, app, test_client):
    return loop.run_until_complete(test_client(app))

3. test_pytest.py

async def test_pytest_test_client(test_cli):
    resp = await test_cli.get('/')
    assert resp.status == 200
    assert (await resp.json()) == {"hello": "world"}

Above works fine for me.

anyway, you got this error, mainly because the loop is already running.
seems like app.run has been executed. i am guessing

1. how did you start pytest ? maybe related, seems like your app.run inside 'name == "main":' has been executed. or you can try remove 'name == "main":'.
2. have you ran app somewhere else ?

two suggestions in general,

1. Create a test app instead of using the global app. It may cause problem if you use the global app setting for testing.
2. Never use __name__ == "__main__": if you can avoid. you can try create entry_points/console_scripts. It's not good pattern to put __name__ == "__main__": in you server.py module.

Anyways, it works fine here, i can't reproduce it from my side.

[wrnrlr]

Yun, thanks for your reply, it has been instrumental helping me further investigate this issue.

I've set a breakpoint on the asyncio's BaseEventLoop.run_forever method and when executing my app in test mode it gets called twice. This behaviour seems to be related to the usage of asyncpg connection pool.

You can reproduce this behaviour with this modified version of your server.py example.

from sanic import Sanic
from sanic.response import json
from asyncpg import create_pool

app = Sanic()

@app.listener('before_server_start')
async def init_db(app, loop):
  app.db = await create_pool(user='postgres', loop=loop, max_size=100)

@app.route("/")
async def test(request):
    return json({"hello": "world"})

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8000)

Any more hints, from my reading of the documentation this is the correct usage of pools.

[yunstanford]

are you talking about the "before_server_start" event has been executed twice ? Can you double check that ? Also Sanic version 0.5.4 has a bug that called before_server_start twice when start server in async mode. But it has been fixed in master branch.

Can you help me confirm that if "before_server_start" has been called twice or not ?

I don't have PostgreSQL instance for testing. But i can test it with mongo tonight.

[wrnrlr]

I did a pip3 install --upgrade git+https://github.com/channelcat/sanic in order to use the latest version but still I see the same behaviour. Look at the screenshots below for my stack trace.

[yunstanford]

Thx! got it, i will push a fix. maybe also a fix for Sanic.

[yunstanford]

ok, it should be fixed. i tested it locally with async mongo client. Please use Sanic master branch.

I will work on pushing a new release for Sanic, because 0.5.4 version of Sanic has a bunch of small issues.

Let me know if you have any question. glad to help.

[wrnrlr]

I've just tested the master branch with my asyncpg example and can confirm this issue has been resolved. Thx for taking care of this, it's good to know the sanic ecosystem is alive an well.

[psiofxt]

I'm wondering if this issue still exists for a bp.listener('before_server_start')? I have an app client that assigns an asyncpg connection pool with a blueprint. In my tests I assign my app the blueprint in the @pytest.yield_fixture and each test function calls this blueprint N times.

So for test_function_1 the before_server_start is called once.
For test_fucntion_2 the before_server_start is called twice.
For test_function_N the before_server_start is called N times.
and so on...

Is there something I'm missing? I've followed the pytest-sanic test_cli fixtures exactly.

Thanks for your time!

[yunstanford]

sorry, probably didn't understand what you asked. All fixtures will be destroyed and recreated for each single unit test.

[psiofxt]

I believe I found the issue, apologies for taking up some of your time. In my app fixture I was assigning my blueprint after initializing a new postgres test db and each time the fixture was recreated it was adding a new bp.