[BUG] Support for ASGI startup events

[gsaiz]

Investigative information

Please provide the following:

• Timestamp: 2021-11-10T10:58:00
• Function App name: (debugging locally)
• Function name(s) (as appropriate):(debugging locally)
• Core Tools version: 3.0.3785 Commit hash: db6fe71b2f05d09757179d5618a07bba4b28826f (64-bit)

Repro steps

Provide the steps required to reproduce the problem:

1. Setup a new Function App, with an HTTP trigger and the following code in func/__init__.py:

import logging

import azure.functions as func
from azure.functions import AsgiMiddleware
from api_fastapi import app

IS_INITED = False


def run_setup(app, loop):
    """Workaround to run Starlette startup events on Azure Function Workers."""
    global IS_INITED
    if not IS_INITED:
        loop.run_until_complete(app.router.startup())
        IS_INITED = True


def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
    asgi_middleware = AsgiMiddleware(app)
    run_setup(app, asgi_middleware._loop)
    return asgi_middleware.handle(req, context)

2. After the 1st request, you will get the following exception:

[2021-11-10T09:58:11.178Z] Executed 'Functions.api_az_function' (Failed, Id=50cbe047-48bd-42be-9314-0194c2e17ec9, Duration=65ms)
[2021-11-10T09:58:11.181Z] System.Private.CoreLib: Exception while executing function: Functions.api_az_function. System.Private.CoreLib: Result: Failure
Exception: RuntimeError: Task <Task pending name='Task-39' coro=<AsgiResponse.from_app() running at C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.8/WINDOWS/X64\azure\functions\_http_asgi.py:65> cb=[_run_until_complete_cb() at C:\Users\User\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py:184]> got Future <Future pending> attached to a different loop

Expected behavior

Provide a description of the expected behavior.

• The Function should be able to fulfil any number of requests.

Actual behavior

Provide a description of the actual behavior observed.

• Because the AsgiMiddleware is instantiated for each call, the event loop is not reused and thus we get the exception.

Known workarounds

Provide a description of any known workarounds.

• Have a global AsgiMiddleware object as shown here:

import logging

import azure.functions as func
from azure.functions import AsgiMiddleware
from api_fastapi import app

IS_INITED = False
ASGI_MIDDLEWARE = AsgiMiddleware(app)


def run_setup(app, loop):
   """Workaround to run Starlette startup events on Azure Function Workers."""
   global IS_INITED
   if not IS_INITED:
       loop.run_until_complete(app.router.startup())
       IS_INITED = True


def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
   run_setup(app, ASGI_MIDDLEWARE._loop)
   return ASGI_MIDDLEWARE.handle(req, context)

Contents of the requirements.txt file:

Provide the requirements.txt file to help us find out module related issues.

azure-functions==1.7.2
fastapi==0.70.0

[gsaiz]

After more digging in azure.functions._http_asgi, I managed to reuse the worker's event loop like this:

import logging

import azure.functions as func
from azure.functions._http_asgi import AsgiResponse, AsgiRequest
from api_fastapi import app

IS_INITED = False


async def run_setup(app):
    """Workaround to run Starlette startup events on Azure Function Workers."""
    global IS_INITED
    if not IS_INITED:
        await app.router.startup()
        IS_INITED = True


async def handle_asgi_request(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
    asgi_request = AsgiRequest(req, context)
    scope = asgi_request.to_asgi_http_scope()
    asgi_response = await AsgiResponse.from_app(app, scope, req.get_body())
    return asgi_response.to_func_response()


async def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
    await run_setup(app)
    return await handle_asgi_request(req, context)

In Improve throughput performance of Python apps in Azure Functions, it is recommended to define the main function as asynchronous.

Shouldn't this then be the preferred way to run ASGI applications?

[martin-greentrax]

I'm facing a similar problem. Based on https://github.com/ecerami/fastapi_azure I have create an Azure function with FastAPI inside. Everything works very smooth so far but my startup event handler is not run

from fastapi import FastAPI
app = FastAPI()

@app.on_event("startup")
def startup_event():
    raise Exception()  # is not raised

[csuriano23]

After more digging in azure.functions._http_asgi, I managed to reuse the worker's event loop like this:

import logging

import azure.functions as func
from azure.functions._http_asgi import AsgiResponse, AsgiRequest
from api_fastapi import app

IS_INITED = False


async def run_setup(app):
    """Workaround to run Starlette startup events on Azure Function Workers."""
    global IS_INITED
    if not IS_INITED:
        await app.router.startup()
        IS_INITED = True


async def handle_asgi_request(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
    asgi_request = AsgiRequest(req, context)
    scope = asgi_request.to_asgi_http_scope()
    asgi_response = await AsgiResponse.from_app(app, scope, req.get_body())
    return asgi_response.to_func_response()


async def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
    await run_setup(app)
    return await handle_asgi_request(req, context)

In Improve throughput performance of Python apps in Azure Functions, it is recommended to define the main function as asynchronous.

Shouldn't this then be the preferred way to run ASGI applications?

This works for me too, but there is no way to perform the app global teardown operations (FastAPI shutdown event), since you have no hook on the running threadpool context of function worker.

This is dirty if you have some channel open (like an open AIOHttp session) :(

Note: You have to put this:

    global IS_INITED
    if not IS_INITED:
        await app.router.startup()
        IS_INITED = True

within a thread lock in order to avoid races

[rbudnar]

I've run across a different issue with the AsgiMiddleware and arrived at basically the same solution as posted above. I noticed that simple async functions do not behave as I would expect an async application should. For example, I tested the following:

from __app__.app import app
import azure.functions as func
import asyncio
import nest_asyncio
nest_asyncio.apply() # as suggested in docs

@app.get("/api/test")
async def test(code: Optional[str] = None):
    await asyncio.sleep(5)
    return "OK"

async def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
    return await AsgiMiddleware(app).handle(req, context)

If you hit this endpoint above with a few concurrent requests, you'll notice that they execute serially instead of concurrently (I assume because the AsgiMiddleware blocks with a call to run_until_complete here). E.g., if I hit this endpoint twice in rapid succession, the second request will take ~10 seconds to complete instead of ~5 seconds.

Using a plain azure function without the middleware works as expected (e.g., the above scenario completes in ~5 seconds with two concurrent requests):

async def main(req: func.HttpRequest, context: func.Context) -> func.HttpResponse:
    await asyncio.sleep(5)
    return "OK"

As mentioned, the posted solution resolves this, but I'm concerned about the implications of this as mentioned above. It would be great to see a robust solution for this in the near future if possible.

[thomasfrederikhoeck]

@rbudnar Yeah I'm struggling with this too Azure-Samples/fastapi-on-azure-functions#4 and it would be nice with an "official" solution and not a private method :-) #911 (comment) also avoids nest_asyncio

[thomasfrederikhoeck]

@rbudnar This blocking problem is fixed now here Azure/azure-functions-python-library#143 which also removes the need for nest_asyncio. But it doesn't solve this initial issue here.