FR: Async requests

[eranhirs]

I am using firebase-admin on an asyncio project and when there are a lot of requests it slows down the whole API.

Surprisingly, I didn't see any other issues about this, I hope I am not duplicating.

ref = yield from db.reference
result = yield from ref.get()

Thanks

[hiranya911]

Can this be not handled in the application code? Basically execute Admin SDK methods like ref.get() on separate threads.

[eranhirs]

It won't scale as I would like and, to my opinion, complicates the code and is harder to track.

[hiranya911]

Most of our users are on Python 2.7, so we cannot directly expose async coroutines from the SDK anytime soon. In the meantime you should be able to get around the scalability issues by using the run_in_executor() method of the asyncio event loop.

ref = db.reference(path) # Doesn't make RPC calls

executor = concurrent.futures.ThreadPoolExecutor(max_workers=8)
loop = asyncio.get_event_loop()
loop.run_in_executor(executor, ref.get) # Make RPC call on a separate thread

[eranhirs]

In case anyone runs into the same problem -
Eventually, I am using the REST API.
Calling it asynchronously with aiohttp.

For auth I am using the database secret which is DEPRECATED, so it's not ideal, but it works for now.

[Natim]

@eranhirs would you feel like starting a async-firebase project that we could contribute to?

[hyperknot]

I wanted to report that having async support for an auth SDK is very important. This is exactly the area where async frameworks make a huge difference, where you do a client side request inside a server side request.

Please consider adding core support and examples for using with ASGI frameworks, like Starlette:
https://github.com/encode/starlette

[Lawouach]

I agree with the general feeling here. Not having async support is making this library a tad less useful nowadays. Not very future proof unfortunately.

[labroid]

+1 on this. My project has a few hundred thousand items that need to be worked via Firebase, and ~1 second latency times make wait times intractable without async support

[labroid]

Python 2.7 is no longer supported. Any progress here?

[hiranya911]

This needs to be first addressed in other GCP Python libraries that we use as dependencies. More specifically, the google-auth library (which we use to make all authorized HTTP calls) needs to support it. From what I can see, they have plans to add async support starting from 2020, but there is no concrete timetable as of now:

googleapis/google-auth-library-python#321
googleapis/google-cloud-python#3103

I can only imagine that this will take a while considering the engineering effort that is involved. Pretty much all existing APIs have to be re-evaluated and redesigned for async support.

If google-auth takes a very long time to introduce async support, we can consider wrapping the existing google-auth APIs in async threads, so that at least firebase-admin can expose an async API to the developer. But even that is going to require a lot of work, based on some prototyping results I've managed to put together.