Async/await， read file error: greenlet.error: cannot switch to a different thread

[woods-dev]

When i try to read file on windows 10, it is work,
but i try to read file on linux, it is error:

Traceback (most recent call last):
File "***.py", line 75, in decorated
return await func(args, kwargs)
File ".py", line 42, in design
file = request.files.get('file').read().decode(encoding='UTF-8')
File "/data/Python/python396/lib/python3.9/site-packages/werkzeug/utils.py", line 97, in get
value = self.fget(obj) # type: ignore
File "/data/Python/python396/lib/python3.9/site-packages/werkzeug/wrappers/request.py", line 499, in files
self._load_form_data()
File "/data/Python/python396/lib/python3.9/site-packages/flask/wrappers.py", line 113, in _load_form_data
RequestBase._load_form_data(self)
File "/data/Python/python396/lib/python3.9/site-packages/werkzeug/wrappers/request.py", line 285, in _load_form_data
data = parser.parse(
File "/data/Python/python396/lib/python3.9/site-packages/werkzeug/formparser.py", line 265, in parse
return parse_func(self, stream, mimetype, content_length, options)
File "/data/Python/python396/lib/python3.9/site-packages/werkzeug/formparser.py", line 150, in wrapper
chunk = stream.read(1024 * 64)
File "/data/Python/python396/lib/python3.9/site-packages/gevent/pywsgi.py", line 320, in read
return self._do_read(length)
File "/data/Python/python396/lib/python3.9/site-packages/gevent/pywsgi.py", line 182, in _do_read
read = reader(length)
File "/data/Python/python396/lib/python3.9/socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "/data/Python/python396/lib/python3.9/site-packages/gevent/_socketcommon.py", line 693, in recv_into
self._wait(self._read_event)
File "src/gevent/_hub_primitives.py", line 317, in gevent._gevent_c_hub_primitives.wait_on_socket
File "src/gevent/_hub_primitives.py", line 322, in gevent._gevent_c_hub_primitives.wait_on_socket
File "src/gevent/_hub_primitives.py", line 304, in gevent._gevent_c_hub_primitives._primitive_wait
File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
File "src/gevent/_hub_primitives.py", line 55, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
File "src/gevent/_waiter.py", line 154, in gevent._gevent_c_waiter.Waiter.get
File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
greenlet.error: cannot switch to a different thread

Environment:
Linux vm10-62-3-2 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Python version:
python3.9.6
Flask version:
flask2.0
Server:
from gevent.pywsgi import WSGIServer
from geventwebsocket.handler import WebSocketHandler
http_server = WSGIServer(('127.0.0.1', 5000), app, handler_class=WebSocketHandler)

[teymour-aldridge]

Do you have more information (i.e. could you provide steps that would make it possible to reproduce this)?

[woods-dev]

Client(requests or axios)

requests.post('/async_upload', files={'file': open('test.jmx')} )

Router:

app.add_url_rule('/async_upload', 'async_upload', View().async_upload, methods=['POST'])

View

class View:

def upload(self):
    if request.method == 'POST':
        file = request.files.get('file')
        loop = asyncio.new_event_loop()
        #  do something
        res = loop.run_until_complete(coro(file))
        loop.close()  

async def async_upload(self):
    if request.method == 'POST':
        file = request.files.get('file')
        #  do something
       await coro(file)            

resolve

when use “async_upload” upload file on linux， it throw error
I have to use another way "upload" to upload file.

[onstop4]

This is probably because the upload method is creating its own loop, while async_upload gets called by Flask through asgiref's AsyncToSync class, which will run it in a subthread, conflicting with gevent. This doesn't explain why it works on Windows though.

[davidism]

Yeah, this is an issue with you creating another event loop manually. That's no longer necessary with Flask 2.0, because an async def view will automatically run in an event loop, so you can await directly. There is probably some way to use asgiref or asyncio to run in a nested loop, but that's outside the scope of what we can help with here, and again should not be necessary.

from flask import Flask
from flask import redirect
from flask import request
from flask import url_for
from gevent.pywsgi import WSGIServer
from werkzeug.utils import secure_filename

app = Flask(__name__)


class View:
    async def upload(self):
        if request.method == "POST":
            file = request.files["file"]
            file.save(secure_filename(file.filename))
            return redirect(url_for("upload"))

        return "<!doctype html><form method=post enctype=multipart/form-data><input type=file name=file><input type=submit></form>"


app.add_url_rule("/", "upload", View().upload, methods=["GET", "POST"])
WSGIServer(("localhost", 5000), app).serve_forever()