Application.run_async leaking memory

[Chris3606]

Upon running the example code in examples\prompts\asyncio-prompt.py, I noticed the memory usage climbed rapidly and consistently on my machine -- simply leaving the application sit without doing anything with it would have python.exe using a gigabyte or more of RAM in a few minutes. This was run on a Windows 10 machine running Python 3.8.1.

A bit of investigation later, it appears that the issue may be the underlying app.run_async call is adding thousands of tasks to event loop during its execution. Modifying line 30 of the asyncio-prompt.py example file from its current print statement to print(f'Tasks: {len(asyncio.all_tasks())}') shows that the number of active tasks in the event loop grows very quickly during execution:

This is only an issue with the current master branch version -- it does not occur in prompt-toolkit 3.0.2 from pip, but I figured the issue might still be of use to report.

[jonathanslenders]

Hi @Chris3606 ,

Thanks you for reporting! This is clearly a bug.
It looks like it only happens on Windows.

I think it's here where it's going wrong somehow: https://github.com/prompt-toolkit/python-prompt-toolkit/blob/master/prompt_toolkit/input/win32.py#L528

[jonathanslenders]

Hi @Chris3606,

I think this should it: #1058
Can you see whether that works?

[Chris3606]

Confirmed, that fixes it, thanks!

While investigating this, though, I did notice a similar but much smaller-scale issue on Linux. Using the change to asyncio_prompt.py above, where the background task prints the number of tasks in the event loop, I noticed that the number of tasks in the list steadily grows at idle on a Linux Mint machine running python 3.7.5 as well (but at a much slower rate). The tasks do disappear when enter is pressed, but I found it a bit odd that the number of tasks was increasing while the program was idling and no input was being given?

[jonathanslenders]

Thanks for pointing out the issue on Linux!
This is fixed in this PR: #1061

We are starting a flush timer when a key is pressed. (When there is a key binding bound to the sequence AB, we wait for the B key press after A has been pressed. But if there's a timeout, we handle the binding for A anyway, even though a B could arrive in the future.) Instead of cancelling the old flush timer, we created a new one and checked with a counter whether the cancellation should happen or not.

I slightly changed the Windows PR:
https://github.com/prompt-toolkit/python-prompt-toolkit/pull/1058/files

[Chris3606]

Interesting! Just tested the master branch with the altered Windows PR merged, it's definitely fixed, thanks!

As for the Linux issue, I can't test too in-depth at the moment, but initially I'm still seeing a similar behavior as before in that case. I printed out the actual tasks in the event loop at each step in the example's background task (as well as the number of tasks), and at first glance there does appear to be a difference in the flush input tasks being queued before #1061 as opposed to after, however the number of tasks queued is still increasing by 3 at idle with no key-presses happening.

I need to make sure that my printing of the tasks/number of tasks itself isn't adding tasks to the loop that ordinarily wouldn't be there, but admittedly if that were the case I would expect to see a similar problem on Windows?