possible memory leak

[gbrandt]

Memory leak on simple application

The attached app, appears to leak memory consistently as tabs are opened and closed to view the page. In my tests the memory went up from 40.1MB to 40.9MB after opening and closing 20 or 30 tabs. In my production app it is in the 10's of megabytes.

You need to install memory_profiler to see the increased usage, the increase is small on only Text elements (for instance) but adding ElevatedButton appears to make the leak bigger.

I have left the app running for hours to test if the GC collects this memory later, but it does not appear to.

Code example to reproduce the issue:

import logging
from memory_profiler import profile

import flet as ft


import flet as ft


@profile(precision=6)
def on_connect(e):
    logging.debug("Connected")


@profile(precision=6)
def on_disconnect(e):
    logging.debug("Disconnected")

@profile(precision=6)
def on_click(e):
    print( "Clicked" )

@profile(precision=6)
def main(page: ft.Page):
    page.on_connect = on_connect
    page.on_disconnect = on_disconnect
    page.controls.append(ft.Text(value="Hello, world!", color="green"))
    page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    page.controls.append(ft.Text(value="Hello, world!", color="green"))
    page.controls.append(ft.Text(value="Hello, world!", color="green"))
    page.controls.append(ft.Text(value="Hello, world!", color="green"))
    page.update()



ft.app(target=main, view=ft.WEB_BROWSER)

This is the memory dump on first connect:

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
    23  40.140625 MiB  40.140625 MiB           1   @profile(precision=6)
    24                                         def main(page: ft.Page):
    25  40.140625 MiB   0.000000 MiB           1       page.on_connect = on_connect
    26  40.140625 MiB   0.000000 MiB           1       page.on_disconnect = on_disconnect
    27  40.140625 MiB   0.000000 MiB           1       page.controls.append(ft.Text(value="Hello, world!", color="green"))
    28  40.140625 MiB   0.000000 MiB           1       page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    29  40.140625 MiB   0.000000 MiB           1       page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    30  40.140625 MiB   0.000000 MiB           1       page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    31  40.140625 MiB   0.000000 MiB           1       page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    32  40.140625 MiB   0.000000 MiB           1       page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    33  40.140625 MiB   0.000000 MiB           1       page.controls.append(ft.Text(value="Hello, world!", color="green"))
    34  40.140625 MiB   0.000000 MiB           1       page.controls.append(ft.Text(value="Hello, world!", color="green"))
    35  40.140625 MiB   0.000000 MiB           1       page.controls.append(ft.Text(value="Hello, world!", color="green"))
    36  40.140625 MiB   0.000000 MiB           1       page.update()

This is the memory dump 20 or 30 tabs later (all opened tabs are closed and a single connection is made again):

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
    23  40.906250 MiB  40.906250 MiB           1   @profile(precision=6)
    24                                         def main(page: ft.Page):
    25  40.906250 MiB   0.000000 MiB           1       page.on_connect = on_connect
    26  40.906250 MiB   0.000000 MiB           1       page.on_disconnect = on_disconnect
    27  40.906250 MiB   0.000000 MiB           1       page.controls.append(ft.Text(value="Hello, world!", color="green"))
    28  40.906250 MiB   0.000000 MiB           1       page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    29  40.906250 MiB   0.000000 MiB           1       page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    30  40.906250 MiB   0.000000 MiB           1       page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    31  40.906250 MiB   0.000000 MiB           1       page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    32  40.906250 MiB   0.000000 MiB           1       page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    33  40.906250 MiB   0.000000 MiB           1       page.controls.append(ft.Text(value="Hello, world!", color="green"))
    34  40.906250 MiB   0.000000 MiB           1       page.controls.append(ft.Text(value="Hello, world!", color="green"))
    35  40.921875 MiB   0.015625 MiB           1       page.controls.append(ft.Text(value="Hello, world!", color="green"))
    36  40.921875 MiB   0.000000 MiB           1       page.update()

Describe the results you expected:
Expected memory to climb on new tab but to drop to original levels on close tab.

Additional information you deem important (e.g. issue happens only occasionally):

Flet version (pip show flet):

Name: flet
Version: 0.3.2
Summary: Flet for Python - easily build interactive multi-platform apps in Python
Home-page: 
Author: 
Author-email: Appveyor Systems Inc. <hello@flet.dev>
License: MIT
Location: /Users/gregor.brandt/Documents/Startbridge/web-app2/fe_env/lib/python3.11/site-packages
Requires: beartype, oauthlib, packaging, repath, requests, watchdog, websocket-client
Required-by: 

Operating system:

MacOS 13.1, using Firefox as browser

Additional environment details:

[FeodorFitsner]

Thanks for bringing up this subject! I'm going to take a look and then we can discuss it here.

[gbrandt]

@FeodorFitsner When exactly is on_disconnect called. The docs say when a tab is closed (for instance). But is it related to the web socket, if the web socket disconnects the message is fired? I am still trying to chase down a concrete example of the memory leak

[gbrandt]

The following program shows the leak as clearly as possible. It is idiomatic flet code, as per examples. If you run the code below and close the browser tab, you will see that the user control is never deleted. In this case the thread runs for ever. Neither will_unmount or __del__ is called. This is the cause of my leaks, a user closes a tab and the controls do not get cleaned up (not just user controls, all controls). If the on_disconnect is called when the web socket disconnects, then a short web socket issue will also cause a memory leak. 0.4.2 and 0.5.0 are identical. The control will never get deleted.

import os
import threading
from datetime import datetime

import flet as ft

class GreeterControl(ft.UserControl):
    def __init__(self):
        super().__init__()
        self.timer = threading.Timer(1, self.on_timer)

    def __del__(self):
        print("Hello from __del__")
        self.timer.cancel()

    def build(self):
        return ft.Text("Hello!")

    def did_mount(self):
        print("Hello from did_mount")
        self.timer.start()

    def will_unmount(self):
        print("Hello from will_unmount")
        self.timer.cancel()

    def on_timer(self):
        print(f"{datetime.now()} Hello from on_timer - {id(self)}")
        self.timer = threading.Timer(1, self.on_timer)
        self.timer.start()

def main( page: ft.Page):

    def on_disconnect( _: ft.ControlEvent):
        print("Hello from on_disconnect")

    def on_connect( _: ft.ControlEvent):
        print("Hello from on_connect")

    def on_close( _: ft.ControlEvent):
        print("Hello from on_close")

    page.on_disconnect = on_disconnect
    page.on_connect = on_connect
    page.on_close = on_close

    page.add( GreeterControl( ) )
    page.update()

os.environ["FLET_APP_LIFETIME_MINUTES"] = "1"

if __name__ == "__main__":
    ft.app( target=main, view=ft.WEB_BROWSER )

[gbrandt]

Also, you would think that you can delete controls on the on_close handler, but simply doing a clear on the controls list does not work, you also have to call update which throws an exception in the on_close hander.

What does work is to clear the control list in on_disconnect and call page.update but that means you have to rebuild the controls on the next on_connect which is not idiomatic flet. (and on_connect is not called on first connection so all kinds of special handling has to be put in place)

[FeodorFitsner]

what error do you get when calling update() in on_close handler?

[gbrandt]

This is the error received when call page.update() in the on_close handler.

[FeodorFitsner]

Right, as the session is already deleted on Flet web server side.

So, the solution would be in on_close going through the page controls recursively and calling will_unmount, right?

[gbrandt]

I think that would work. I think that maybe a helper function clear_controls or something like that. It could take in a page or control and walk the control tree and call unmount. But the on_close should do it automatically. That would allow a user to clear the controls on a page or inside another control and have it work as expected without having to call update.

[gbrandt]

What also has to occur is that whatever is holding the reference to the control needs to lose the reference so that the control can be garbage collected.

[FeodorFitsner]

A reference to a Page instance is removed from a holding dict on session close: https://github.com/flet-dev/flet/blob/main/sdk/python/packages/flet/src/flet/flet.py#L294-L296

I don't know, is there anything else preventing it from proper garbage-collecting?

[gbrandt]

I have to test after this first fix is out...but I think that that the on_xxx handlers are not cleared and keep their references. My main is a class and I have to manually unsubscribe my on_xxx before the class will GC.

[gbrandt]

Any movement forward on this, I'd love to help with the testing. it is a major blocker for my app.

[FeodorFitsner]

I'm currently looking into that. Want to check it for 0.5.0 release.

[FeodorFitsner]

I've managed to fix it! Will be in tomorrow's release.

[FeodorFitsner]

To verify that memory is released when a user session is "closed" I used the following application:

import logging

import flet as ft
from memory_profiler import profile

@profile(precision=6)
def on_connect(e):
    logging.debug("Connected")

@profile(precision=6)
def on_disconnect(e):
    logging.debug("Disconnected")

@profile(precision=6)
def on_close(e):
    logging.debug("Closed")

@profile(precision=6)
def on_click(e):
    print("Clicked")

@profile(precision=6)
def main(page: ft.Page):
    page.on_connect = on_connect
    page.on_disconnect = on_disconnect
    page.on_close = on_close
    page.controls.append(
        ft.Text(data=f"a" * (1024 * 1024 * 128), value="Hello, world!", color="green")
    )
    page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    page.controls.append(ft.ElevatedButton("Click me", on_click=on_click))
    page.controls.append(ft.Text(value="Hello, world!", color="green"))
    page.controls.append(ft.Text(value="Hello, world!", color="green"))
    page.controls.append(ft.Text(value="Hello, world!", color="green"))
    page.update()

ft.app(target=main, view=ft.WEB_BROWSER)

The trick there is Text.data value which "eats" 128 MB of RAM for each user session.

Additionally, I set session lifetime to 1 minute via environment variable:

export FLET_APP_LIFETIME_MINUTES=1

Running the app and opening 5 tabs I can see python process takes 669 MB:

Closing all 5 tabs and waiting for 1 minute I can observe "close" event handler called for all 5 sessions:

python process takes 29 MB (669 - 128 * 5):