Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak 'ais.decode' method #210

Open
Shal-Ziar opened this issue Jul 15, 2021 · 2 comments
Open

Memory leak 'ais.decode' method #210

Shal-Ziar opened this issue Jul 15, 2021 · 2 comments

Comments

@Shal-Ziar
Copy link

Dear all,

I have a piece of software running in the cloud that uses the libais library, over long-term running I've noticed a consistent memory increase. After spending some time with tracemalloc I've found that it possibly lies somewhere in the libais library.

I'll include some code snippets of my code as well as a few snippets from the tracemalloc dumps
In my code I've linked the decode method as follows:

During class initialisation

self.ais_decode_method = ais.decode
    def _parser(self, msg):
        """
        This function should be sent the raw AIS message, it will put the decoded message into a internal dictionary.

        :param msg:
        :type msg:
        :return:
        :rtype:
        """
        s = msg.split(",")
        # Check if a message is present and if it has a valid header
        if not self.check_header(s[0]):
            self._print_hook(f"Received non AIS message {s[0]}")
            return False

        if not len(s) == 7:
            return False

        # variables to keep track of the type of AIS message
        msg_size = int(s[1])
        msg_part = int(s[2])

        # Check if it is a multiline message
        try:
            # One line AIS messages
            if msg_size == 1:
                # clear old message since it is a new call
                self.ais_decode = self.ais_decode_method(msg)
                return True

            # Two line AIS messages
            elif msg_size == 2:
                if msg_part == 1:
                    self.buffer = msg
                    self.ais_decode.clear()
                    return False
                else:
                    self.ais_decode = self.ais_decode_method(self.buffer, msg)
                    return True
        except Exception as e:
            self._print_hook(f"AIS message failed to decode:\n " f"{e}")
            # Clear old message decode
            self.ais_decode = {}
            self.buffer = ""

The tracemalloc snippet:

After 1 minute of running (note that the line is the self.ais_decode(msg) line):

cloud-state-machine/.venv/lib/python3.8/site-packages/cap_converter/ais_c/ais_converter.py:159: size=254 KiB, count=7072, average=37 B

After 16 minutes

cloud-state-machine/.venv/lib/python3.8/site-packages/cap_converter/ais_c/ais_converter.py:159: size=751 KiB, count=14489, average=53 B

As this code runs 24/7 in the cloud we see this memory use steadily increase until the code reboots because it runs outside of set parameters.

@Shal-Ziar Shal-Ziar changed the title Memory leak during long runs Memory leak 'ais.decode' method Jul 15, 2021
@Caliber-X
Copy link

@Shal-Ziar
Any updates on this?
I'm also facing the same issue with "ais.decode" and "ais.compatibility.gpsd.mangle" modules, while running multithreadly
Any help is appreciated

@Shal-Ziar
Copy link
Author

@Caliber-X, No I've opted to switch to another library.
This library doesn't seem to get active support.
pyais

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants