Skip to content

Fix memory exhaustion problem caused by the logger#1161

Merged
xwings merged 2 commits intoqilingframework:devfrom
Phat3:fix/log_mem_exhaustion
May 27, 2022
Merged

Fix memory exhaustion problem caused by the logger#1161
xwings merged 2 commits intoqilingframework:devfrom
Phat3:fix/log_mem_exhaustion

Conversation

@Phat3
Copy link
Contributor

@Phat3 Phat3 commented May 25, 2022

Checklist

Which kind of PR do you create?

  • This PR only contains minor fixes.
  • This PR contains major feature update.
  • This PR introduces a new function/api for Qiling Framework.

Coding convention?

  • The new code conforms to Qiling Framework naming convention.
  • The imports are arranged properly.
  • Essential comments are added.
  • The reference of the new code is pointed out.

Extra tests?

  • No extra tests are needed for this PR.
  • I have added enough tests for this PR.
  • Tests will be added after some discussion and review.

Changelog?

  • This PR doesn't need to update Changelog.
  • Changelog will be updated after some proper review.
  • Changelog has been updated in my PR.

Target branch?

  • The target branch is dev branch.

One last thing


Hi!

This PR is a proposed fix for a memory exhaustion problem I found while creating multiple Qiling objects. Here's a snippet of code that triggers the bug:

import time
import qiling


with open('/usr/bin/true', 'rb') as f:
    CODE = f.read()

def create_ql_instance():
    ql = qiling.Qiling(code=CODE, archtype='x8664', ostype='linux')

for i in range(1000):
    create_ql_instance()
    print("Allocating")
    time.sleep(0.1)

This snippet will eventually saturate the RAM and the SWAP because the Qiling objects are not properly garbage collected.
The problem here is that when the Qiling object gets out of scope the Python root log manager still holds a reference to it. This is caused by the fact that the logger formatter needs to have a reference to it so it can retrieve the current thread information while formatting the log message.
Substituting the reference with a weak reference solves the problem since now when the object goes out of scope only weak references are pointing to it and therefore the garbage collector is allowed to free the space.

Phat3 added 2 commits May 25, 2022 09:49
Signed-off-by: Sebastiano Mariani <smariani@vmware.com>
Signed-off-by: Sebastiano Mariani <smariani@vmware.com>
@xwings xwings merged commit 105cf06 into qilingframework:dev May 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants