Skip to content

Conversation

theanarkh
Copy link
Contributor

@theanarkh theanarkh commented Nov 26, 2022

fix diagnostics channel memory leak.
cc @Qard

const { subscribe, unsubscribe } = require('diagnostics_channel');

function noop() {}

console.log(process.memoryUsage().heapUsed);

for (let i = 0; i < 1000000; i++) {
    subscribe(String(i), noop);
    unsubscribe(String(i), noop);
}

gc();

console.log(process.memoryUsage().heapUsed);
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Nov 26, 2022
@theanarkh theanarkh force-pushed the fix_diagnostics_channel_memory_leak branch from ae3c2f9 to 2026e9c Compare November 26, 2022 19:21
@Qard
Copy link
Member

Qard commented Nov 26, 2022

LGTM. Not really how users are intended to use diagnostics_channel, but how users are meant to use things and how they actually use them doesn't always match, so seems like a reasonable fix to me. 😅

Copy link
Member

@anonrig anonrig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@theanarkh theanarkh added the request-ci Add this label to start a Jenkins CI on a PR. label Nov 27, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Nov 27, 2022
@nodejs-github-bot
Copy link
Collaborator

@RaisinTen
Copy link
Member

Could you add a test for this? Maybe by comparing the values of process.memoryUsage().heapUsed before and after the loop and gc call?

@theanarkh
Copy link
Contributor Author

Could you add a test for this? Maybe by comparing the values of process.memoryUsage().heapUsed before and after the loop and gc call?

I think it is difficult to do that, do you have any idea? Thanks!

@RaisinTen
Copy link
Member

I think it is difficult to do that, do you have any idea? Thanks!

I meant using the code you shared in the PR description with a slight modification - asserting that the value of process.memoryUsage().heapUsed after the loop+gc is less than or equal to the value before. Like this:

// Flags: --expose-gc
'use strict';

// This test ensures that diagnostic channel references aren't leaked.

require('../common');
const { ok } = require('assert');

const { subscribe, unsubscribe } = require('diagnostics_channel');

function noop() {}

const heapUsedBefore = process.memoryUsage().heapUsed;

for (let i = 0; i < 1000000; i++) {
    subscribe(String(i), noop);
    unsubscribe(String(i), noop);
}

gc();

const heapUsedAfter = process.memoryUsage().heapUsed;

ok(heapUsedBefore >= heapUsedAfter);

Is it difficult because it's too flaky?

@theanarkh theanarkh force-pushed the fix_diagnostics_channel_memory_leak branch from 2026e9c to cf83822 Compare November 28, 2022 14:37
@theanarkh
Copy link
Contributor Author

theanarkh commented Nov 28, 2022

Thanks for the suggestion. When i add the test(i = 1000000), i find the output is as follows.

before gc 5350312
after  gc 15322320

But when i change the code channels[name] = null; to delete channels[name];, the output is as follows.

before gc 5353152
after  gc 4639440

Then i take the snapshot before and after the loop. I find It takes up ten more megabytes of memory in channels[name] = null;(But when i is 100, the output of two cases is the same) 😄.
image

@theanarkh theanarkh force-pushed the fix_diagnostics_channel_memory_leak branch from cf83822 to 4fb75ce Compare November 28, 2022 15:02
Copy link
Member

@RaisinTen RaisinTen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@theanarkh theanarkh added the request-ci Add this label to start a Jenkins CI on a PR. label Nov 29, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Nov 29, 2022
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@theanarkh theanarkh added the commit-queue Add this label to land a pull request using GitHub Actions. label Nov 29, 2022
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Nov 29, 2022
@nodejs-github-bot nodejs-github-bot merged commit d579bc1 into nodejs:main Nov 29, 2022
@nodejs-github-bot
Copy link
Collaborator

Landed in d579bc1

ErickWendel pushed a commit to ErickWendel/node that referenced this pull request Nov 30, 2022
PR-URL: nodejs#45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
targos pushed a commit that referenced this pull request Dec 12, 2022
PR-URL: #45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
danielleadams pushed a commit that referenced this pull request Dec 30, 2022
PR-URL: #45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
danielleadams pushed a commit that referenced this pull request Dec 30, 2022
PR-URL: #45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
danielleadams pushed a commit that referenced this pull request Jan 3, 2023
PR-URL: #45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
danielleadams pushed a commit that referenced this pull request Jan 4, 2023
PR-URL: #45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
danielleadams pushed a commit that referenced this pull request Jan 5, 2023
PR-URL: #45633
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants