Optional Memory benchmark flag in benchmarking scripts

[yashLadha]

Is your feature request related to a problem? Please describe.
Recently i am trying. to reduce the memory allocation in some implementation and wanted to benchmark my solution, to avoid speed reduction and promote memory reduction. However, to my surprise, compare.js script currently only does the speed benchmarking.

Describe the solution you'd like
I wanted to propose a flag that enables the memory benchmarks in the comparison script. This can be used along with the speed benchmarking to infer that changes do not reduce the speed of already running code.

Describe alternatives you've considered
Open to possible hacks but would be great if it is included.

[bnoordhuis]

Can you - very precisely - explain what you want to measure and why?

Remember that Node.js is a garbage-collected runtime. Just looking at (for example) peak RSS is often meaningless.

A higher allocation rate in a long-running program normally is visible as an increase in CPU time (because the GC has to do more work), not as a bigger memory footprint.

[yashLadha]

@bnoordhuis I am just walking through the implementation of the priority queue and say that we were using array implementation to represent the priority queue. Whenever there is a new insertion it checks for the memory that the length is sufficient to insert the node or not and if not it increases the memory by the power of 2. That is all very efficient but when we do removal we do not track for the level, like if we removed all the node from level 3 but still we are keeping the memory for that level in the array of the priority queue. As we increase the level to n the number of elements that are remaining empty is 2 ** n which can be freed for now as they are not being used. So I made the change in implementation but wanted to know its impact on the memory being used.

[bnoordhuis]

That's an example where you will be hard-pressed to observe a change in memory usage directly. I assume your optimization is replacing heap[size + 1] = undefined with heap.splice(size)?

Those arrays are managed by V8. If they grow big, it means the GC has to do more work to clean up dead objects in order to fulfill new allocations. It won't however meaningfully change peak memory usage in long-running programs because the size of the heap is bounded.

In short-running programs you might see a difference in peak RSS, but only if the program exits before it fills up the heap. 130,000 arrays entries take up about 1 MB on x64 so only a really pathological test case would show a meaningful difference there.

What you could do is compare the output of --trace_gc and --trace_gc_verbose. The output of those flags isn't easily machine-parseable though.

[yashLadha]

Basically i am doing the same thing that was done for increasing the array length.

    if (size > 64 && (heap.length >> 1) === (size / 2))
      heap.length >>= 1;