[RF] Problem of memory leak (?) with RooDataSet

[marcescalier]

Describe the bug

I open a workspace (RooWorkspace) that contains a pdf and various parameters. From these I do a loop on pseudo-experiments ("toys") that generate a RooDataSet.

For each toy in the loop, after generation, I delete the RooDataset.

But the memory, as reported by the class ProcInfo_t, never decreases.

It means that either ProcInfo_t is not actually showing the current memory information, or that is shows the "integrated memory used since the beginning",
or that RooDataSet would have a memory leak ?
or that I have a bug in my program ?

I provide a minimum reproducible example.
See for example : "EXAMPLE OF PROBLEM"

The minimum program is here :
/afs/cern.ch/work/e/escalier/public/ForOthers/ProblemMemory

Expected behavior

the expected behaviour would be that when I delete the RooDataSet, the memory would reduce.

To Reproduce

The README shows the quick steps for allowing to read the root file

Setup

I used root 6.18.00 to reproduce the problem on a french cluster of computer

On lxplus, I reproduce the problem, with root 6.20.02

Additional context

[jalopezg-git]

Hi @marcescalier,

I have assigned this to Jonas; he will take a look as soon as he can. :-)

Cheers.

[marcescalier]

Thank you. Good luck to Jonas if he finds time for the investigation.

[marcescalier]

Hello
Are there news ?

[guitargeek]

Hi, I'm glad you asked! I tried to reproduce this issue myself a few weeks ago actually, but I couldn't see the memory increase. Which ROOT version are you using?

If this is still a problem for you with the current ROOT version, I will continue the investigation.

[marcescalier]

Thank you.
I was using root 6.18.00.

Which version of root do you use without having the problem ?

Could you check that in "EXAMPLE Of PROBLEM", you really have a decrease of memory consumption ?

I tried again (same problem)
-on lxplus, with root 6.20.02 : same problem
proof : the deletion of a RooDataSet does not reduce the memory consuption :

just after creation of dataset toy that uses previous toy per category
res memory=243.1718750000 Mbytes
vir memory=568.2500000000 Mbytes
EXAMPLE OF PROBLEM : the memory does not decrease !
just after deletion of toy per category
res memory=243.3007812500 Mbytes
vir memory=568.2500000000 Mbytes

-on a french cluster of computer, with root 6.26.04 : same problem
just after creation of dataset toy that uses previous toy per category
res memory=315.2851562500 Mbytes
vir memory=711.7109375000 Mbytes
EXAMPLE OF PROBLEM : the memory does not decrease !
just after deletion of toy per category
res memory=315.3359375000 Mbytes
vir memory=711.7109375000 Mbytes

[marcescalier]

To summarize : the problem is not a memory increase, but the fact that when we delete the RooDataSet,

so there is an intrinsic memory leak of roofit : the allocated memory is not given back to the system when we delete the RooDataset.

[guitargeek]

Okay! Then it's very likely related to the memory pool, which will be disabled in ROOT 6.28 as of this PR:
#8324

Can you please try if you still see you problem with the ROOT nighlty build?
https://root.cern/install/nightlies/