Memory deallocation in ESMC

[AbhiHampiholi]

Requirements

• Reviewed ESMF Reference Manual
• Searched GitHub Discussions

Affiliation(s)

AirMettle, Inc

ESMF Version

8.6.0

Issue

Hello,
In my code, all the allocated ESMC objects are reported as memory leaks despite calling the ESMC destroy functions and the Finalize function at the end.
I noticed that the call to ESMC_GridDestroy calls the ESMCI::Grid::destroy(&gridp) which has signature static int destroy(Grid **grid, bool noGarbage=false); since no second argument is passed, the noGarbage flag is always false by default and inside the destroy function the delete is always skipped

  // optionally delete the complete object and remove from garbage collection
if (noGarbage){
  VM::rmObject(*gridArg); // remove object from garbage collection
  delete (*gridArg);      // completely delete the object, free heap
}

I've attached a reproducible example here (copied from the example @danrosen25 sent me yesterday) test_memleak.tar.gz

valgrind --leak-check=full ./exeMemLeak reveals the memory leaks. Let me know if I am missing something or if this is not-reproducible. If there isn't anything wrong with my setup, the fix might be to provide a way to set noGarbage=true while calling ESMCI::Grid::destroy() but I am not sure.

Autotag

@oehmke

[danrosen25]

@theurich
Hi Gerhard, is there an issue in the ESMC garbage collection for the c library? ESMC_Finalize is leaving traces of ESMC objects.

[theurich]

Both sides, Fortran via ESMF_Finalize(), and C via ESMC_Finalize() end up in the same place for garbage collection handling, in ESMCI::VM::finalize(). There is currently code commented out inside ESMCI::VM::finalize() that deals with deleting the garbage collection entries. As I recall, on some systems strange issues were popping up under some large applications, which might have had to do with cross-referencing objects (like Grids used in multiple fields, etc.) and the order of the take down. The quick "fix" was to comment out those delete lines. Since this is in the Finalize() method, and the application is about to shut down anyway, memory leaks are of no practical concern at that time. But certainly tools like valgrind would report them.

As for the ESMC_GridDestroy() not providing noGarbage, it is true that this does not allow the user to remove the garbage collection entry for the destroyed Grid explicitly. However, unless you had a large loop over GridCreate()/GridDestroy(), which seems artificial, you'd not want to use noGarbage anyway. The default of destroying without noGarbage set is what you typically want for your application. The entry in the garbage collection protects the user code from hard to diagnose SEGV issues, and instead is able to log more meaningful error messages in case an object was used after Destroy() was called on it. This is one of the main purposes of the ESMF garbage collection. It is therefore not recommended to use noGarbage (even under the Fortran API where it is available in most Destory() calls), unless you are doing some tight looping over Create()/Destory().

[AbhiHampiholi]

I see. Thanks for clarifying this @theurich @danrosen25

[danrosen25]

@theurich Would a better solution to be to add a flag to the ESMCI::VM::finalize routine which does run garbage collection? Or will this fail in all cases anyway?

@AbhiHampiholi The short answer is that you're encountering expected behavior. ESMC objects submitted to the garbage collection, which is your only option in ESMC, will be reported in valgrind as a memory leak.

[AbhiHampiholi]

Yeah since it is expected, I am okay with it for now. The problem for me down the road would be the application will fail the "does this have a memory leak" test and throw errors during test time. So I may have to try uncommenting those lines and clean up all ESMC objects if that is expected to work on all but a few cases.

[theurich]

I don't think we want to clutter any of the APIs with a flag that is expected to go away in the future (i.e. when garbage collection works for all cases and does not leave memory leaks). Until then though I agree it would be nice to have an easy way (at run-time) to activate/deactivate the commented out pieces of code in question inside ESMCI::VM::finalize().
Consistent with the rest of the system would be the introduction of an environment variable that fits the ESMF_RUNTIME_* pattern, such as ESMF_RUNTIME_GARBAGE_EMPTY (or better name). When set (to anything but OFF) we activate the currently commented out code.