Heap tracking hook

[RobMeades]

Enhancement

When working in a constrained environment (e.g. NXP1768) with increasingly large applications (e.g. mbedClient), being able to track what's happening to the available RAM is really important. The current run-time stats are good but not quite good enough: one can add calls to mbed_stats_heap_get() throughout the visible code, but that does not account for the stuff that happens in libraries and, also, it does not allow you to see what that last fatal failed malloc() that triggered the "Operator new out of memory" was for: was it a few 10's of bytes, easy to free up, or was it for a 4k block, in which case give up and go home?

What would be useful for these situations, as @0xc0170 suggested, is a hook into malloc() and free() allowing each attempt to deploy heap to be logged by the user. Then you can really see what's happening and decide whether it's time to go home or not.

[RobMeades]

The other issue with the existing heap tracking mechanism is that (as documented) it doesn't take into account allocator overhead or heap fragmentation. For instance, I had a trace like this:

Heap: 185 malloc(s) (0 failed), total 19828, alloced 13699 (high watermark 13699), left 6129 (at .\main.cpp:263).
Heap: 258 malloc(s) (0 failed), total 19828, alloced 14845 (high watermark 14860), left 4983 (at .\main.cpp:265).

So, since I apparently have spare heap, I took 1.5 kbytes back for a static buffer. The next time I ran the same code the trace was this:

Heap: 185 malloc(s) (0 failed), total 18804, alloced 13955 (high watermark 13955), left 4849 (at .\main.cpp:263).
Operator new out of memory

So I conclude that mbed_stats_heap_get() and the amount of actual malloc()able space are quite some distance apart.

[ciarmcom]

ARM Internal Ref: MBOTRIAGE-319

[SenRamakri]

@RobMeades - Mbed-os does have the memory tracing capability to track every malloc/free from the application/libraries.
See https://os.mbed.com/docs/v5.9/tutorials/optimizing.html#runtime-memory-tracing
Is this what you are looking for?

[SenRamakri]

Hi @RobMeades - Thanks for your question and as I mentioned above if you are looking for mechanism to track each malloc/realloc/free etc we already have a mechanism( https://os.mbed.com/docs/v5.9/tutorials/optimizing.html#runtime-memory-tracing ) which tracks them and prints each alloc/free to terminal. We also have a tool which can analyze this output and generate a heap report at that point. See below link for how to use that tool.
https://github.com/ARMmbed/mbed-os-mem-report

At this point, we are closing this issue as no further action is required from our side, but if you think this answer does not address your concern feel free to re-open this issue.