[memporf] Can `-fmemory-profile-use` be used together with `-fmemory-profile`?

[zcfh]

Background

I am currently trying to use memprof to optimize my program, but the test did not find obvious performance gains. I now hope to develop a tool to observe whether the optimization is really effective.

My idea is to use memory-profile and memory-profile-use at the same time, so that the memory access after memprof optimization can be collected. Then do some post-processing on this information to determine whether the correct clone is made. Furthermore, can a heat map be drawn based on the memory access situation, similar to BOLT.

Question

1. I made the first attempt and found that -fmemory-profile and -fmemory-profile-use cannot be used at the same time. Is this not allowed by design?
2. Are there other observation tools? I looked through some options and only found memprof-export-to-dot, but this can only prove whether the clone is done, and cannot reflect whether the clone result is good enough.

[teresajohnson]

What memory allocator do you normally use? afaik only tcmalloc has support for the hints that memprof currently applies. There are also a couple of internal options that need to be flipped to get those hints - what options are you using in your builds?

You can't currently use -fmemory-profile to measure anything interesting about the results as with instrumentation a special allocator, which doesn't know anything about the hints.

[zcfh]

@teresajohnson Thanks for your reply. I will add some information.

What memory allocator do you normally use? afaik only tcmalloc has support for the hints that memprof currently applies.

My service uses jemalloc, because the performance of jemalloc is better than tcmalloc in internal tests, but I don't know where the advantage comes from. But when I verified memprof, I used tcmalloc as the control group.

what options are you using in your builds?

Regarding the compilation options, I referred to this discussion 59165 .

## Instrumentation CPPFLAGS 
-gmlt -funique-internal-linkage-names -fdebug-info-for-profiling -mno-omit-leaf-frame-pointer  -fno-omit-frame-pointer -fno-optimize-sibling-calls -fmemory-profile -O3
## Instrumentation LDFLAGS
-fuse-ld=lld -Wl,--no-rosegment  -Wl,-build-id -no-pie 

## memprof optimization CPPFLAGS
-fdebug-info-for-profiling -gmlt -funique-internal-linkage-names -fmemory-profile-use=memprof.prof -flto=thin -O3
## memprof optimization LDFLAGS
-Wl,-mllvm,-enable-memprof-context-disambiguation -Wl,-mllvm,-supports-hot-cold-new -Wl,-mllvm,-optimize-hot-cold-new  -fuse-ld=lld -no-pie -Wl,--no-rosegment

In addition, when linking the binary, it can be found that the binary does use some symbols with hot_cold.
It's just that during the performance test, no performance gain was obtained. This may be related to the service characteristics, and its memory access may not be the bottleneck. So I hope there will be an observation tool that can draw a heat map of memory access, similar to BOLT heatmap.

measure anything interesting about the results as with instrumentation a special allocator, which doesn't know anything about the hints.

My idea is not to observe whether the instruction uses special allocator through -fmemory-profile. Instead, I hope to use instrumentation on the binary optimized by memprof, so that I can get the memory access information after memprof optimization and compare it with that before optimization.
When I have the memory access situation, I can refer to profdata and do some symbolic processing. At this time, I can get whether the instruction uses special allocator. It is equivalent to performing the workflow of memprof profile generation again, and there is a way to get the memory access information of the optimized binary.
But this is just my idea, which may be a bit whimsical.

[teresajohnson]

Sorry for the slow reply!

Those options look correct to me. We are working on improving the coverage of cold bytes through improvements in matching and cloning, and this is still a WIP. A few changes have very recently been enabled upstream and there are more in the works. In the meantime, we have also experimented with non-context sensitive hinting for allocations that are mostly cold. This can be done by giving "-mllvm -memprof-matching-cold-threshold=N" where N is the percent of cold bytes required to mark the allocation cold (in the initial compile / memprof matching step, not at LTO time).

The benefit we expect is from having tcmalloc co-locate cold allocations and use smaller page sizes for them. I'm told the default configuration of tcmalloc should do this (using hugepages for non-cold data), so in theory out of the box use of tcmalloc should do this.

In theory, I suppose you could add support to the memprof profiling allocation intercepters to accept the hot cold hints and see whether the cold data is allocated through cold hinted allocs and vice versa, possibly through some modifications to the profiling runtime.

Another option is to use some flags we have added to dump out matching and hinting stats. The former is the -memprof-print-match-info flag given to the matching compile step. The latter requires -memprof-report-hinted-sizes to be given during both the initial matching compile and the LTO link.

[zcfh]

In theory, I suppose you could add support to the memprof profiling allocation intercepters to accept the hot cold hints and see whether the cold data is allocated through cold hinted allocs and vice versa, possibly through some modifications to the profiling runtime.

Thank you for the reminder. It does seem unnecessary to perform binary instrumentation on the memprof-optimized binaries. Intercepting and processing during the first round of optimization should suffice to obtain the information I need.

Additionally, I have another question: Are there any suitable benchmarks for memprof? Testing with my online service is too cumbersome. Using simple demos doesn't allow for performance comparisons. And even with Clang, I haven't observed any performance improvements (is Clang not memory-bound or my test went wrong ?).