Track the total usage of the code cache

[a7ehuo]

To support -XX:+PrintCodeCache(eclipse-openj9/openj9#8184), the high water
mark of the code cache usage needs to be tracked.
The usage is tracked when increasing/decreasing the code cache repository
take place and also the reuse of the free blocks in the code cache.

The code cache usage reported in the javacore dump file will
reflect this change because increase/decreaseFreeSpaceInCodeCacheRepository
will also be called when the free blocks are reused/freed.

Added TR_PrintCodeCacheUsage to TR_CompilationOptions. It's set
when -XX:+PrintCodeCache is passed to the JVM or when it's set
as an environment variable.

Also added information on the config size, the total free size,
and the used size for each code cache in printOccupancyStats()
which is used by OpenJ9 to show the code cache usage
when TR_PrintCompMem is set.

Signed-off-by: Annabelle Huo Annabelle.Huo@ibm.com

[mpirvu]

This solution tracks code cache usage in 4 places:

1. addFreeBlock2WithCallSite
2. findFreeBlock
3. increaseFreeSpaceInCodeCacheRepository
4. decreaseFreeSpaceinCodeCacheRepository

The last two are used only if a code cache repository is used so it's not general enough (or 32 bit or for Power we don't use a repository)

[mpirvu]

I am proposing one of two solutions:

1. Everywhere we call increase/decreaseFreeSpaceInCodeCacheRepository add a call to increase/decreaseCurrTotalUsedInBytes
2. Replace increase/decreaseFreeSpaceInCodeCacheRepository with increase/decreaseCurrTotalUsedInBytes and insert increase/decreaseFreeSpaceInCodeCacheRepository into increase/decreaseCurrTotalUsedInBytes

[a7ehuo]

I'd prefer solution 2 to reduce the number of the calls that has to be inserted at the callers.

2. Replace increase/decreaseFreeSpaceInCodeCacheRepository with increase/decreaseCurrTotalUsedInBytes and insert increase/decreaseFreeSpaceInCodeCacheRepository into increase/decreaseCurrTotalUsedInBytes

[mpirvu]

This new solution will change the values we used to report in the javacore because we now track changes in the freedBlocks, but I guess it's more precise so it's better.

I have another suggestion: introduce the JIT option that will control the printing of the code cache information. Then, in openj9, you can turn on this option after parsing the corresponding VM option.

enum TR_CompilationOptions
   {
   ...
    TR_PrintCodeCacheUsage                           = 0x00000040 + 6,
   };

   {"enablePrintCodeCacheUsage", "O\tPrint code cache usage at shutdown", SET_OPTION_BIT(TR_PrintCodeCacheUsage), "F", NOT_IN_SUBSET},

and in openj9 you may want to do
self()->setOption(TR_PrintCodeCacheUsage) at the right point

[a7ehuo]

@Leonardo2718 Ready for review. Thanks!
@mpirvu FYI.

[mpirvu]

LGTM

[Leonardo2718]

LGTM 👍

[Leonardo2718]

@genie-omr build all

[a7ehuo]

aix_ppc-64 build 1867 failed with the following error. It doesn't seem to be related to this change. The same failure seems to have started in PullRequest-aix_ppc-64 build 1860 (https://ci.eclipse.org/omr/job/PullRequest-aix_ppc-64/1860/console).

08:31:09  [ 21%] Building CXX object fvtest/compilertest/CMakeFiles/testcompiler.dir/__/__/compiler/codegen/ScratchRegisterManager.cpp.o
08:31:09  [ 21%] Linking CXX shared library libomrsig.a
08:31:09  ld: 0711-415 WARNING: Symbol omrsig_primary_signal is already exported.
08:31:09  ld: 0711-415 WARNING: Symbol omrsig_handler is already exported.
08:31:09  ld: 0711-415 WARNING: Symbol signal is already exported.
08:31:09  ld: 0711-415 WARNING: Symbol omrsig_primary_sigaction is already exported.
08:31:09  ld: 0711-415 WARNING: Symbol sigaction is already exported.
08:31:09  ld: 0711-415 WARNING: Symbol sigset is already exported.
08:31:09  ld: 0711-415 WARNING: Symbol sigignore is already exported.
08:31:09  ld: 0711-415 WARNING: Symbol bsd_signal is already exported.
08:31:09  ld: 0711-415 WARNING: Symbol sysv_signal is already exported.
08:31:09  ld: 0711-317 ERROR: Undefined symbol: .pthread_sigmask
08:31:09  ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.
08:31:09  make[2]: *** [omrsigcompat/CMakeFiles/omrsig.dir/build.make:90: omrsigcompat/libomrsig.a] Error 8
08:31:09  make[1]: *** [CMakeFiles/Makefile2:1656: omrsigcompat/CMakeFiles/omrsig.dir/all] Error 2
08:31:09  make[1]: *** Waiting for unfinished jobs....
08:31:09  Scanning dependencies of target omrutiltest
08:31:09  [ 21%] Building CXX object fvtest/compilertest/CMakeFiles/testcompiler.dir/__/__/compiler/codegen/StorageInfo.cpp.o
08:31:09  [ 21%] Building CXX object fvtest/utiltest/CMakeFiles/omrutiltest.dir/main.cpp.o
08:31:10  "/home/u0020236/workspace/PullRequest-aix_ppc-64/compiler/infra/OMRMonitor.hpp", line 50.46: 1540-1101 (W) A return value of type "int" is expected.

[Leonardo2718]

aix_ppc-64 build 1867 failed with the following error. It doesn't seem to be related to this change [...]

That's correct. There's a discussion about that failure on https://eclipse-omr.slack.com/archives/CDEA8CP1Q.

If we can't find a solution for the problem soon, I'll just merge this PR as is.

[Leonardo2718]

@genie-omr build aix

[Leonardo2718]

All builds have now passed. Merging.