Add Xgc options for suballocator heap size and quick allocation

[ThanHenderson]

1. Use VMEM_ALLOC_QUICK by default for allocateRegion in
   allocate_memory32
2. Adds -Xgc:suballocatorQuickAllocDisable option that
   disables the default VMEM_ALLOC_QUICK
3. Adds -Xgc:suballocatorIncrementSize option that replaces the
   HEAP_SIZE_BYTES macro and controls the heap increment size
4. Adds an omrport_copy_suballocator_globals procedure that correctly
   initializes the PPG suballoctor globals for memCheckPortLib when
   -Xcheck is provided
5. Updates related documentation

Addresses: #7190
Signed-off-by: Nathan Henderson nathan.henderson@ibm.com

[ThanHenderson]

FYI @babsingh

This relates to #7190 (comment)

However, there are a few issues that arose during testing.

The patch adds fields to the PPG_mem_mem32_subAllocHeapMem32 structure for the suballocatorIncrementSize (heap size) and suballocatorQuickAlloc, both of which are set from the corresponding GC extensions fields that are either the default values or the values specified via added the command line options.

In omrmem32helpers.c, startup_memory32 zeroes out the PPG_mem_mem32_subAllocHeapMem32 fields and is called during port library initialization. Then, MM_MemoryManager::createVirtualMemoryForHeap calls omrport_control to set the PPG_mem_mem32_subAllocHeapMem32 suballocatorIncrementSize and suballocatorQuickAlloc to the default or user supplied value. I chose this approach because it is also what is done for the previously implemented suballocatorCommitSize option.

PPG_mem_mem32_subAllocHeapMem32.suballocatorIncrementSize replaces the previous macro HEAP_SIZE_BYTES and is used in the allocate_memory32 procedure. The problem is that it turns out that createVirtualMemoryForHeap isn't always called before allocate_memory32, this is observable in some sanity.functional tests (3 of them resulting in failures on PPCAIX), and thus in those cases, it isn't updated from 0 to the default value or user defined value.

I am trying to figure out where/when the best time to set these PPG values to ensure that it precedes all calls to allocate_memory32. An obvious answer would be to set them just in startup_memory32, but that would talk some work around integration of GCExtensionsBase.hpp C++ code into the omrmem32helpers.c C code. I am going to go that route initially, but I am open to suggestions here.

Update edit:
PPG_mem_mem32_subAllocHeapMem32.suballocatorIncrementSize is only not set properly if -Xcheck is passed because memCheck_initialize sets up its own port lib and doesn't call createVirtualMemoryForHeap. All the allocations done via the memcheck port lib do not have the PPG_mem_mem32 options set apart from in startup_memory32.

[babsingh]

where/when the best time to set these PPG values ...

Try setting PPG values before protectedInitializeJavaVM is invoked; see jvminit.c; there are calls to j9port_control for initializing port library globals (PPG).

[ThanHenderson]

Try setting PPG values before protectedInitializeJavaVM is invoked

The problem is that we need to update the PPG values on the memCheckPortLib. They are set correctly for the normal port library for the VM, just not propagated to memcheck.

I've got something working where the memCheckPortLib PPG values are set when entering the memoryCheck_allocate_memory32 function that is used exclusively by and only for memcheck and precedes each call to allocate_memory32. It calls a port function called omrport_copy_subAllocHeapGlobals that then calls omrport_control on the memCheckPortLib with the fields of interest.

The only potential issue that I see with this is that there is more control flow in every call to memoryCheck_allocate_memory32, however modern branch predictors should be able to handle this quite readily since the PPG values only change on the first call to memoryCheck_allocate_memory32, every other call would see the same control flow without any memory writes on the subsequent calls.

[babsingh]

Minor formatting nitpicks.

[babsingh]

@keithc-ca @dmitripivkine Can you also review this PR?

[dmitripivkine]

@amicic FYI

[keithc-ca]

Depends on: eclipse-openj9/openj9#19872

A dependency in that direction is not reasonable; and cyclic dependencies even less so.

[dmitripivkine]

Looks good to me

[ThanHenderson]

@0xdaryl could you start testing and get this merged?

[0xdaryl]

Jenkins build all

Running initial testing on commits that are here.

[ThanHenderson]

I had forgotten to add some #if defined(OMR_ENV_DATA64) around some definitions so some of the builds were failing. They will need to be relaunched.

[0xdaryl]

Jenkins build all

[0xdaryl]

Jenkins build all

[ThanHenderson]

Who can I contact about the CI machines? The windows machines have been offline for a while now.

[0xdaryl]

re/ Windows CI machines. They were offline pending critical security updates. I understand that's been completed, but additional work needs to be done to verify that and get them back online.

[ThanHenderson]

Adam told me that there is no ETA for the Windows machines to be back online. All of the other ci builds passed. I have one more commit to push addressing Keith's issues with the comments. They won't affect the behaviour at all so I don't think we need to rerun all the builds.

[0xdaryl]

@keithc-ca : please formally approve if you're OK with the final changes

[pshipton]

These changes are causing failures on AIX across all Java versions.
memleaksTest_0,cmdLineTester_GCRegressionTests_2,cmdLineTester_GCRegressionTests_3
WARNING: Out of memory in allocate_memory at /home/jenkins/workspace/Build_JDK8_ppc64_aix_OMR/openj9/runtime/vm/montable.c:153, returning NULL (3664 bytes were requested)

I created a build without this change and the failing tests pass.
https://hyc-runtimes-jenkins.swg-devops.com/view/Test_grinder/job/Grinder/42578

I'm creating a revert PR so we can get the pending OMR changes promoted.

[ThanHenderson]

These changes are causing failures on AIX across all Java versions.

This happens when -Xcheck is passed. And since the corresponding OpenJ9 patch wasn't used to build, the memCheckPortLib isn't initialized properly. The OMR acceptance needs to be built with the corresponding OpenJ9 patch.