[LibOS,PAL/Linux-SGX] EDMM: Introduce Lazy Free optimization

[vijaydhanraj]

Description of the feature

Lazy free optimization introduces a manifest syntax that specifies the percentage of the total heap that can be freed in a lazy manner. Until this threshold is met, Gramine doesn't release any dynamically allocated memory and any new allocation request is first served from this pool.

How it works is that the percentage is converted to a threshold (in bytes) and whenever a memory range is freed by the application, the freed size is accumulated. When the accumulated free size pool grows above the threshold, the memory ranges are removed from the enclave.

This optimization helps reduce the expensive enclave entries/exits associated with the dynamic freeing of EPC pages.

(Pls see #683 for more discussion on this)

Why Gramine should implement it?

May help to improve the runtime performance of the application.

[mkow]

Lazy free optimization introduces a manifest syntax that specifies the percentage of the total heap that can be freed in a lazy manner.

I don't understand this definition, it's kind of recursive? If you "allow to free 20% of the total heap space", then your "total heap space" is that new, smaller size and then you can free again and again, effectively freeing more than 20%? Could you explain it clearer how exactly is this supposed to work?

Also, do we really need a manifest switch for this? If we can pick a good value then it will most likely work for 99% of the apps. I.e. I'm not sure whether it's worth to add such interface to the manifest.

[dimakuv]

Also, do we really need a manifest switch for this? If we can pick a good value then it will most likely work for 99% of the apps. I.e. I'm not sure whether it's worth to add such interface to the manifest.

I'm neutral on this, though in my experimence, having an explicit way to modify some value in programs/frameworks is always good. And a good default value hides this from 99% of the users, but the rest 1% (who really needs to fine-tune for some specific workload) will be happy to have such an option.

[vijaydhanraj]

If you "allow to free 20% of the total heap space", then your "total heap space" is that new, smaller size

This means that 80% of the total heap space is not freed and only 20% is allowed to be freed, right? But this is not what I meant. The idea is to free a certain threshold in a lazy manner.

For example, if we set the threshold to 20%, it means we don't free the actual heap (from user perspective is freed, but Gramine does this trick) and keep it in a pool until it hits 20% of the total heap size. Once it hits this threshold, we do the actual free until we hit half of this limit (say 10%) or we just fall below the threshold. (This is something we need to discuss and finalize)

Any new allocation request will be served first from this lazy pool.

Also, do we really need a manifest switch for this?

I think for the initial phase we can keep this as it is easier to tune and find the optimum threshold. If it doesn't add much value, we can remove it.

[mkow]

Ok, I understand now how you want this threshold to work, thanks for the explanation.

[dimakuv]

In recent tests, this optimization was shown to be not effective enough (we implemented other EDMM optimizations which yield higher benefits). Reassigned the priority to P3 (very low).