MemoryCache entry eviction is based on system memory usage, not cache memory usage

[karimsalem1]

Description

MemoryCache accepts a PhysicalMemoryLimitPercentage property which is documented as

The percentage of physical memory that the cache can use, expressed as an integer value from 1 to 100. If the cache size exceeds the specified limit, the memory cache implementation removes cache entries.

This implies that if the cache is consuming X% of the machine's physical memory, the cache will start evicting entries to reduce memory pressure.

However, the actual behavior is different: the implementation monitors total system-wide physical memory usage — not the memory used by the cache. As a result, cache entries may be evicted even if the cache is consuming a trivial amount of memory, simply because overall system memory is under pressure.

We discovered this issue in production as the MemoryCache was wrongly evicting items despite the insignificant cache size and the physicalMemoryLimitPercentage set to 40%.

Reproduction Steps

1. Instantiate a MemoryCache instance with physicalMemoryLimitPercentage set to a value lower than your machine's current system memory usage (can be found in TaskManager > Performance > Memory).
2. Periodically add items to a small cache without expiration policies, registering a RemovedCallback to log entry removals to the console along with their RemovedReason.
3. Observe that items are being removed from the cache with RemovedReason set to Evicted, despite the cache itself not consuming significant memory.

Repro file added: MemoryCacheTest.txt

Expected behavior

The entries should not be evicted from the cache, because the cache size is very small. It does not add up to the physical memory limit percentage defined.

Actual behavior

Cache entries are being regularly evicted because there is overall memory pressure on the machine.

Regression?

Unlikely to be a regression.

Known Workarounds

Set PhysicalMemoryLimitPercentage to 0 or a sufficiently high value to minimize unwanted cache evictions triggered by system-wide memory pressure.

Configuration

• Reproable in .NET 8 and .NET Framework 4.8.
• Microsoft Windows 11 Enterprise - 10.0.22631 Build 22631
• x64
• Should not be specific to this configuration.

Other information

The relevant code can be found in PhysicalMemoryMonitor.cs. The Windows implementation uses the GlobalMemoryStatusEx to retrieve the system-wide physical memory usage, rather than the cache's actual memory footprint.

This seems to be intentional, as noted in the class comment:

PhysicalMemoryMonitor monitors the amount of physical memory used on the machine and helps us determine when to drop entries to avoid paging and GC thrashing.

Therefore, this is likely not a code bug, but rather a documentation issue. The current behavior may lead to confusion for service owners who interpret the logic as tracking cache memory usage, potentially resulting in unexpected cache eviction behavior.

Proposed resolutions:

• Clarify the documentation to reflect the current design, making it explicit that eviction decisions are based on global memory pressure, not cache-specific usage.
• Update the implementation to reflect cache memory usage, if that was the intended behavior, although that would be a breaking change.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-extensions-caching
See info in area-owners.md if you want to be subscribed.

[KalleOlaviNiemitalo]

MemoryCache on .NET Framework uses System.SizedReference to measure the actual size of the cached objects. The .NET Core runtime however does not support SizedReference. See #24200 and #74676.

[karimsalem1]

Thanks @KalleOlaviNiemitalo. I'm not referring to how cache evictions don't happen because .NET Core can't measure the cache size. This issue is about how the MemoryCache is evicting entries based on the system-wide (not cache-specific) physical memory usage, which contradicts the documentation. This issue is repro-able in Framework as well.

[StephenMolloy]

I am heads-down in an internal team sprint for the next couple of weeks, so I can't really look closely at this right now. But I believe the documentation is incorrect and the actual behavior is correct. The way the documentation is written seems to be describing CacheMemoryLimitMegabytes, which as noted in this thread does not work on .Net Core since the cache does not have a sized-ref like it has in NetFx to keep track of it's size.

To give a brief history here, SRC.MemoryCache was originally split off from System.Web.Caching.Cache in NetFx because we found that lots of people were trying to use the ASP.Net cache in non-ASP.Net apps - but doing so brought along a lot of overhead and baggage from System.Web that non-web applications don't need. So SRC.MemoryCache was created. But it is largely just a port of System.Web.Caching.Cache.

And from that perspective, the ASP.Net cache cared about the overall memory load on the machine because ASP.Net was a multi-tenant host whose internal cache is actually trying to keep all the apps running in w3wp.exe in line so 1) The worker process doesn't recycle, and 2) All the worker processes on a machine don't consume all the memory on the server rendering it incapable of serving requests efficiently. Point #2 is where this limit comes from.

But yeah, I think this is a doc issue.