Limit remote batch size

[mjp41]

When processing a remote batch, the system will process every single message that was available at the start of processing. This can lead to a long pause time if there have been a considerable number of frees to this thread.

This commit introduces a new mechanism to only process messages up to a limit of 1MiB. The limit is configurable using CMake.

Choosing too small a limit can cause freeing to never catch up with the incoming messages.

[nwf]

This seems entirely sensible.

Might it be worth having different thresholds for the different times that we are servicing the message queue?

• on a relative fast path (say, a small fast free list is empty and we're grabbing the next slab) or
• already being on a slow path (say, when we're going to the backend for a new chunk) or
• on a really slow path (say, when going even further back to the shared pool)

I could sort of imagine that making the latter of those consume the entire already pending queue but the former be a little more "hope we're in a steady state" flavor of choosy, but that's just an intuition that isn't backed by any kind of data. :)

[mjp41]

@nwf I have been wondering about something similar. If we had a remote per size class per allocator, then we could process the message queues to get a new free list, and only process other sizes if we needed to get a new slab. I think there are some great opportunities for further optimisations here.

[nwf]

Oh hey, that's really clever!

Just musing and catching up to things you already know...

• It's a middle-ground between "every allocator has one remote" of snmalloc today and "every slab is its own remote" of mimalloc.
• The existing send code doesn't need to change to support this, if slab construction puts the appropriate RemoteAllocator* in the Pagemap. (All objects being returned, and the slab being recycled, is a good, natural barrier, barring double-free, on other Allocators not having a copy of the RemoteAllocator* from the Pagemap. It'd be cute to be able to change it during a slab's lifecycle, but probably is more work than it'd be worth.)
• After BatchIt, it seems like there's a reasonable chance that the first message we process is big enough to be a fast free list, skipping some of the slab lifecycle machinery (at least in the no-mitigations case).

[mjp41]

The pagemap combines the remote pointer with a size class. I think we could do this, but we'd need the remotes to be at a well defined alignment to be able to read the size class cheaply for various mitigations.

[mjp41]

@nwf I propose we merge this as is, and raise an issue to continue this discussion?

[nwf]

Yeah, that sounds like the right plan to me. :)