multiprocessing.managers. SharedMemoryManager should let you name the created object

[slerman12]

Python's new truly-shared memory (https://docs.python.org/3/library/multiprocessing.shared_memory.html) is great, however using the SharedMemoryManager is impractical since the created list or numpy array can't be custom-named.

Why would we want to custom-name?

Well, say process 3 creates a new object and process 5 wants to access it. Without a shared naming protocol, this isn't possible. Process 3 has to read the name for its newly created object and somehow signal it to process 5 in order for process 5 to ever know about it and access it.

We should be able to name the objects created with the SharedMemoryManager, with the name= parameter, same as the syntax for creating those objects without the manager.

Thanks,
Sam

Linked PRs

• gh-101014: support named SharedMemory and ShareableList in SharedMemo… #102170
• gh-101014: support named SharedMemory and ShareableList in SharedMemo… #102171

[zpz]

I feel there are alternatives to this feature. After you create a piece of shared memory, you get a proxy to it. If you need any other process to use this piece, just pass this proxy to the other process.

But pickling in multiprocessing.managers has a bug, and the shared_memory part of it is not as friendly as it can be. mpservice.multiprocessing.server_process contains some fixes and enhancements.

I'm thinking of creating a new issue with some bug fixes. I'm researching existing issues in prep for that.

[amitojc]

I feel there are alternatives to this feature. After you create a piece of shared memory, you get a proxy to it. If you need any other process to use this piece, just pass this proxy to the other process.

This will run into a problem because apparently, the 'proxy' is an actual file descriptor. So calling "ShareableList" and storing its proxy too many times will run into the OSError: [Errno 24] Too many open files error.