[Core] Add multiproc_worker_utils for multiprocessing-based workers

[njhill]

Mechanics for running multiprocessing-based worker processes and dispatching tasks to them via multiprocessing queues.

The class ProcessWorkerWrapper is equivalent to RayWorkerWrapper.

Each ProcessWorkerWrapper has a task queue for passing tasks to the corresponding worker process.

There is a shared result queue for receiving the results from all worker processes. ResultHandler is a thread that processes this result queue, completing the corresponding waiting futures as results come in.

WorkerMonitor is a thread that uses the multiprocessing wait() method to monitor the state of all worker processes and tear things down cleanly if any exit unexpectedly - including completing all waiting futures with an error and terminating the remaining processes.

_run_worker_process is the function that runs in each of the worker processes, in a loop waiting on their corresponding task queue.

This is in preparation for a MultiprocessingGPUExecutor.

[njhill]

@zhuohan123 this is the file we already talked through, with some minor updates:

• Renamed VllmLocalWorker to ProcessWorkerWrapper (in-line with the recent renaming of VllmRayWorker to RayWorkerWrapper)
• Changed this class to not inherit from multiprocessing.Process, instead just have the Process as a field (I agree this makes it clearer)
• Fix some mypy typing issues

[zhuohan123]

LGTM! Left a small comment.

[cermeng]

@njhill I know the PR is already merged, but I left some comments which could be related to issue #6219. I would appreciate it if you could address these comments, and I can work further on this issue