Multiprocess heartbeat and cancel improvement research

[cretz]

Is your feature request related to a problem? Please describe.

Right now we're using multiprocessing.Manager()'s queue for heartbeating and event for cancellation. Then we move from the sync queue onto the sync heartbeat which schedules an async heartbeat task that we try to ensure runs when the activity completes.

Describe the solution you'd like

This is potentially flaky and/or confusing. Try to streamline the threaded/multiprocess heartbeat queuing.

[cretz]

Also, heartbeating too fast can hold up the event loop which we don't like