Skip to content

actually allow concurrent predictions and refactor runner #1500

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 40 commits into from
Closed

actually allow concurrent predictions and refactor runner #1500

wants to merge 40 commits into from

Conversation

technillogue
Copy link
Contributor

@technillogue technillogue commented Jan 27, 2024 •
edited
Loading

  • add concurrency to config
  • this basically works!
  • more descriptive names for predict functions
  • maybe pass through prediction id and try to make cancelation do both?
  • don't cancel from signal handler if a loop is running. expose worker busy state to runner
  • move handle_event_stream to PredictionEventHandler
  • make setup and canceling work

technillogue and others added 30 commits January 22, 2024 13:36
@technillogue
run CI for this branch the same way as for main
0df9b82 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
Revert "Revert PR "async runner" (#1352)"
513e837 
This reverts commit 828eee9.
@technillogue
Revert "Revert PR "create event loop before predictor setup" (#1366)"
db88489 
This reverts commit b533c6b.
@technillogue
lints
3444169
@technillogue @nickstenning
minimal async worker (#1410)
73a6de9 
* async Worker._wait and its consequences
* AsyncPipe so that we can process idempotent endpoint and cancellation rather than _wait blocking the event loop
* test_prediction_cancel can be flaky on some machines
* separate _process_list to be less surprising than isasyncgen
* sleep wasn't needed
* suggestions from review
* suggestions from review
* even more suggestions from review

---------

Signed-off-by: technillogue <technillogue@gmail.com>
Co-authored-by: Nick Stenning <nick@whiteink.com>
@technillogue
format
27b288f
@technillogue
select utility for racing awaitables
45afedc 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
start mux
54b34e3 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
tag events with id, read pipe in a task, get events from mux
0fbadea 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
use async pipe for async child loop
3239230 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
_shutting_down vs _terminating
58afbfb 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
select shutdown event
a70b027 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
keep reading events during shutdown, but call terminate after the las…
74a3dac 
…t Done

Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
imagine a world with context managers for our highly stateful resources
0ce8bd3 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
emit heartbeats from mux.read
597816e 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
don't use _wait. instead, setup reads event from the mux too
83b325b 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
worker semaphore and prediction ctx
c1256ea 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
where _wait used to raise a fatal error, have _read_events set an err…
18640dc 
…or on Mux, and then Mux.read can raise the error in the right context. otherwise, the exception is stuck in a task and doesn't propagate correctly

Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
fix event loop errors for <3.9
4469766 
before 3.10, async primitives stored the current loop when created. since then, they use get_running_loop()

Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
keep track of predictions in flight explicitly and use that to route …
7399614 
…logs

Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
don't wait for executor shutdown
de29ee7 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
progress: check for cancelation in task done_handler
d4e5c6f 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
let mux check if child is alive and set mux shutdown after leaving re…
13fec6c 
…ad event loop

Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
close pipe when exiting
2f687b7 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
note a different way to do eager state checks with a decorator
2f193bf 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
predict requires IDLE or PROCESSING
4015925 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
idk, try adding a BUSY state distinct from PROCESSING when we no long…
e47868c 
…er have capacity

Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
move resetting events to setup() instead of _read_events()
b441ea0 
previously this was in _read_events because it's a coroutine that will have the correct event loop. however, _read_events actually gets created in a task, which can run *after* the first mux.read call by setup. since setup is now the first async entrypoint in worker and in tests, we can safely move it there

Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
add concurrency to config
e939c0b 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
this basically works!
eaceff7 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
more descriptive names for predict functions
2c28885 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
maybe pass through prediction id and try to make cancelation do both?
d44e467 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
don't cancel from signal handler if a loop is running. expose worker …
74da750 
…busy state to runner

Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
move handle_event_stream to PredictionEventHandler
b416c97 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
make setup and canceling work
07e86b8 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
drop some checks around cancelation
39ea156 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
try out eager_predict_state_change
b145c5d 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
try to make idempotent endpoint return the same result and fix tests …
0b1ad74 
…somewhat

Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
fix idempotent tests
8480e51 
Signed-off-by: technillogue <technillogue@gmail.com>
@technillogue
fix remaining errors?
9fb111e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@technillogue