Skip to content

Improve API inter-process queue fairness #1526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 33 commits into from
Nov 6, 2020
Merged

Improve API inter-process queue fairness #1526

merged 33 commits into from
Nov 6, 2020

Conversation

RobertLucian
Copy link
Member

@RobertLucian RobertLucian commented Nov 3, 2020 •
edited by deliahu
Loading

Closes #839, closes #1298, closes #814

checklist:

  • run make test and make lint
  • test manually (i.e. build/push all images, restart operator, and re-deploy APIs)

@RobertLucian
Add nginx boiler plate config
d79abe3
@RobertLucian
Merge branch 'master' into fix/inter-process-fairness
4283b5e
@RobertLucian
Work on the supervisor process
68c63c2
@RobertLucian
Work on nginx
ecdbb60
@RobertLucian
Add stop scripts for all serving processes
80686f6
@RobertLucian
Deny all but to the predict route or empty route
79f253c
@RobertLucian
Improve performance by adding thread pool + set higher read timeout
cbe33cb
@RobertLucian
Fix API readiness
c0c3e06
@RobertLucian
Make lint
9da8cc3
@RobertLucian
Add CORS to nginx & rm from uvicorn
b822c37
@RobertLucian
Terminate nginx gracefully
2107437
@RobertLucian
Increase grace period to 60s for k8s pods
f86f775
@RobertLucian
Make lint
158fc3d
@RobertLucian
Properly shut down nginx when deleting a pod
c8d3dad
@RobertLucian
Use jinja2 to generate the nginx config
3ca88b2
@RobertLucian
Fix jinja generating issues
c699e2c
@RobertLucian
Adapt all predictor images to use s6-overlay
ee0ebe1
@RobertLucian
Fix linting
8d08afc
@RobertLucian
Fixes for the predictor images
952f829
@RobertLucian
Improve comments semantics
661806c
@RobertLucian
Stop sidecars when the serving API container has stopped
a057188
@RobertLucian
Pass in just the required value
5592b53
@RobertLucian
CORTEX_SERVING_PORT not required for waitAPIContainer func
b95a2b9
@RobertLucian
Fix bug when batching is done
9b1fcad
@RobertLucian RobertLucian requested a review from deliahu November 3, 2020 04:26
deliahu
deliahu reviewed Nov 4, 2020
View reviewed changes
@RobertLucian RobertLucian merged commit c8da085 into master Nov 6, 2020
@RobertLucian RobertLucian deleted the fix/inter-process-fairness branch November 6, 2020 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@deliahu deliahu deliahu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@RobertLucian @deliahu