Skip to content

Update in-flight request counter, switch to FastAPI + Uvicorn #838

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 64 commits into from
Mar 4, 2020
Merged

Update in-flight request counter, switch to FastAPI + Uvicorn #838

Changes from 1 commit
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
aa6cee5
FastAPI demo
deliahu Feb 24, 2020
72abaed
Switch gunicorn to uvicorn
vishalbollu Feb 25, 2020
c0aaaf9
flask to fast-api rough
vishalbollu Feb 26, 2020
b79b7d1
Merge branch 'master' of github.com:cortexlabs/cortex into fastapi
deliahu Feb 27, 2020
d3df3af
Cleanup gunicorn code
vishalbollu Feb 27, 2020
0f2aacd
Add max_queue_length
deliahu Feb 27, 2020
7de8652
Update run.sh
deliahu Feb 27, 2020
0c56233
Set somaxconn
deliahu Feb 27, 2020
1c005dd
Use regular function instead of async for predict endpoint
vishalbollu Feb 27, 2020
da08f18
Update log formatter
deliahu Feb 28, 2020
4363113
Remove api is live log line
deliahu Feb 28, 2020
bbf4304
Update tf.requirements.txt
deliahu Feb 28, 2020
37dd3ec
Add pyyaml
deliahu Feb 28, 2020
263061f
Use configuration in APIs
deliahu Feb 28, 2020
75359d8
Add healthchecks and liveness probes
vishalbollu Feb 28, 2020
789498e
Add comment for uvicorn config
deliahu Feb 28, 2020
1a8fc9c
Add license
deliahu Feb 28, 2020
7efa210
Update avg request header
deliahu Feb 28, 2020
23fb6d6
Use background tasks and dedicated threadpool for requests
vishalbollu Feb 28, 2020
241ec7a
Merge branch 'fastapi' of github.com:cortexlabs/cortex into fastapi
vishalbollu Feb 28, 2020
14f09e5
Remove extra log
deliahu Feb 28, 2020
2fcd86d
Put latency metrics post request and use json-tricks if default json …
vishalbollu Feb 28, 2020
e8115d6
Merge branch 'fastapi' of github.com:cortexlabs/cortex into fastapi
vishalbollu Feb 28, 2020
d70e4bb
Add timestamp readiness probe
deliahu Feb 28, 2020
74f0799
Merge branch 'fastapi' of github.com:cortexlabs/cortex into fastapi
deliahu Feb 28, 2020
5130d9b
Write current time to api readiness file
deliahu Feb 28, 2020
e410f5f
Update liveness and readiness checks
deliahu Feb 29, 2020
e1c9453
Update OOM calculation
deliahu Feb 29, 2020
2294d08
Move request tracking file deletion to background
deliahu Feb 29, 2020
c1a0b51
Use request ID for request tracker files
deliahu Feb 29, 2020
e2268cd
Move request_start_time
deliahu Feb 29, 2020
f69e581
Fix lint
deliahu Feb 29, 2020
9705cfe
Remove unnecessary sed
vishalbollu Mar 1, 2020
5c6c47a
Revert compute spec in example
vishalbollu Mar 1, 2020
f70976e
Rename start_time to total_time in metrics calculation
vishalbollu Mar 1, 2020
7551540
Remove unecessary exception variable delcartions
vishalbollu Mar 1, 2020
1b432eb
Remove spacing in tf.requirements.txt
vishalbollu Mar 2, 2020
f7c1952
Reorder serve.py
vishalbollu Mar 2, 2020
df7d74b
Undo compute commit in image-classifier
vishalbollu Mar 2, 2020
743a3ea
Update liveness check
deliahu Mar 2, 2020
af807d5
Update serve.py
deliahu Mar 2, 2020
e948769
Misc cleanup
deliahu Mar 2, 2020
bd84de4
Merge branch 'master' of github.com:cortexlabs/cortex into fastapi
deliahu Mar 2, 2020
c238020
Update autoscaling.md
deliahu Mar 3, 2020
1362395
Update docs
deliahu Mar 3, 2020
9185612
Update autoscaling.md
deliahu Mar 3, 2020
3125261
Update autoscaling.md
deliahu Mar 3, 2020
301a4bf
Remove whitespace
deliahu Mar 3, 2020
764dd01
Update docs
deliahu Mar 3, 2020
5538e10
queue_length -> replica_concurrency
deliahu Mar 3, 2020
885433a
Check if MetricDataResults has atleast 1 element
vishalbollu Mar 3, 2020
2ca47a3
Move request tracking to critical path instead of background task
vishalbollu Mar 3, 2020
f1a0633
Update autoscaler logs
deliahu Mar 3, 2020
a9b5886
Update autoscaling.md
deliahu Mar 3, 2020
f417eef
Divide max_replica_concurrency by number of workers
vishalbollu Mar 4, 2020
19d2f38
Update autoscaling documentation
vishalbollu Mar 4, 2020
4f0241a
Update autoscaling.md
deliahu Mar 4, 2020
965b528
Set backlog to so_max_conn
vishalbollu Mar 4, 2020
058b252
Use rounding instead of integer divide to calculate CORTEX_MAX_WORKER…
vishalbollu Mar 4, 2020
4819973
Merge branch 'fastapi' of github.com:cortexlabs/cortex into fastapi
vishalbollu Mar 4, 2020
486bdbc
Rename pytorch sentiment-analyzer
deliahu Mar 4, 2020
1daedd9
Accept any free form json
vishalbollu Mar 4, 2020
3c7d983
Update autoscaling.md
vishalbollu Mar 4, 2020
1688a5f
Merge branch 'master' into fastapi
deliahu Mar 4, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Undo compute commit in image-classifier
  • Loading branch information
@vishalbollu
vishalbollu committed Mar 2, 2020
commit df7d74b54a4440e58a1c6938a5186ab29af84103
8 changes: 4 additions & 4 deletions examples/pytorch/image-classifier/cortex.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
predictor:
type: python
path: predictor.py
# compute:
# cpu: 1
# gpu: 1
# mem: 4G
compute:
cpu: 1
gpu: 1
mem: 4G