Extend KFServing component with autoscaling and server mode #2315
Conversation
| @@ -1,6 +1,6 @@ | |||
| FROM python:3.6-slim | |||
|
|
|||
| RUN pip install kubernetes==9.0.0 kfserving==0.1.1 requests==2.22.0 | |||
| RUN pip install kubernetes==9.0.0 kfserving==0.1.1 requests==2.22.0 Flask==1.1.1 flask-cors==3.0.8 | |||
There was a problem hiding this comment.
Nit. It's a good habit to use pip3, since some images have pip pointing to python2.
| ) | ||
| print(model_status) | ||
| try: | ||
| print(model_status['status']['url'] + ' is the knative domain header. $CLUSTER_IP:31380 is the endpoint to post predictions to the model.') |
There was a problem hiding this comment.
why 31830 is hardcoded?
There was a problem hiding this comment.
The default istio ingress gateway is hosted with nodePort 31380. I updated this section with some bash command to get the LoadBalancer IP to make it more clarify.
| print(model_status['status']['url'] + ' is the knative domain header. $CLUSTER_IP:31380 is the endpoint to post predictions to the model.') | ||
| print('Sample test curl below: ') | ||
| print('curl -X GET -H "Host: ' + url.sub('', model_status['status']['url']) + '" $CLUSTER_IP:31380') | ||
| except: |
| @@ -28,7 +28,11 @@ def kfservingPipeline( | |||
| canary_model_uri='gs://kfserving-samples/models/tensorflow/flowers', | |||
There was a problem hiding this comment.
I would suggest removing this file. We will come up with a sample later in samples repo imho
|
/lgtm |
| @@ -8,13 +8,15 @@ inputs: | |||
| - {name: Canary Model Traffic Percentage, type: String, default: '0', description: 'Optional Traffic to be sent to default model'} | |||
| - {name: Namespace, type: String, default: 'kubeflow', description: 'Kubernetes namespace where the KFServing service is deployed.'} | |||
| - {name: Framework, type: String, default: 'tensorflow', description: 'Machine Learning Framework for Model Serving.'} | |||
| - {name: default_custom_model_spec, type: String, default: '{}', description: 'Custom runtime default custom model container spec.'} | |||
| - {name: canary_custom_model_spec, type: String, default: '{}', description: 'Custom runtime canary custom model container spec.'} | |||
| - {name: Default Custom Model Spec, type: String, default: '{}', description: 'Custom runtime default custom model container spec.'} | |||
There was a problem hiding this comment.
JFYI: If you use type: JsonObject, then the argument to this input will be automatically JSON-serialized.
e.g. kfserve_op(..., spec={'model_version': 1.7}), ....
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Ark-kun The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
These updates coincide with MM 0.9 release. Signed-off-by: Paul Van Eck <pvaneck@us.ibm.com>
Extend KFServing component with the following features:
/assign @animeshsingh
This change is