Flask template for Machine Learning model deployment

A simple example of a Python web service for real time machine learning model deployment. It is based on this post

This includes Docker integration and SHAP explanations for the deployed model.

Website | Source | Issues

Installation

Requirements

• docker
• docker-compose (Recommended)

Before using

Make sure that you have a model in the main directory. You can launch the example using the following line in order to create a quick classification model.

$ python ./example/build_linear_binary.py

or one of the scripts in the ./example folder

Configuration

• variables.env: Controls API parameters via environment variables

• requirements.txt: Controls Python packages installed inside the container

• model.joblib: Model saved inside a dictionary with this format

  {
      "model": trained_model,
      "metadata": {"features": [
          {"name": "feature1", "type": "numeric", "accepts_missing": True},
          {"name": "feature2", "type": "numeric", "default": -1, "accepts_missing": False},
          {"name": "feature3", "type": "category", "categories": ["A", "B"], "accepts_missing": True}]}
  }

Run the service

On Docker

Build the image (this has to be done every time the code or the model changes)

$ docker-compose build

Create and run the container

$ docker-compose up

On local Python environment

Create the environment

$ conda create -n flask_ml python=3
$ conda activate flask_ml

Install requirements

$ pip install -r ./requirements-service.txt  
$ pip install -r ./requirements.txt  

Run the API service

$ python service.py  

Usage of the API

This example considers that the API was launched locally without docker and with the default parameters (localhost at port 5000) and its calling the example model.

For /predict endpoint the JSON string in the payload of hte request can take two forms:

1. The first, the payload is a record or a list of records with one value per feature. This will be directly interpreted as the input for the model.

2. The second, the payload is a dictionary with 1 or 2 elements. The key _data is mandatory because this will be the input for the model and its format is expected to be a record or a list of records. On the other hand, the key _samples (optional) will be used to obtain different explanations.

If _samples is not given, then the explanations returned are the raw output of the trees, which varies by model (for binary classification in XGBoost this is the log odds ratio). On the contrary, if _samples is given, then the explanations are the output of the model transformed into probability space (note that this means the SHAP values now sum to the probability output of the model). See the SHAP documentation for details.

Check the API's health status

Endpoint: /health

$ curl -X GET http://localhost:5000/health
up

Is model ready?

Endpoint: /ready

$ curl -X GET http://localhost:5000/ready
ready

Get information about service

Endpoint: /service-info

$ curl -X GET http://localhost:5000/service-info

{
  "debug": true,
  "running-since": 1563355369.6482198,
  "serving-model-name": "model.joblib",
  "serving-model-type": "SKLEARN_MODEL",
  "version-template": "2.2.0"
}

Get information about the model

Endpoint: /info

$ curl -X GET http://localhost:5000/info

{
  "metadata": {
    "features": [
      {
        "default": -1,
        "importance": 0.2,
        "name": "feature1",
        "type": "numeric"
      },
      {
        "default": -1,
        "importance": 0.1,
        "name": "feature2",
        "type": "numeric"
      },
      {
        "default": -1,
        "importance": 0.3,
        "name": "feature3",
        "type": "numeric"
      }
    ]
  },
  "model": {
    "type": "<class 'sklearn.ensemble.forest.RandomForestClassifier'>",
    "predictor_type": "<class 'sklearn.ensemble.forest.RandomForestClassifier'>",
    "is_explainable": false,
    "task": "BINARY_CLASSIFICATION",
    "class_names": ["0", "1"]
  }
}

Compute predictions

Endpoint: /predict

$ curl -d '[{"feature1": 1, "feature2": 1, "feature3": 2}, {"feature1": 1, "feature2": 1, "feature3": 2}]' -H "Content-Type: application/json" -X POST http://localhost:5000/predict

{
  "prediction": [0, 0]
}

Predict probabilities

Endpoint: /predict?proba=1

$ curl -d '{"feature1": 1, "feature2": 1, "feature3": 2}' -H "Content-Type: application/json" -X POST "http://localhost:5000/predict?proba=1"

{
  "prediction": [{
    "0": 0.8,
    "1": 0.2
  }]
}

Get features of the Model with features importances

Endpoint: /features

$ curl -X GET "http://localhost:5000/features"

[
  {
    "default": -1,
    "importance": 0.2,
    "name": "feature1",
    "type": "numeric"
  },
  {
    "default": -1,
    "importance": 0.1,
    "name": "feature2",
    "type": "numeric"
  },
  {
    "default": -1,
    "importance": 0.3,
    "name": "feature3",
    "type": "numeric"
  }
]

Get SHAP explanations

Endpoint: /predict?proba=1&explain=1

$ curl -d '{"feature1": 1, "feature2": 1, "feature3": 2}' -H "Content-Type: application/json" -X POST "http://localhost:5000/predict?proba=1&explain=1"

{
  "explanation": {
    "feature1": 0.10000000149011613,
    "feature2": 0.03333333383003871,
    "feature3": -0.1666666691501935
  },
  "prediction": [{
    "0": 0.7,
    "1": 0.3
  }]
}