MLFLow Tracking Server Based on Docker and AWS S3

This repo provides a docker image of MLFLow Tracking Server which is based on sqlite, an internal file system for metadata (e.g. parameters, metrics) and an AWS S3 Bucket for files and artifacts.

Prerequisites

Before you start the MLFlow Tracking Server, you must create an AWS S3 Bucket and the corresponding credentials.

1. AWS Account
2. AWS CLI
3. Create a new AWS S3 Bucket
4. Add a new AWS User to to the system
5. Assign the following policy to the user:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:HeadBucket"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::<_AWS_BUCKET_NAME_>",
                "arn:aws:s3:::<_AWS_BUCKET_NAME_>/*"
            ]
        }
    ]
}

Run the MLFlow Tracking Server

$ docker run \
    --rm \
    --name mlflow-tracking-server \
    -p 5000:5000 \
    -e PORT=5000 \
    -e FILE_DIR=/mlflow \
    -e AWS_BUCKET=<YOUR_AWS_BUCKET> \
    -e AWS_ACCESS_KEY_ID=<YOUR_AWS_ACCESS_KEY_ID> \
    -e AWS_SECRET_ACCESS_KEY=<YOUR_AWS_SECRET_ACCESS_KEY> \
    foxrider/mlflow-tracking-server:0.2.0

Access to http://127.0.0.1:5000

Environment Variables for the MLFlow Tracking Server

Required

Key	Description
FILE_DIR	Directory for artifacts and metadata (e.g. parameters, metrics)
AWS_BUCKET	Name of AWS S3 Bucket that will contain the artifacts
AWS_ACCESS_KEY_ID	AWS-Access-Key that you have created in the Prerequisite section
AWS_SECRET_ACCESS_KEY	AWS-Secret-Access-Key that you have created in the Prerequisite section

Optional

Key	Description	Default
PORT	Value for listen directive	5000

Test the MLFlow Tracking Server

Prerequisite

1. Install the AWS CLI
2. Add the AWS credentials to the client. You can either set the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY on your client or use the AWS CLI command aws configure.

Simple Python Script

See this link

Contribution

Pull requests are welcome :)

Project Initiator

Florian Muchow

Contributors

Thanks to all the contributors