This repository contains a Flask application that uses the SentenceTransformer model to compute the similarity between two input sentences. The application is containerized using Docker and is configured to run in both development and production environments using gunicorn.
A Simple Server to Compute the score of similarity between two strings.
- Flask Backend: A simple API built with Flask that accepts two sentences and returns their cosine similarity score.
- Sentence Transformers: Uses the SentenceTransformer model to encode sentences and compute their similarity.
- YAML Configurations: Reads settings such as model name, device configuration, and server options from a YAML file (config.yaml).
- Gunicorn: In production, the app is served using gunicorn for improved performance.
- Python 3.x (if you want to run it locally without Docker)
- docker-compose (optional, if you want to use it to manage services)
- Clone the Repository
- Modify the config.yaml
- Build and Run the Docker Container Build the Docker image and run the container:
docker build -t mlserver .
docker run -p 5000:5000 mlserverBy default, the application runs in production mode with gunicorn. 4. Running in Development Mode If you want to run the app in development mode with Flask’s built-in server and auto-reload, override the FLASK_ENV environment variable:
docker run -p 5000:5000 -e FLASK_ENV=development mlserver- Accessing the API Once the Docker container is running, you can send a POST request to the server to compute the sentence similarity.
Example curl command:
curl -v -H "Content-type: application/json" --data '{"s1":"This is a Surface Studio Laptop","s2":"That is a car"}' http://127.0.0.1:5000The response will contain the cosine similarity score between the two sentences. Here is a sample JSON response:
{
'status': 'success',
's1': 'This is a Surface Studio Laptop',
's2': 'That is a car',
'score': 0.08295086771249771
}- Show the Logs
docker logs -f mlserver- Docke scripts A few handy shell scripts:
- build.sh: builds the docker container.
- stop.sh: stops the docker container.
- run.sh: runs the ML server.
- restart.sh: restarts the ML server aka stop.sh and then run.sh.
- build-and-run.sh: is the combination of build.sh and restart.sh.
Make sure you source setup-env.sh to set the variables first.
To run the application locally without Docker:
- Install Dependencies First, ensure you have Python 3.x installed, then install the necessary Python packages:
pip install -r requirements.txt- Start the Flask Application Start the app using the following command:
FLASK_ENV=development flask runThe server will be available at http://127.0.0.1:5000.
You can use the docker-compose or docker compose to build and start the container:
docker-compose up --build -dTo view the logs using docker-compose, run:
docker-compose logs -fTo restart the docker-compose container, run:
docker-compose down # Stop the container
docker-compose up -d # Start the container in detached modeOr simply:
docker-compose restart flask-appThe application reads the following configurations from config.yaml:
- Model Settings: Specify the model name (e.g., paraphrase-MiniLM-L6-v2), device (auto, cuda, or cpu), and precision (float32 or float16).
- Caching: Enable or disable caching for sentence embeddings.
- Server Settings: Configure the host, port, and number of gunicorn workers.
In production, the app is served using gunicorn. You can customize the number of workers by modifying the config.yaml file.
The application exposes port 5000 by default. This can be modified in the docker run command or the config.yaml file.
Use the following script to perform a basic integration test — it builds the Docker image, starts the server locally, sends a request, and verifies that the response has a 'status' of 'OK' with a status code of 200. In particular, there are two tests:
integration-tests-docker.sh tests the Docker image and integration-tests-docker-compose.sh tests the docker-compose.yml.
source ./setup-env.sh
## on success, exit code is 0.
## on failure, exit code is 1.
## test basic docker setup
./tests/integration-tests-docker.sh
## on success, exit code is 0.
## test docker compose
## on failure, exit code is 1.
./tests/integration-tests-docker-compose.sh
This project is licensed under the MIT License.
Contribution are absolutely welcome! Please follow the guidance here
If you like this and want to support me in continuous development, you can do the following:
