Merge pull request #37 from kaylieee/image-input-sample

Add Image Input Sample

samples/ImageInput/README.md

@@ -0,0 +1,31 @@
+# Sample Application using Azure OpenAI Assistants with Image Input Support (Python)
+
+This sample includes a simple Python [Quart](https://quart.palletsprojects.com/en/latest/) app that streams responses from OpenAI Assistant to an HTML/JS frontend using Server-Sent Events (SSEs). The application supports both image (.jpg/jpeg, .webp, .gif, .png) and text inputs.
+
+The sample is designed for use with [Docker containers](https://www.docker.com/), both for local development and Azure deployment. For Azure deployment to [Azure Container Apps](https://learn.microsoft.com/azure/container-apps/overview), please use this [template](https://github.com/Azure-Samples/openai-chat-app-quickstart) and replace the `src` folder content with this application.
+
+## Local development with Docker
+
+This sample includes a `docker-compose.yaml` for local development which creates a volume for the app code. That allows you to make changes to the code and see them instantly.
+
+1. Install [Docker Desktop](https://www.docker.com/products/docker-desktop/). If you opened this inside Github Codespaces or a Dev Container in VS Code, installation is not needed. ⚠️ If you're on an Apple M1/M2, you won't be able to run `docker` commands inside a Dev Container; either use Codespaces or do not open the Dev Container.
+
+2. Make sure that the `.env` file exists.
+
+3. Store keys and endpoint information (Azure) for the OpenAI resource in the `.env` file. The key should be stored in the `.env` file as `AZURE_OPENAI_API_KEY or OPENAI_API_KEY`. This is necessary because Docker containers don't have access to your user Azure credentials.
+
+4. Start the services with this command:
+
+    ```shell
+    docker-compose up --build
+    ```
+
+5. Click 'http://localhost:50505' in the browser to run the application.
+
+## Example run
+
+![image-input-screenshot](../../assets/ImageInputAssistant.png)
+
+## Deployment to Azure
+
+As mentioned earlier, please integrate this app using [template](https://github.com/Azure-Samples/openai-chat-app-quickstart) and following the Azure Container App deployment steps there.

samples/ImageInput/docker-compose.yaml

@@ -0,0 +1,10 @@
+services:
+  app:
+    build:
+      context: ./src
+    env_file:
+      - .env
+    ports:
+      - 50505:50505
+    volumes:
+      - ./src:/code

samples/ImageInput/src/.dockerignore

@@ -0,0 +1,3 @@
+.git*
+.venv/
+**/*.pyc

samples/ImageInput/src/Dockerfile

@@ -0,0 +1,35 @@
+# ------------------- Stage 0: Base Stage ------------------------------
+FROM python:3.11-alpine AS base
+
+WORKDIR /code
+
+# Install tini, a tiny init for containers
+RUN apk add --update --no-cache tini
+
+# Install required packages for cryptography package
+# https://cryptography.io/en/latest/installation/#building-cryptography-on-linux
+RUN apk add gcc musl-dev python3-dev libffi-dev openssl-dev cargo pkgconfig
+
+# ------------------- Stage 1: Build Stage ------------------------------
+FROM base AS build
+
+COPY requirements.txt .
+
+RUN pip3 install -r requirements.txt
+
+COPY . .
+
+# ------------------- Stage 2: Final Stage ------------------------------
+FROM base AS final
+
+RUN addgroup -S app && adduser -S app -G app
+
+COPY --from=build --chown=app:app /usr/local/lib/python3.11 /usr/local/lib/python3.11
+COPY --from=build --chown=app:app /usr/local/bin /usr/local/bin
+COPY --from=build --chown=app:app /code /code
+
+USER app
+
+EXPOSE 50505
+
+ENTRYPOINT ["tini", "gunicorn", "quartapp:create_app()"]

samples/ImageInput/src/config/image_input_assistant_config.yaml

@@ -0,0 +1,19 @@
+name: image_input
+instructions: You are a helpful assistant capable of answering questions.
+model: gpt-4-turbo-2024-04-09
+assistant_id: 
+file_references: []
+tool_resources:
+  code_interpreter:
+    files: {}
+  file_search:
+    vector_stores: []
+functions: []
+file_search: false
+code_interpreter: false
+output_folder_path: /code/output
+ai_client_type: OPEN_AI
+assistant_type: assistant
+completion_settings: null
+assistant_role: user
+config_folder: null

samples/ImageInput/src/gunicorn.conf.py

@@ -0,0 +1,20 @@
+import multiprocessing
+import os
+
+from dotenv import load_dotenv
+
+load_dotenv()
+
+max_requests = 1000
+max_requests_jitter = 50
+log_file = "-"
+bind = "0.0.0.0:50505"
+
+if not os.getenv("RUNNING_IN_PRODUCTION"):
+    reload = True
+
+num_cpus = multiprocessing.cpu_count()
+workers = 1 #(num_cpus * 2) + 1
+worker_class = "uvicorn.workers.UvicornWorker"
+
+timeout = 120

samples/ImageInput/src/pyproject.toml

@@ -0,0 +1,20 @@
+[project]
+name = "quartapp"
+version = "1.0.0"
+description = "Create a simple chat app using Quart and OpenAI"
+dependencies = [
+    "quart",
+    "werkzeug",
+    "gunicorn",
+    "uvicorn[standard]",
+    "openai",
+    "azure-identity",
+    "aiohttp",
+    "python-dotenv",
+    "pyyaml",
+    "azure-ai-assistant"
+    ]
+
+[build-system]
+requires = ["flit_core<4"]
+build-backend = "flit_core.buildapi"

samples/ImageInput/src/quartapp/__init__.py

@@ -0,0 +1,22 @@
+# Copyright (c) Microsoft. All rights reserved.
+# Licensed under the MIT license. See LICENSE.md file in the project root for full license information.
+
+import logging
+import os
+
+from quart import Quart
+
+
+def create_app():
+    if os.getenv("RUNNING_IN_PRODUCTION"):
+        logging.basicConfig(level=logging.INFO)
+    else:
+        logging.basicConfig(level=logging.DEBUG)
+
+    app = Quart(__name__)
+
+    from . import chat  # noqa
+
+    app.register_blueprint(chat.bp)
+
+    return app