Skip to content

Conversation

@williamw
Copy link
Collaborator

@williamw williamw commented Oct 1, 2025

This PR adds complete Docker containerization for the GenAI Cookbook, allowing users to run both the MAX model server and Next.js web application in a single container.

Dockerfile Architecture

The Dockerfile builds on Modular's official base images with GPU support:

  • GPU Selection: Uses ARG GPU_TYPE=nvidia to allow switching between NVIDIA and AMD base images via --build-arg GPU_TYPE=amd
  • Base Image: modular/max-${GPU_TYPE}-base provides MAX engine with GPU drivers
  • Runtime: Installs Node.js 18.x, npm, and pm2 for process management
  • Build Process: Copies dependencies, runs npm install and npm run build for production Next.js bundle
  • Entry Point: Uses pm2-runtime to orchestrate multiple processes from ecosystem.config.js

Process Management with pm2

The container runs two processes managed by pm2 via ecosystem.config.js:

  1. max-llm: Starts the MAX model server
    - Runs max serve --model ${MAX_MODEL} (defaults to google/gemma-3-27b-it)
    - Listens on port 8000
    - Auto-restarts on crashes
  2. web-app: Starts the Next.js application
    - Waits for MAX health endpoint (wait-on http://0.0.0.0:8000/health)
    - Runs npm start on port 3000
    - Auto-configured to use the local MAX endpoint

This orchestration ensures the MAX server is ready before the web app starts, eliminating race conditions.

Documentation Updates

Added "Running with Docker" section to README.md:

  • Build commands for NVIDIA (default) and AMD GPUs using --build-arg
  • Run commands with appropriate device flags (--gpus all for NVIDIA, --device /dev/kfd --device /dev/dri for AMD)
  • Configuration guide explaining environment variables (HF_TOKEN, MAX_MODEL), ports (8000, 3000), and volume mounts
  • Updated default model from meta-llama/Llama-3.1-8B-Instruct to google/gemma-3-27b-it throughout documentation

Users can now build and run the entire stack with a single docker build and docker run command.

@williamw williamw merged commit 7535959 into main Oct 3, 2025
1 check passed
@williamw williamw deleted the billw/docker branch October 3, 2025 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants