Multi GPUs in docker nvidia container

[ljubomirb]

When starting, I've noticed text:

Namespace(share=False, server='0.0.0.0', port=None, inbrowser=False)
Free VRAM 11.4376220703125 GB
High-VRAM Mode: False

I would interpret this as using one gpu (30xx series, 12gb). However, I started this in docker (having docker nvidia container working), and --gpus passed.
Since i have 2 gpus, used by ollama in same configuration, I was expecting some other text.
So, will it support multiple GPUs in this configuration?

In any case, when everything started to be downloaded, I stopped, since it look like tens od GB, so I will wait for official docker compose so I don't need to redo it again.

[edit]: my setup was in docker in linux

if anyone is interested, my patched version - just to try it (and based on some pull i've seen here)

# Use the Python 3.10 slim image as the base
FROM python:3.10-slim

# Set the working directory to /app
WORKDIR /app

# Install necessary system dependencies (for OpenCV and other libraries)
RUN apt-get update && apt-get install -y \
    libsm6 \
    libxext6 \
    libgl1 \
    libglib2.0-0 \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Install PyTorch with CUDA support (choose the appropriate CUDA version as needed)
RUN pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# Copy the requirements.txt file and install Python dependencies
COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy the rest of the application code into the container
COPY . .

# Expose port for Gradio (if you are using Gradio for serving the model)
EXPOSE 7860

# Set the default command to run the demo script
CMD ["python", "demo_gradio.py", "--server", "0.0.0.0"]

docker build -t framepack-container .

docker run --gpus all -it -p 7860:7860 --name framepack-dev2 framepack-container

[Kin-Zhang]

Related pull request I found, looks like is from your Dockerfile: #106

[ljubomirb]

Related pull request I found, looks like is from your Dockerfile: #106

I think I may have used that one for start (the author of pull is original author), and I do hope official dockerfile or compose file is created.
However, my main concern is - am I missing something in my setup, so only one gpu is (as it looks from text during startup) used, or use of multiple GPUs is not to expect?

[Kin-Zhang]

I think it's not Docker issue. But looks like this repo currently only supports one GPU. I found there are some issues also mentioning whether it have multi-GPU support.

[odrevet]

I am confused, what does your patched version adds besides comments from the original dockerfile 9f58794

[ljubomirb]

I am confused, what does your patched version adds besides comments from the original dockerfile 9f58794

Exactly that!

To explain, since obviously needed - my question was: will you support docker? Also in same comment I've noted that I tested that (all this in docker), for which I found possible issue that it is using only one GPU, and for clarity, I've pasted that patch, and also, under that docker pack, a set of commands to build and run docker image from that patch.
In essence, I've attempted to short time for someone to test, since everything is already in comment, and also, to explain my setup to test all this great repo.

The issue was, again - i had only one GPU active, and I also noted - before testing further, docker patch needs to be accepted, otherwise one would have to re-download again everything once you've applied patch.