Cannot generate videos with Google Colab (was previously working): RuntimeError: Could not import backend "EGL": Could not initialize

[oca112]

Discussed in #162

^{Originally posted by oca112 April 7, 2023}
I've setup a google colab with fast_stable_diffusion here: https://colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast_stable_diffusion_AUTOMATIC1111.ipynb

When trying to generate a video from the inpainted mesh, I get the following error - any ideas on how to troubleshoot this?

Error completing request Arguments: ('batch/batch_outputs/02_01-0000.ply', '375', '30', 0, '-0.037, +0.037, -0.05', '0.03, 0.03, 0.05, 0.03', False, 0, 2) {} Traceback (most recent call last): File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 56, in f res = list(func(*args, **kwargs)) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 37, in f res = func(*args, **kwargs) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/scripts/depthmap.py", line 851, in run_makevideo fn_saved = run_3dphoto_videos(fn_mesh, basename, outpath, num_frames, num_fps, crop_border, vid_traj, x_shift_range, y_shift_range, z_shift_range, [''], dolly, fnExt, vid_ssaa) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/scripts/depthmap.py", line 739, in run_3dphoto_videos vispy.use(app='egl') File "/usr/local/lib/python3.9/dist-packages/vispy/util/wrappers.py", line 93, in use use_app(app) File "/usr/local/lib/python3.9/dist-packages/vispy/app/_default_app.py", line 47, in use_app default_app = Application(backend_name) File "/usr/local/lib/python3.9/dist-packages/vispy/app/application.py", line 47, in __init__ self._use(backend_name) File "/usr/local/lib/python3.9/dist-packages/vispy/app/application.py", line 240, in _use raise RuntimeError(msg) RuntimeError: Could not import backend "EGL": Could not initialize

[KaruroChori]

Same with me while using it in a docker container. It might be that it fails in headless mode.

[thygate]

Make sure a vispy backend is installed, and that you're running on a colab with GPU ..

See vt-vl-lab/3d-photo-inpainting#23

[aduchon]

EDIT: a recent change in colab has broken a number of AI-related notebooks I use. So this may just be another.

I'm having similar issues with colab. I couldn't figure out how to translate the docker solution to colab. Minimal example:

os.environ["PYOPENGL_PLATFORM"] = "egl" 
!PYOPENGL_PLATFORM=egl python -c "from OpenGL import EGL"
print(os.environ['PYOPENGL_PLATF`ORM']) 
!pip install vispy

import vispy
vispy.use(app="egl") # the call that fails at   File "/content/drive/MyDrive/stable-diffusion-webui-colab/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/scripts/depthmap.py", line 875, in run_3dphoto_videos

Even this fails even though the environment is set.

[<ipython-input-1-39c338be6993>](https://localhost:8080/#) in <cell line: 8>()
      6 
      7 import vispy
----> 8 vispy.use(app="egl")

3 frames
[/usr/local/lib/python3.9/dist-packages/vispy/app/application.py](https://localhost:8080/#) in _use(self, backend_name)
    238                 if not try_others:
    239                     # Fail if user wanted to use a specific backend
--> 240                     raise RuntimeError(msg)
    241                 elif key in imported_toolkits:
    242                     # Warn if were unable to use an already imported toolkit

RuntimeError: Could not import backend "EGL":
Could not initialize

[danilw]

Same here, I was using my own scripts for rendering/video recording on Colab, just googled this issue.
Tried today to use my Colab script that I used before, it was working on Colab few months ago, crash with same error:

!pip install vispy

import vispy
vispy.use('egl')

RuntimeError: Could not import backend "EGL":
Could not initialize

[Sergio70]

the same thing happened to me in google colab :(

[thygate]

I have not tried to run it on colab, but perhaps this guy is onto something ..
remove frame buffer for jupyter
hyqhyq3@6573d7d
https://github.com/vispy/jupyter_rfb/

[pionerstepansky]

same problem(
hyqhyq3@6573d7d doesnt help

[conor-is-my-name]

I am also having these issues. I can't get EGL to work in docker. Here's what I have, but can't get over the goal line.

FROM alpine/git:2.36.2 as download

COPY clone.sh /clone.sh

RUN . /clone.sh taming-transformers https://github.com/CompVis/taming-transformers.git 24268930bf1dce879235a7fddd0b2355b84d7ea6
&& rm -rf data assets **/*.ipynb

RUN . /clone.sh stable-diffusion-stability-ai https://github.com/Stability-AI/stablediffusion.git 47b6b607fdd31875c9279cd2f4f16b92e4ea958e
&& rm -rf assets data//*.png data//.jpg data/**/.gif

RUN . /clone.sh CodeFormer https://github.com/sczhou/CodeFormer.git c5b4593074ba6214284d6acd5f1719b6c5d739af
&& rm -rf assets inputs

RUN . /clone.sh BLIP https://github.com/salesforce/BLIP.git 48211a1594f1321b00f14c9f7a5b4813144b2fb9
RUN . /clone.sh k-diffusion https://github.com/crowsonkb/k-diffusion.git 5b3af030dd83e0297272d861c19477735d0317ec
RUN . /clone.sh clip-interrogator https://github.com/pharmapsychotic/clip-interrogator 2486589f24165c8e3b303f84e9dbbea318df83e8

FROM alpine:3.17 as xformers
RUN apk add --no-cache aria2
RUN aria2c -x 5 --dir / --out wheel.whl 'https://github.com/AbdBarho/stable-diffusion-webui-docker/releases/download/5.0.3/xformers-0.0.20.dev528-cp310-cp310-manylinux2014_x86_64-pytorch2.whl'

RUN apk update && apk upgrade
RUN apk add --no-cache
build-base
mesa-dev
mesa-gl

FROM python:3.10.9-slim
ENV VISPY_APP_LIBRARY=egl
ENV DISPLAY=:0
ENV CUDA_VISIBLE_DEVICES=0

ENV DEBIAN_FRONTEND=noninteractive PIP_PREFER_BINARY=1

RUN apt-get update && apt install fonts-dejavu-core rsync git jq moreutils aria2 -y &&
apt-get install -y libgles2-mesa-dev && apt-get clean

RUN --mount=type=cache,target=/cache --mount=type=cache,target=/root/.cache/pip
aria2c -x 5 --dir /cache --out torch-2.0.0-cp310-cp310-linux_x86_64.whl -c
https://download.pytorch.org/whl/cu118/torch-2.0.0%2Bcu118-cp310-cp310-linux_x86_64.whl &&
pip install /cache/torch-2.0.0-cp310-cp310-linux_x86_64.whl torchvision --index-url https://download.pytorch.org/whl/cu118

RUN --mount=type=cache,target=/root/.cache/pip
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git &&
cd stable-diffusion-webui &&
git reset --hard d7aec59c4eb02f723b3d55c6f927a42e97acd679 &&
pip install -r requirements_versions.txt

RUN --mount=type=cache,target=/root/.cache/pip
--mount=type=bind,from=xformers,source=/wheel.whl,target=/xformers-0.0.20.dev528-cp310-cp310-manylinux2014_x86_64.whl
pip install /xformers-0.0.20.dev528-cp310-cp310-manylinux2014_x86_64.whl

ENV ROOT=/stable-diffusion-webui

COPY --from=download /repositories/ ${ROOT}/repositories/
RUN mkdir ${ROOT}/interrogate && cp ${ROOT}/repositories/clip-interrogator/data/* ${ROOT}/interrogate
RUN --mount=type=cache,target=/root/.cache/pip
pip install -r ${ROOT}/repositories/CodeFormer/requirements.txt

RUN --mount=type=cache,target=/root/.cache/pip
pip install pyngrok
git+https://github.com/TencentARC/GFPGAN.git@8d2447a2d918f8eba5a4a01463fd48e45126a379
git+https://github.com/openai/CLIP.git@d50d76daa670286dd6cacf3bcd80b5e4823fc8e1
git+https://github.com/mlfoundations/open_clip.git@bb6e834e9c70d9c27d0dc3ecedeebeaeb1ffad6b

Note: don't update the sha of previous versions because the install will take forever

instead, update the repo state in a later step

TODO: either remove if fixed in A1111 (unlikely) or move to the top with other apt stuff

RUN apt-get -y install libgoogle-perftools-dev && apt-get clean
ENV LD_PRELOAD=libtcmalloc.so

ARG SHA=89f9faa63388756314e8a1d96cf86bf5e0663045
RUN --mount=type=cache,target=/root/.cache/pip
cd stable-diffusion-webui &&
git fetch &&
git reset --hard ${SHA} &&
pip install -r requirements_versions.txt

RUN --mount=type=cache,target=/root/.cache/pip pip install -U opencv-python-headless

RUN apt install -y libfontconfig1-dev wget ffmpeg libsm6 libxext6 libxrender-dev mesa-utils-extra libegl1-mesa-dev libgles2-mesa-dev xvfb libglfw3-dev
ENV DISPLAY=:0

RUN pip install scipy matplotlib scikit-image

COPY . /docker

RUN
python3 /docker/info.py ${ROOT}/modules/ui.py &&
mv ${ROOT}/style.css ${ROOT}/user.css && \

one of the ugliest hacks I ever wrote \

sed -i 's/in_app_dir = ./in_app_dir = True/g' /usr/local/lib/python3.10/site-packages/gradio/routes.py &&
git config --global --add safe.directory ''

WORKDIR ${ROOT}

COPY ./startup.sh /${ROOT}
RUN bash startup.sh

ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
ENV NVIDIA_VISIBLE_DEVICES=all
ENV CLI_ARGS=""
EXPOSE 7862
ENTRYPOINT ["/docker/entrypoint.sh"]
CMD python -u webui.py --listen --port 7862 --api ${CLI_ARGS}

[simonmcnair]

I believe this requires cuda-libraries-dev-11-8 to work.

[rybi]

Did anyone solve it?

[danilw]

@rybi base on this vispy/vispy#2469 (comment)
and this https://www.reddit.com/r/StableDiffusion/comments/12t8tc7/comment/jh2rwe1/?utm_source=share&utm_medium=web2x&context=3

quote from reddit:

Colab PM here.

Sorry. We prioritize interactive notebook compute for free of charge tier users - the webui is just using Colab as a convenient free GPU. We try to be cool, but usage has really been increasing lately and it's very noticeable now in our costs. I am constrained with what we can do with our budget!

I believe - since vispy used by "everyone", maybe even more than WebUI, to make videos on google-colab - they make it not work on their side same as with WebUI case.

While vispy perfectly fine working from local-colab or python-notebook local-setup. I can be wrong ofc, but this all information I know and it lead to this conclusion.