cannot import name '_C' from 'groundingdino'

[mvoodarla]

I'm able to run it properly on a local GPU machine I've got but when I move this to the cloud on a docker image, I'm getting this issue. Have a feeling there's some system package I'm missing or something but not 100% sure.

[SlongLiu]

Thanks for your issue.

We have not tested the model on docker. Can you provide more details of the outputs?

[Innary]

@mvoodarla I think maybe it's because Rust is not installed.

[mvoodarla]

This is during inference, not training. It's likely some system package thing, I'll keep investigating.

[fkodom]

@mvoodarla I believe @Innary is right -- I had the same issue locally until I installed both cargo and rustc.

sudo apt update && sudo apt install -y cargo rustc

Also needed to build the image using NVIDIA container runtime. By default, Docker uses the non-NVIDIA runtime for building the image, which causes issues when building the custom CUDA ops for GroundingDINO.

Easiest way I found to override that is to update /etc/docker/daemon.json:
[Link to StackOverflow discussion]

{
    "runtimes": {
        "nvidia": {
            "path": "/usr/bin/nvidia-container-runtime",
            "runtimeArgs": []
         } 
    },
+   "default-runtime": "nvidia" 
}

With that change, this Dockerfile is working for me. Able to load the model, run inference on GPU, etc.

ARG UBUNTU_VERSION="20.04"
ARG CUDA_VERSION="11.6.1"
ARG CUDA_OPSET="devel"
ARG PY_VERSION="3.10"

FROM nvidia/cuda:$CUDA_VERSION-$CUDA_OPSET-ubuntu$UBUNTU_VERSION
ENV DEBIAN_FRONTEND="noninteractive"

# TODO: Check if any of these 'apt' packages are unnecessary
RUN apt update --fix-missing && apt upgrade -y \
    && apt install -y build-essential cargo curl ffmpeg git libsm6 libxext6 rustc software-properties-common unzip \
    && apt clean \
    && rm -rf /var/lib/apt/lists/*

# Setup Python using the 'ppa:deadsnakes/ppa' apt repository
# NOTE: ARG variables are cleared after each FROM statement. A simple workaround
# to this is to re-declare the ARG without a default value:
# https://github.com/moby/moby/issues/34129#issuecomment-315852422
ARG PY_VERSION
RUN add-apt-repository ppa:deadsnakes/ppa \
    && apt update \
    && apt install -y python$PY_VERSION-dev python$PY_VERSION-distutils \
    && update-alternatives --install /usr/bin/python python /usr/bin/python$PY_VERSION 10 \
    && curl -sS https://bootstrap.pypa.io/get-pip.py | python \
    && apt clean \
    && rm -rf /var/lib/apt/lists/*

# Create the working directory for our code
RUN mkdir /app
WORKDIR /app

# Install dependencies
# - Bug in 'setuptools==66.0.0' with running 'python setup.py develop'
# - 'tokenizers' gives a version error with 'packaging>21.3'
# - Be sure to install PyTorch with CUDA==11.6, so extensions build for GroundingDINO
COPY requirements.txt .
RUN pip install -r requirements.txt \
    --no-cache-dir \
    --index-url https://download.pytorch.org/whl/cu116 \
    --extra-index-url https://pypi.org/simple

# Install GroundingDINO
# NOTE: Use latest commit hash as of 2023-03-27
RUN git clone https://git@github.com/IDEA-Research/GroundingDINO \
    && cd GroundingDINO \
    && git checkout 858efccbad1aed50644f0185e49f4254a9af7560 \
    && python setup.py develop

where my requirements.txt file looks like this. (Still need to pin other package versions.)

addict
matplotlib
ninja
opencv-python
# 'tokenizers' gives a version error with 'packaging>21.3'
packaging==21.3
pycocotools
# Bug in 'setuptools==66.0.0' with running 'python setup.py develop'
setuptools==65.3.0
timm
# NOTE: Must include '--index-url https://download.pytorch.org/whl/cu116' when installing these PyTorch versions.
torch==1.12.1+cu116
torchvision==0.13.1+cu116
torchaudio==0.12.1 
transformers
yapf

[Enverrrr]

Hey I want to infer on my laptop i've totally installed repository however when i tried to infer with inference_on_a_image.py i took same error. How can i solve that

[Innary]

Hey I want to infer on my laptop i've totally installed repository however when i tried to infer with inference_on_a_image.py i took same error. How can i solve that

Perhaps it's caused by the lack of Rust-related environment installed on your system.

[1750585724]

I cannot install GroundingDINO. Can you help me?

[ByChelsea]

I installed Rust but this issue still exits... and I do not use docker :(
Can anyone help me? please...

I‘ve solved this issue, but for me, it's nothing to do with Rust. Simply python3.7 does not work, and I fogot to change it to python3.8...

[xu5zhao]

I encountered the same problem and solved the problem by looking at this issue!

The key is setting up CUDA_HOME correctly before installing repo~ here is an example：

export CUDA_HOME=/usr/local/cuda-11.6/
pip install -e .

If the GroundingDINO/groundingdino/_C.cpython-38-x86_64-linux-gnu.so file is generated, it indicates success.

If you still encounter problems, please refer to the issue under another project : https://github.com/IDEA-Research
/
Grounded-Segment-Anything ,for a more detailed explanation.

[darshats]

Hi,
We are facing the same error deploying Dino to azure ml. The build pipeline uses cpu machines. Hence entire setup.py is run on a cpu box.
First we had the failure of import torch in setup.py. This we got around by adding torch to conda.yml used by azure ml build pipeline. But this pulls in a cpu version of torch. The result is that at runtime when code is deployed to a gpu machine in azure, it complains about torch not being compiled for gpu.

Q:

1. is there a way we can simplify setup.py to not do dynamic compilation of the cpp files. Is there a solution where we can use precompiled _C.* files instead of creating on the fly? That would solve a whole lot of problems
2. the deformable attention seems to use interop to cpp. Is there any way to do this purely within pytorch and avoid issue 1.

[HassanBinHaroon]

@xu5zhao and all Try this easy and quick inference on Google Colab. GroundingDINO-Inference

[TemugeB]

A quick fix is to just use the pytorch implementation of deformable attention.

Change this line:

GroundingDINO/groundingdino/models/GroundingDINO/ms_deform_attn.py

Line 330 in 654f5e8

if torch.cuda.is_available() and value.is_cuda:

to:

        if not torch.cuda.is_available():

I don't know how much performance hit this will cause but since we are infering on single images, probably negligible.

Also make sure to send the model to cuda after you load it:

model = load_model("groundingdino/config/GroundingDINO_SwinT_OGC.py", "weights/groundingdino_swint_ogc.pth")
model = model.to('cuda:0')

[Kurinosuke118]

@mvoodarla
How was this issue eventually resolved?
I am also facing the same issue. Could you please advise?

[Kurinosuke118]

@fkodom

RUN git clone https://git@github.com/IDEA-Research/GroundingDINO
&& cd GroundingDINO
&& git checkout 858efcc
&& python setup.py develop

Have you made any changes to the contents of the GroundingDINO's requirements.txt?