Jetson gpu cuda support

[alekslyse]

As its a build that support gpu and cuda have any tried to use a nvidia jetson nano that got about 120 cuda cores?

[pospielov]

Hi, unfortunately, Nvidia Jetson Nano has ARM CPU which is not currently supported by CompreFace. So even CPU version won't work

[SilvioGiancola]

I recently tried on a Jetson TX1 (aarch64). The docker composes without displaying any issue, but http://localhost:8000/login is not reachable. Will it be supported in a future release? What are the current limitations/challenges that impede the the docker container to run on aarch64?

[pospielov]

There are several problems with supporting arm CPU:

1. There is no easy way to automate such build without an arm-based computer. The first option is to find a contributor who has arm device and wants to build CompreFace for Arm. But the downside will be that we won't be able to publish Arm version if he leaves the team. The second option is Docker Buildx. As far as I know, it's in an experimental stage now, but this is the only good option. And we still need to understand how to use it.
2. Not all libraries are built for Arm CPU. E.g. as far as I know, you can't just install Tensorflow from pip. Best case scenario - we find ready to use guide and build a fully compatible Tensorflow. Worst scenario - we build only TensorFlow lite and will probably have a separate code for arm platforms.
3. There are lots of options to run neural networks on arm based devices. E.g. if you use Jetson Nano, you'll expect to use full Tensorflow, because you have a good CUDA accelerator. But others could want to use Google Coral, they will want to use TensorFlow Lite because this is the only option. Another option - Intel neural compute stick 2, which also could be run on Raspberry, but can use only the Intel Open Vino framework. So we need to find a way to support all of these frameworks and do not make our code too complex to support.
   I personally believe that in the near future arm based devices will become more popular(we already have MacBook on arm CPU) and more people spend their time expanding arm ecosystem. CompreFace will definitely support arm devices in the future, the question only - when.

[SilvioGiancola]

Thank you for your very detailed answer, it is very appreciated.
Looking forward further development for ARM-based architectures in a (near?) future :)

[marcfielding1]

For Jetson devices I have several Nano's sitting around I can run tests on, I really would like to see this for, believe it or not, a home assistant project, part of it is an internal tool called "Kane" that integrates with every device in our office(smart tv's, laptops, phones, etc) - part of it for our home shopping hub.

Re the comment on building/buildX I can try and help, just chuck me some tasks and I'll see if I can "make it work"?

[marcfielding1]

I've got just under a week free by the way so I can get a good run at some research/groundwork if you guys respond quickly :-)

[pospielov]

We would really like to integrate with the home assistant as well, so your research would be so helpful.
What I suggest to start from:
The critical point here is to build compreface-core image.
I think we can start building it on Jetson Nano, so we won't need to setup buildX.
To build it, clone the repository and run:
docker build -t embedding-calculator --build-arg SKIP_TESTS=true .
I think It will fail on RUN pip --no-cache-dir install -r requirements.txt step as there won't be all libraries compiled for ARM in the pip repository.
Then we need to collect the list of these libraries and think about what should we do - probably we will need to compile them ourselves.
For faster response, you can send your skype (or Whatsapp, or Telegram, any other messenger) to your email: compreface.support@exadel.com

[marcfielding1]

Heya, sure so I had a thought you can just the actions buildx plugin for this sorta stuff, let me pull the image though and try it locally! Obviously don't leave token as string ya want it in a secret.

I'll ping my WhatsApp there shortly!

Something like this, although this is just out of a workflow I have for a NodeJS project. I can't remember if the names of build targets are right though.

jobs:
  docker-build:
    runs-on: ubuntu-latest
    steps:
      - name: Cache dependencies
        uses: actions/cache@v2
        with:
          path: ~/.npm
          key: npm-${{ hashFiles('package-lock.json') }}
          restore-keys: npm-          
      -
        name: Login to DockerHub
        uses: docker/login-action@v1 
        with:
          username: "marcfielding1"
          password: "TOKEN"
      - uses: actions/setup-node@v2
        with:
          node-version: '14.0.0'
      -
        name: Set up QEMU
        uses: docker/setup-qemu-action@v1
      -
        name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1          
      -
        name: Build and push
        uses: docker/build-push-action@v2
        with:
          platforms: linux/arm,linux/arm64,linux/amd64
          push: true
          tags: repo/image:latest

[marcfielding1]

Heya,

Sooo getting there, there is currently a problem with Scikit-image, I've found a solution here from the NVIDIA forums that I'm going to try out.

Build output below:

Sending build context to Docker daemon  10.91MB
Step 1/36 : ARG BASE_IMAGE
Step 2/36 : FROM ${BASE_IMAGE:-python:3.7-slim}
 ---> a1776b632fa3
Step 3/36 : RUN apt-get update && apt-get install -y build-essential cmake git wget unzip         curl yasm pkg-config libswscale-dev libtbb2 libtbb-dev libjpeg-dev         libpng-dev libtiff-dev libavformat-dev libpq-dev libfreeimage3     && rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 5610457fda7b
Step 4/36 : SHELL ["/bin/bash", "-c"]
 ---> Using cache
 ---> 5cc5df239fc2
Step 5/36 : WORKDIR /app/ml
 ---> Using cache
 ---> 3099534ddcb5
Step 6/36 : COPY requirements.txt .
 ---> Using cache
 ---> 63e0c2611094
Step 7/36 : RUN pip --no-cache-dir install -r requirements.txt
 ---> Running in 234b16d98c44
Collecting attrs==20.2.0
  Downloading attrs-20.2.0-py2.py3-none-any.whl (48 kB)
Collecting cached-property==1.5.2
  Downloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Collecting colour==0.1.5
  Downloading colour-0.1.5-py2.py3-none-any.whl (23 kB)
Collecting flasgger==0.9.5
  Downloading flasgger-0.9.5-py2.py3-none-any.whl (3.8 MB)
Collecting Flask==1.1.2
  Downloading Flask-1.1.2-py2.py3-none-any.whl (94 kB)
Collecting gdown~=3.12
  Downloading gdown-3.13.0.tar.gz (9.3 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Collecting Werkzeug==1.0.1
  Downloading Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
Collecting mock~=4.0.2
  Downloading mock-4.0.3-py3-none-any.whl (28 kB)
Collecting pytest~=6.1.2
  Downloading pytest-6.1.2-py3-none-any.whl (272 kB)
Collecting pytest-mock~=3.3.1
  Downloading pytest_mock-3.3.1-py3-none-any.whl (11 kB)
Collecting requests~=2.24.0
  Downloading requests-2.24.0-py2.py3-none-any.whl (61 kB)
Collecting pylama~=7.7.1
  Downloading pylama-7.7.1-py2.py3-none-any.whl (35 kB)
Collecting Pillow~=8.0.1
  Downloading Pillow-8.0.1-cp37-cp37m-manylinux2014_aarch64.whl (2.1 MB)
Collecting imagecodecs~=2020.5.30
  Downloading imagecodecs-2020.5.30.tar.gz (9.0 MB)
^TCollecting numpy~=1.19.5
  Downloading numpy-1.19.5-cp37-cp37m-manylinux2014_aarch64.whl (12.4 MB)
Collecting scipy~=1.5.4
  Downloading scipy-1.5.4-cp37-cp37m-manylinux2014_aarch64.whl (24.4 MB)
Collecting opencv-python~=4.4.0
  Downloading opencv-python-4.4.0.46.tar.gz (88.9 MB)
  Installing build dependencies: started
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Collecting scikit-learn~=0.23.2
  Downloading scikit-learn-0.23.2.tar.gz (7.2 MB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Collecting scikit-image~=0.17.2
  Downloading scikit-image-0.17.2.tar.gz (29.8 MB)
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3l0blh2u/scikit-image_870a4d563b4d40359e07b1cde9c82659/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3l0blh2u/scikit-image_870a4d563b4d40359e07b1cde9c82659/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-uaakw08x
         cwd: /tmp/pip-install-3l0blh2u/scikit-image_870a4d563b4d40359e07b1cde9c82659/
    Complete output (7 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-3l0blh2u/scikit-image_870a4d563b4d40359e07b1cde9c82659/setup.py", line 234, in <module>
        'build_ext': openmp_build_ext(),
      File "/tmp/pip-install-3l0blh2u/scikit-image_870a4d563b4d40359e07b1cde9c82659/setup.py", line 58, in openmp_build_ext
        from numpy.distutils.command.build_ext import build_ext
    ModuleNotFoundError: No module named 'numpy'
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/54/fd/c1b0bb8f6f12ef9b4ee8d7674dac82cd482886f8b5cd165631efa533e237/scikit-image-0.17.2.tar.gz#sha256=bd954c0588f0f7e81d9763dc95e06950e68247d540476e06cb77bcbcd8c2d8b3 (from https://pypi.org/simple/scikit-image/) (requires-python:>=3.6). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement scikit-image~=0.17.2 (from versions: 0.7.2, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.9.1, 0.9.3, 0.10.0, 0.10.1, 0.11.2, 0.11.3, 0.12.0, 0.12.1, 0.12.2, 0.12.3, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.14.2, 0.14.3, 0.14.5, 0.15.0, 0.16.2, 0.17.1, 0.17.2, 0.18.0rc0, 0.18.0rc1, 0.18.0rc2, 0.18.0, 0.18.1, 0.18.2rc1, 0.18.2rc2, 0.18.2, 0.18.3)
ERROR: No matching distribution found for scikit-image~=0.17.2
The command '/bin/bash -c pip --no-cache-dir install -r requirements.txt' returned a non-zero code: 1

[pospielov]

this is strange because you installed the NumPy package
hmm... Could you try to check if numpy is there?
I mean you can do it even without docker just try to install numpy on your Jetson and check if you can import it.
I think it should be like:

pip3 install numpy=1.19.5
python3
from numpy.distutils.command.build_ext import build_ext

[marcfielding1]

Quick update from me the issue now imagecodecs, installing numpy manually via RUN in the dockerfile somehow fixed the problem with pip install requirements.

Just figuring out the installation packages required. Turns out imagecodecs is a bit of beast.

[pospielov]

https://scikit-image.org/docs/dev/install.html#id5
scikit-image has tips for arm CPU, including how to handle imagecodecs