NanoDetTrack.mp4
The model tries to keep track of the individual objects found in the scenes. The example video follows the walkers as they stroll along. Obvious, not a simple task. Not only does the detection needs to be at a robust level. You also have to solve occlusion as the walkers pass each other.
Paper: https://arxiv.org/pdf/2110.06864v2.pdf
Special made for a bare Raspberry Pi 4, see Q-engineering deep learning examples
| Model | size | objects | mAP | RPi 4 64-OS 1950 MHz |
|---|---|---|---|---|
| Tensorflow lite | 300x300 | 80 | 21.0 | 24.6 FPS |
| NanoDet | 320x320 | 80 | 20.6 | 11.8 FPS |
| YoloX | 416x416 | 80 | 25.8 | 7.2 FPS |
To run the application, you have to:
- A raspberry Pi 4 with a 64-bit operating system. It can be the Raspberry 64-bit OS, or Ubuntu 18.04 / 20.04. Install 64-bit OS
- The Tencent ncnn framework installed. Install ncnn
- OpenCV 64 bit installed. Install OpenCV 4.5
- Code::Blocks installed. (
$ sudo apt-get install codeblocks)
To extract and run the network in Code::Blocks
$ mkdir MyDir
$ cd MyDir
$ wget https://github.com/Qengineering/NanoDet-Tracking-ncnn-RPi_64-bit/archive/refs/heads/main.zip
$ unzip -j master.zip
Remove master.zip, LICENSE and README.md as they are no longer needed.
$ rm master.zip
$ rm LICENSE
$ rm README.md
Your MyDir folder must now look like this:
├── include
│ ├── BYTETracker.h
│ ├── dataType.h
│ ├── kalmanFilter.h
│ ├── lapjv.h
│ └── STrack.h
├── NanoDet_Byte_Track.cbp
├── nanodet_m.bin
├── nanodet_m.param
├── palace.mp4
├── src
│ ├── BYTETracker.cpp
│ ├── kalmanFilter.cpp
│ ├── lapjv.cpp
│ ├── nanodet.cpp
│ ├── STrack.cpp
│ └── utils.cpp
├── tree.txt
└── Walkers.mp4
Install eigen-3.3.9 google or baidu(code:ueq4) (if not done earlier by Tensorflow Lite).
unzip eigen-3.3.9.zip
cd eigen-3.3.9
mkdir build
cd build
cmake ..
sudo make install
To run the application load the project file NanoDet_Byte_Track.cbp in Code::Blocks. More info or
if you want to connect a camera to the app, follow the instructions at Hands-On.
Many thanks to nihui and ifzhang
