DeepStream-Yolo

NVIDIA DeepStream SDK 7.1 / 7.0 / 6.4 / 6.3 / 6.2 / 6.1.1 / 6.1 / 6.0.1 / 6.0 / 5.1 configuration for YOLO models

For now, I am limited for some updates. Thank you for understanding.

YOLO-Pose: https://github.com/marcoslucianops/DeepStream-Yolo-Pose

YOLO-Seg: https://github.com/marcoslucianops/DeepStream-Yolo-Seg

YOLO-Face: https://github.com/marcoslucianops/DeepStream-Yolo-Face

Important: please export the ONNX model with the new export file, generate the TensorRT engine again with the updated files, and use the new config_infer_primary file according to your model

Improvements on this repository

Support for INT8 calibration
Support for non square models
Models benchmarks
Support for Darknet models (YOLOv4, etc) using cfg and weights conversion with GPU post-processing
Support for D-FINE, RT-DETR, CO-DETR (MMDetection), YOLO-NAS, PPYOLOE+, PPYOLOE, DAMO-YOLO, Gold-YOLO, RTMDet (MMYOLO), YOLOX, YOLOR, YOLO11, YOLOv10, YOLOv9, YOLOv8, YOLOv7, YOLOv6, YOLOv5u and YOLOv5 using ONNX conversion with GPU post-processing
GPU bbox parser
Custom ONNX model parser
Dynamic batch-size
INT8 calibration (PTQ) for Darknet and ONNX exported models

Getting started

Requirements

DeepStream 7.1 on x86 platform

DeepStream 7.0 on x86 platform

DeepStream 6.4 on x86 platform

DeepStream 6.3 on x86 platform

DeepStream 6.2 on x86 platform

DeepStream 6.1.1 on x86 platform

DeepStream 6.1 on x86 platform

DeepStream 6.0.1 / 6.0 on x86 platform

DeepStream 5.1 on x86 platform

DeepStream 7.1 on Jetson platform

DeepStream 7.0 on Jetson platform

DeepStream 6.4 on Jetson platform

DeepStream 6.3 on Jetson platform

DeepStream 6.2 on Jetson platform

DeepStream 6.1.1 on Jetson platform

DeepStream 6.1 on Jetson platform

DeepStream 6.0.1 / 6.0 on Jetson platform

DeepStream 5.1 on Jetson platform

Supported models

Basic usage

1. Download the repo

git clone https://github.com/marcoslucianops/DeepStream-Yolo.git
cd DeepStream-Yolo

2. Download the `cfg` and `weights` files from Darknet repo to the DeepStream-Yolo folder

3. Compile the lib

3.1. Set the CUDA_VER according to your DeepStream version

export CUDA_VER=XY.Z

x86 platform

DeepStream 7.1 = 12.6
DeepStream 7.0 / 6.4 = 12.2
DeepStream 6.3 = 12.1
DeepStream 6.2 = 11.8
DeepStream 6.1.1 = 11.7
DeepStream 6.1 = 11.6
DeepStream 6.0.1 / 6.0 = 11.4
DeepStream 5.1 = 11.1

Jetson platform

DeepStream 7.1 = 12.6
DeepStream 7.0 / 6.4 = 12.2
DeepStream 6.3 / 6.2 / 6.1.1 / 6.1 = 11.4
DeepStream 6.0.1 / 6.0 / 5.1 = 10.2

3.2. Make the lib

make -C nvdsinfer_custom_impl_Yolo clean && make -C nvdsinfer_custom_impl_Yolo

4. Edit the `config_infer_primary.txt` file according to your model (example for YOLOv4)

[property]
...
custom-network-config=yolov4.cfg
model-file=yolov4.weights
...

NOTE: For Darknet models, by default, the dynamic batch-size is set. To use static batch-size, uncomment the line

...
force-implicit-batch-dim=1
...

NOTE: There are multiple configuration examples available. For the standard YOLOv8 model, use config_infer_primary_yoloV8.txt. If you're using the custom YOLOv8 model (the model with batched input, NHWC order and custom output), use config_infer_primary_yoloV8_custom.txt.

5. Run

Gstreamer Pipeline

NvInfer

Standard YoloV8 model (yolov8s.onnx):

gst-launch-1.0 filesrc location=traffic-video.mp4 ! queue name=filesrc ! qtdemux ! h264parse config-interval=1 ! queue name=decoder ! nvv4l2decoder  ! queue name=stream0 ! streammux.sink_0 nvstreammux name=streammux width=1920 height=1080 batch-size=1 batched-push-timeout=5000000 ! nvinfer config-file-path=config_infer_primary_yoloV8.txt name=primary-nvinference-engine1 unique-id=1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! queue ! nvvideoconvert ! queue name=nvvideoconvert ! nvdsosd name=on_screen_display process-mode=0 display-text=True display-bbox=True ! queue name=osd ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h265enc bitrate=4000000 ! h265parse config-interval=1 ! queue name=file_codec_parser ! qtmux ! queue name=file_muxer ! filesink location=result.mp4

Custom YoloV8 model (yolo_obj-yolov8s_640x640-v1.onnx):

gst-launch-1.0 filesrc location=traffic-video.mp4 ! queue name=filesrc ! qtdemux ! h264parse config-interval=1 ! queue name=decoder ! nvv4l2decoder  ! queue name=stream0 ! streammux.sink_0 nvstreammux name=streammux width=1920 height=1080 batch-size=1 batched-push-timeout=5000000 ! nvinfer config-file-path=config_infer_primary_yoloV8_custom.txt name=primary-nvinference-engine1 unique-id=1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! queue ! nvvideoconvert ! queue name=nvvideoconvert ! nvdsosd name=on_screen_display process-mode=0 display-text=True display-bbox=True ! queue name=osd ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h265enc bitrate=4000000 ! h265parse config-interval=1 ! queue name=file_codec_parser ! qtmux ! queue name=file_muxer ! filesink location=result.mp4

NvInferServer

NOTE: Make sure to place your onnx model in the directory set in the nvinferserver configuration (config-file-path property). For the provided example (config_nvinferserver.txt) it should be located in triton_model_repo/yolov8_custom_onnx/1/yolo_obj-yolov8s_640x640-v1.onnx

Custom YoloV8 model (yolo_obj-yolov8s_640x640-v1.onnx):

gst-launch-1.0 filesrc location=traffic-video.mp4 ! queue name=filesrc ! qtdemux ! h264parse config-interval=1 ! queue name=decoder ! nvv4l2decoder  ! queue name=stream0 ! streammux.sink_0 nvstreammux name=streammux width=1920 height=1080 batch-size=1 batched-push-timeout=5000000 ! nvinferserver batch-size=1 config-file-path=config_nvinferserver.txt name=primary-nvinference-engine1 unique-id=1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! queue ! nvvideoconvert ! queue name=nvvideoconvert ! nvdsosd name=on_screen_display process-mode=0 display-text=True display-bbox=True ! queue name=osd ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h265enc bitrate=4000000 ! h265parse config-interval=1 ! queue name=file_codec_parser ! qtmux ! queue name=file_muxer ! filesink location=result.mp4

Deepstream App

deepstream-app -c deepstream_app_config.txt

NOTE: The TensorRT engine file may take a very long time to generate (sometimes more than 10 minutes).

NOTE: If you want to use YOLOv2 or YOLOv2-Tiny models, change the deepstream_app_config.txt file before run it

...
[primary-gie]
...
config-file=config_infer_primary_yoloV2.txt
...

6. Difference between the standard model configuration and the custom model

The differences in the configuration for the custom model compared to the standard model are as follows:

onnx-file: As expected, a different ONNX file is used for the custom model.
model-engine-file: If the engine hasn't been generated yet, this path is not relevant. Once the engine is generated, you can specify its name here so it can be reused in future runs.
network-input-order: This setting changes because the custom model now uses an input with NHWC layout.
batch-size: The custom model supports dynamic batch input, so this value can be different from 1. (If changed, a new engine will be generated.)
parse-bbox-func-name: Since the custom model has a different output format, a new parsing function called NvDsInferParseYoloDynamic was created specifically for it. If you're using the standard model, you should continue using the existing function by setting this field to NvDsInferParseYolo.

Docker usage

x86 platform

nvcr.io/nvidia/deepstream:7.1-gc-triton-devel
nvcr.io/nvidia/deepstream:7.1-triton-multiarch

Jetson platform

nvcr.io/nvidia/deepstream:7.1-triton-multiarch

NOTE: To compile the nvdsinfer_custom_impl_Yolo, you need to install the g++ inside the container

apt-get install build-essential

NOTE: With DeepStream 7.1, the docker containers do not package libraries necessary for certain multimedia operations like audio data parsing, CPU decode, and CPU encode. This change could affect processing certain video streams/files like mp4 that include audio track. Please run the below script inside the docker images to install additional packages that might be necessary to use all of the DeepStreamSDK features:

/opt/nvidia/deepstream/deepstream/user_additional_install.sh

NMS Configuration

To change the nms-iou-threshold, pre-cluster-threshold and topk values, modify the config_infer file

[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300

NOTE: Make sure to set cluster-mode=2 in the config_infer file.

Notes

Sometimes while running gstreamer pipeline or sample apps, user can encounter error: GLib (gthread-posix.c): Unexpected error from C library during 'pthread_setspecific': Invalid argument. Aborting.. The issue is caused because of a bug in glib 2.0-2.72 version which comes with Ubuntu 22.04 by default. The issue is addressed in glib 2.76 and its installation is required to fix the issue (https://github.com/GNOME/glib/tree/2.76.6).
- Migrate glib to newer version
```
pip3 install meson
pip3 install ninja
```
  NOTE: It is recommended to use Python virtualenv.
```
git clone https://github.com/GNOME/glib.git
cd glib
git checkout 2.76.6
meson build --prefix=/usr
ninja -C build/
cd build/
ninja install
```
- Check and confirm the newly installed glib version:
```
pkg-config --modversion glib-2.0
```
Sometimes with RTSP streams the application gets stuck on reaching EOS. This is because of an issue in rtpjitterbuffer component. To fix this issue, a script has been provided with required details to update gstrtpmanager library.
```
/opt/nvidia/deepstream/deepstream/update_rtpmanager.sh
```

Extract metadata

You can get metadata from DeepStream using Python and C/C++. For C/C++, you can edit the deepstream-app or deepstream-test codes. For Python, your can install and edit deepstream_python_apps.

Basically, you need manipulate the NvDsObjectMeta (Python / C/C++) and NvDsFrameMeta (Python / C/C++) to get the label, position, etc. of bboxes.

My projects: https://www.youtube.com/MarcosLucianoTV

Name		Name	Last commit message	Last commit date
Latest commit History 218 Commits
.github		.github
docs		docs
nvdsinfer_custom_impl_Yolo		nvdsinfer_custom_impl_Yolo
triton_model_repo/yolov8_custom_onnx		triton_model_repo/yolov8_custom_onnx
utils		utils
LICENSE.md		LICENSE.md
README.md		README.md
config_infer_primary.txt		config_infer_primary.txt
config_infer_primary_codetr.txt		config_infer_primary_codetr.txt
config_infer_primary_damoyolo.txt		config_infer_primary_damoyolo.txt
config_infer_primary_dfine.txt		config_infer_primary_dfine.txt
config_infer_primary_goldyolo.txt		config_infer_primary_goldyolo.txt
config_infer_primary_ppyoloe.txt		config_infer_primary_ppyoloe.txt
config_infer_primary_ppyoloe_plus.txt		config_infer_primary_ppyoloe_plus.txt
config_infer_primary_rtdetr.txt		config_infer_primary_rtdetr.txt
config_infer_primary_rtmdet.txt		config_infer_primary_rtmdet.txt
config_infer_primary_yolo11.txt		config_infer_primary_yolo11.txt
config_infer_primary_yoloV10.txt		config_infer_primary_yoloV10.txt
config_infer_primary_yoloV2.txt		config_infer_primary_yoloV2.txt
config_infer_primary_yoloV5.txt		config_infer_primary_yoloV5.txt
config_infer_primary_yoloV5u.txt		config_infer_primary_yoloV5u.txt
config_infer_primary_yoloV6.txt		config_infer_primary_yoloV6.txt
config_infer_primary_yoloV7.txt		config_infer_primary_yoloV7.txt
config_infer_primary_yoloV8.txt		config_infer_primary_yoloV8.txt
config_infer_primary_yoloV8_custom.txt		config_infer_primary_yoloV8_custom.txt
config_infer_primary_yoloV9.txt		config_infer_primary_yoloV9.txt
config_infer_primary_yolonas.txt		config_infer_primary_yolonas.txt
config_infer_primary_yolonas_custom.txt		config_infer_primary_yolonas_custom.txt
config_infer_primary_yolor.txt		config_infer_primary_yolor.txt
config_infer_primary_yolox.txt		config_infer_primary_yolox.txt
config_infer_primary_yolox_legacy.txt		config_infer_primary_yolox_legacy.txt
config_nvinferserver.txt		config_nvinferserver.txt
deepstream_app_config.txt		deepstream_app_config.txt
deepstream_app_config_yoloV8_custom.txt		deepstream_app_config_yoloV8_custom.txt
labels.txt		labels.txt

License

ridgerun-ai/DeepStream-Yolo

Folders and files

Latest commit

History

Repository files navigation