YOLOv8
using TensorRT accelerate !
-
Install TensorRT follow
TensorRT official website
-
Install python requirement.
pip install -r requirement.txt
-
(optional) Install
ultralytics
package for ONNX export or TensorRT API building.pip install ultralytics
You can download pretrained pytorch model by:
wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n.pt wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8s.pt wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8m.pt wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8l.pt wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8x.pt
You can export ONNX or Engine using the origin ultralytics
repo .
Please see more information in Normal.md
.
You can export your onnx model by ultralytics
API
and add postprocess into model at the same time.
python3 export.py \
--weights yolov8s.pt \
--iou-thres 0.65 \
--conf-thres 0.25 \
--topk 100 \
--opset 11 \
--sim \
--input-shape 1 3 640 640 \
--device cuda:0
--weights
: The PyTorch model you trained.--iou-thres
: IOU threshold for NMS plugin.--conf-thres
: Confidence threshold for NMS plugin.--topk
: Max number of detection bboxes.--opset
: ONNX opset version, default is 11.--sim
: Whether to simplify your onnx model.--input-shape
: Input shape for you model, should be 4 dimensions.--device
: The CUDA deivce you export engine .
You will get an onnx model whose prefix is the same as input weights.
If you just want to taste first, you can download the onnx model which are exported by YOLOv8
package and modified by me.
You can export TensorRT engine from ONNX by build.py
.
Usage:
python3 build.py \
--weights yolov8s.onnx \
--iou-thres 0.65 \
--conf-thres 0.25 \
--topk 100 \
--fp16 \
--device cuda:0
--weights
: The ONNX model you download.--iou-thres
: IOU threshold for NMS plugin.--conf-thres
: Confidence threshold for NMS plugin.--topk
: Max number of detection bboxes.--fp16
: Whether to export half-precision engine.--device
: The CUDA deivce you export engine .
You can modify iou-thres
conf-thres
topk
by yourself.
You can export TensorRT engine by trtexec
tools.
Usage:
/usr/src/tensorrt/bin/trtexec \
--onnx=yolov8s.onnx \
--saveEngine=yolov8s.engine \
--fp16
If you installed TensorRT by a debian package, then the installation path of trtexec
is /usr/src/tensorrt/bin/trtexec
If you installed TensorRT by a tar package, then the installation path of trtexec
is under the bin
folder in the path you decompressed
Please see more information in API-Build.md
Notice !!! We don't support YOLOv8-seg model now !!!
You can infer images with the engine by infer.py
.
Usage:
python3 infer.py \
--engine yolov8s.engine \
--imgs data \
--show \
--out-dir outputs \
--device cuda:0
--engine
: The Engine you export.--imgs
: The images path you want to detect.--show
: Whether to show detection results.--seg
: Whether to infer with segment model.--out-dir
: Where to save detection results images. It will not work when use--show
flag.--device
: The CUDA deivce you use.--profile
: Profile the TensorRT engine.
You can infer with c++ in csrc/detect/end2end
.
Please set you own librarys in CMakeLists.txt
and modify CLASS_NAMES
and COLORS
in main.cpp
.
export root=${PWD}
cd src/detect/end2end
mkdir build
cmake ..
make
mv yolov8 ${root}
cd ${root}
Usage:
# infer image
./yolov8 yolov8s.engine data/bus.jpg
# infer images
./yolov8 yolov8s.engine data
# infer video
./yolov8 yolov8s.engine data/test.mp4 # the video path
Please see more information in Segment.md
See more in README.md
If you want to profile the TensorRT engine:
Usage:
python3 infer.py --engine yolov8s.engine --profile
If you need to break away from pytorch and use tensorrt inference,
you can get more information in infer-no-torch.py
,
the usage is the same as the pytorch version, but its performance is much worse.
You can use cuda-python
or pycuda
for inference.
Please install by such command:
pip install cuda-python
# or
pip install pycuda
Usage:
python3 infer-no-torch.py \
--engine yolov8s.engine \
--imgs data \
--show \
--out-dir outputs \
--method cudart
--engine
: The Engine you export.--imgs
: The images path you want to detect.--show
: Whether to show detection results.--seg
: Whether to infer with segment model.--out-dir
: Where to save detection results images. It will not work when use--show
flag.--method
: Choosecudart
orpycuda
, default iscudart
.--profile
: Profile the TensorRT engine.