TensorRT Windows Compatibility Patch Release

This release is mainly for the compatibility of TensorRT C++ example (#389) and Python interface (#396) on Windows systems. In addition, this release introduces module helpers named is_module_available and requires_module to power the lazy inits of third-party libraries (#366, #367).

Besides, we:

• Fix circular reference of load_from_ultralytics (#369, #373, #374)
• Add PyYAML to requirements (#363)
• Add CLI tool for converting YOLO-format to MSCOCO (#375)
• Upgrade dependencies (#372, #386, #398)
• Update badges and fix docs (#388, #391)

We're grateful for our community, which helps us improving yolort by submitting issues and PRs, and providing feedback and suggestions. The following persons have contributed patches for this release:

@IamNaQi @mattpopovich @Yoh-Z @zhiqwang

Compatibility updates for PyTorch 1.11.0 and AutoAnchor supports

This release is mainly compatibility updates for PyTorch 1.11.0 (#359) and to support loading the checkpoints trained with AutoAnchor mechanism in YOLOv5 (#285, #352).

Besides, we

• Move bias initializations from private methods to constructors (#351)
• Upgrade PyTorch minimal version to 1.8.0 (#359)
• Added support for Python 3.10 (only works with PyTorch 1.11) (#359)
• Remove lightning strong dependencies (#355, #358)
• Update tutorials with Visualizer (#354)
• Small grammar and spelling fixes (#357)

We're grateful for our community, which helps us improving yolort by submitting issues and PRs, and providing feedback and suggestions. The following persons have contributed patches for this release:

@mattpopovich @zhiqwang

Full Changelog: v0.6.1...v0.6.2

Connecting with TorchVision's model banks

This release adds an example to showcase how to use the model banks provided by TorchVision to build a new YOLOv5 model, introduces Visualizer to display the predictions and fixes an incompatible bug in YOLOv5 Detect Layer.

Highlights

Build a new YOLOv5 model via TorchVision's model banks

We add an example to showcase how to construct a YOLOv5-Lite model with TorchVision's pre-trained MobileNetV3-Small FPN as the backbone.

import torch
from yolort.models.yolo_lite import yolov5_mobilenet_v3_small_fpn

model = yolov5_mobilenet_v3_small_fpn()
model = model.eval()

images = torch.rand(4, 3, 640, 640)
out = model(images)

• Use C3 following the model specification of YOLOv5 (#343)
• Construct YOLOv5 models with TorchVision MobileNetV3 backbone (#342)

Introduce a new visualization interface

from torchvision.io import read_image
from yolort.models import yolov5n6
from yolort.utils import Visualizer

model = yolov5n6(pretrained=True, score_thresh=0.45)
model = model.eval()
img_path = "bus.jpg"
preds = model.predict(img_path)

metalabels_path = "coco.names"
image = read_image(img_path)
v = Visualizer(image, metalabels=metalabels_path)
v.draw_instance_predictions(preds[0])
v.imshow(scale=0.5)

• Add Visualizer for visualization (#341)

Bugfixes

• Fix YOLOv5 AnchorGenerator compatibility (#345)
• Add missing version param in export_tensorrt (#335)

Code Quality

• Update Visualizer in docs and PR Recommendations (#346)
• Bump versions on GH Action (#344, #338, #337)

Contributors

We're grateful for our community, which helps us improving yolort by submitting issues and PRs, and providing feedback and suggestions. The following persons have contributed patches for this release:

@Luwill6, @zhiqwang

Full Changelog: v0.6.0...v0.6.1

TensorRT deployment Support, Pre-Processing alignment and more

This release provides TensorRT deployment support both for Python and C++ interface and supports pre-processing alignment with the official yolov5.

Highlights

Use yolort to deploy YOLOv5 on TensorRT

Unlike other pipelines that deal with YOLOv5 on TensorRT, we embed the whole post-processing into the serialized engine with TensorRT's EfficientNMS_TRT plugin. This pipeline could simplify deployment and gain in speed due to TensorRT's efficient implementation of this plugin. Check out this tutorial detailing yolort's model conversion and use cases. Here is a Python pipeline to deploy YOLOv5 on TensorRT.

import torch
from yolort.runtime import PredictorTRT

# Load the serialized TensorRT engine
engine_path = "yolov5n6.engine"
device = torch.device("cuda")
y_runtime = PredictorTRT(engine_path, device=device)

# Perform inference on an image file
predictions = y_runtime.predict("bus.jpg")

• Support exporting yolort to TensorRT serialized engine (#238, #242, #243, #252, #254, #263, #275, #288, #311, #312, #326)
• Add TensorRT Python inference pipeline (#251, #253, #256, #259, #283, #290, #307, #309)
• Add TensorRT C++ inference example (#245, #257, #258, #261, #262, #305, #321, #323, #324)

Alignment with the official yolov5 Pre-Processing

Pre-processing misalignment has been an obstacle to getting yolort to land, and we've fixed that in this release.

• Fix the consistency of pre-processing with yolov5 (#293)
• Support fixed shape inference in pre-processing (#301)
• Fix pre-processing for TVM VM inference (#302)

Backwards Incompatible Changes

• Rename relaying to relay (#313)
• Move Lightning wrapper into trainer (#314)
• Deprecate the methods run_on_image function in PredictorORT (#327)

Improvement

• Support loading the latest version of yolov5 (#235)
• Formal documentation support (#224, #225, #292)
• Cleanup AnchorGenerator and PostProcess (#203)
• Revert SPPF to SPP for downstream compatible (#240)
• Support custom image size when exporting ONNX models (#250, #264, #269)
• Upgrade with latest PyTorch version in GH Actions (#255)
• Update PyTorch to 1.10 in TVM relay VM tutorial (#287)
• Enhance PredictorORT to adapt to more scenarios (#319, #320, #327)
• Support publishing 🐍 to PyPI.org 📦 with GH Actions (#229, #241, #306, #328, #329)

Bugfixes

• Small fixes to CONTRIBUTING guide (#218)
• Fix type casting in ONNX Runtime C++ example (#221)
• Fix TorchScript exporting for custom checkpoints (#267)
• Fix non-agnostic parameter in tutorial (#284)
• Fix a bug and document RandomZoomOut (#289)
• Compatible with Windows for LibTorch deployment (#303)

Documentation improvements

• Add comparison docs for yolov5 & yolort (#300)
• Update Docs and Readme (#223, #299, #304, #308, #315)
• Add Fidan and Shiquan as co-authors for great contributions (#231, #258)

Code Quality

• Update to .clang-format (#222)
• Enable pip cache dependencies in GH Actions (#226)
• Move CONTRIBUTING and CODE_OF_CONDUCT into .github (#228)
• Cleanup unit-test (#246)
• Fixed some spelling/typos (#276)
• Set OpenCV as optional (#286)
• [pre-commit.ci] pre-commit autoupdate (#244, #249, #260, #270)
• Install yolort in GH Actions (#294)
• Replaced to_tensor() with pil_to_tensor() + convert_image_dtype() (#298)
• Remove CLI tools for detection and training temporarily (#325)

Contributors

We're grateful for our community, which helps us improving yolort by submitting issues and PRs, and providing feedback and suggestions. The following persons have contributed patches for this release:

@datumbox @lzmisscc @mattpopovich @ncnnnnn @ShiquanYu @triple-Mu @yang-gis @zhiqwang @2428513107

Special thanks to @ShiquanYu for his great contributions on TensorRT C++ example!

Support upstream yolov5 v6.0 models

This release mainly supports upstream YOLOv5 v6.0 models.

Highlights

We set the model to r6.0 by default, and rest of the usage interface is the same as before.

from yolort.models import yolov5n6

# Load model
model = yolov5n6(upstream_version="r6.0", pretrained=True, score_thresh=0.45)
model.eval()

# Perform inference on an image file
predictions = model.predict("bus.jpg")
# Perform inference on a list of image files
predictions = model.predict(["bus.jpg", "zidane.jpg"])

New Features

• Fully support upstream YOLOv5 v6.0 models (#194, #195, #196, #199, #204, #206, #213, #215) Thanks @deepage
• Support PyTorch 1.10 (#207)
• Support skipping preprocess when exporting ONNX (#193)

Backwards Incompatible Changes

• Set main as the base branch (#200)
• Rename _yolov5_darknet_pan to build_model (#198)

Improvement

• Use torch.jit.trace in unit-test for easier use in downstream (#205) Thanks @nihui
• Update model graph visualization images (#212)
• [pre-commit.ci] pre-commit autoupdate (#191, #197)

Bugfixes

• Fix non-renderable images on PyPI (#209, #210)
• Fix the anchor configuration mechanism (#201, #202)
• Small fixes to the docstrings (#208)

Feature teaser and model checkpoints for r5.0

Last modified date: 2021-10-26

Leave a block of space to store the model checkpoints.

NOTE: All checkpoints here make use of sha256 hash.

Model Checkpoints for r6.0

Last modified date: 2021-10-24

Just leave a block of space to store the model checkpoints.

NOTE: All checkpoints here make use of sha256 hash.

Dependency fix release

This release fixes the dependencies
No functional changes.

Introduce yolort.runtime Interface

This release improves support of ONNXRuntime by introducing the yolort.runtime interface and adding a C++ interface example. It also improves support for loading the checkpoint trained with ultralytics/yolov5.

Highlights

Intruduce the yolort.runtime Interface

We introduce the yolort.runtime interface to accelerate the application on the Python frontend. For example, you can define the runtime and do inferencing with ONNXRuntime as follows

from yolort.runtime import PredictorORT

detector = PredictorORT("best.onnx")
img_path = "bus.jpg"
scores, class_ids, boxes = detector.run_on_image(img_path)

ONNXRuntime C++ Inference Example

We adopt the dynamic shape mechanism when exporting the ONNX model, and within this, we can embed both pre-processing (letterbox) and post-processing (nms) into the model graph, which simplifies the deployment strategy. And @itsnine have provided an example on how to use it in here.

Load Checkpoint from Official YOLOv5

We now provide an interface for loading the checkpoint weights trained from the official yolov5 directly.

from yolort.models import YOLOv5

# 'yolov5s.pt' is downloaded from https://github.com/ultralytics/yolov5/releases/download/v5.0/yolov5s.pt
ckpt_path_from_ultralytics = "yolov5s.pt"
model = YOLOv5.load_from_yolov5(ckpt_path_from_ultralytics, score_thresh=0.25)

model.eval()
img_path = "test/assets/bus.jpg"
predictions = model.predict(img_path)

Backwards Incompatible Changes

• Move common utils to yolort.v5 (#167)
• Rename the YOLOModule to YOLOv5 (#179)
• Move graph visualization tools to yolort.relaying (#165)

New Features

• [PROTOTYPE] Add ncnn deployment example (#145)
• [PROTOTYPE] Train models from scratch (#115, #116, #124, #125, #128, #143)
• Support loading checkpoint from official yolov5 (#167, #168, #169, #179)
• Add ONNXRuntime C++ inference example (#171, #172, #174, #186), thanks @itsnine and @xiguadong
• Introduce yolort.runtime interface (#176, #178)
• Add pre-commit.ci hooks (#182)
• Add CLI tool for exporting ONNX models (#175), thanks @runrunrun1994
• Add CLI tool for evaluating mAP metrics (#148, #150)

Improvement

• Add alignment with ultralytics yolov5 tutorial (#117, #118, #121), thanks @xiguadong
• Refactor graph visualization tools (#126, #127, #165), thanks @xiguadong
• Update TorchVision C++ interfaces to 0.9.0+ (#136, #138), thanks @xiguadong
• Replace cxxopts with cmdline (#137)
• Update dependencies (#161, #162, #163, #164, #181)
• Simplify the unit-test (#149, #151, #152)
• Support autoenvolved anchors from ultralytics (#120), thanks @Tomakko
• Workaround on YOLOTransform to support fixed size input (#109)
• Unify onnx and JIT resize implementations (#105)

Bugfixes

• Fix exception (#183)
• Fix dtype in AnchorGenerator (#111, #113, #166), thanks @dkloving
• Made targets argument in YOLOTransform truly optional (#108), thanks @dkloving

Documents

• Update CONTRIBUTING.md (#185)
• Update badges and ONNXRuntime usage (#184)
• New issue templates (#173, #180)
• Fix docstring (#110, #158)
• Added more descriptive error logging (#140), thanks @mattpopovich
• Add yolort logo (#106)

Add mAP metrics and cleanup data pipeline

This release introduces mAP metrics for COCO datasets and refactor of the data pipeline to make it more clear for users.

Add COCOEvaluator and cleanup data pipeline

• Cleanup and refactor data pipeline (#103, #88)
• Fix unittest for coco evaluator (#91)
• Add mAP Metrics and validation methods in tasks (#93, #89)

Backwards Incompatible Changes

• Rename YoloHead to YOLOHead (#82)
• Rename GeneralizedYOLOTransform to YOLOTransform (#103)

New Features

• Keep consistency of transform (#102)
• Add Transformer (TAN) based model structure (#75, #81)
• Support training with vanilla module (#87)
• Add Docs with Sphinx (#85)
• Decoupling path dependence in notebooks (#84)
• Allow loading of user-trained ultralytics models (#79) , thanks @dkloving
• Add PyTorch 1.8 to CI (#76)

Bug Fixes

• Bug fixes in read_image_to_tensor (#97) , thanks @stereomatchingkiss

Documents

• Add comments about requirements (#96)
• Replaced print statements with logging library (#95) , thanks @itsabhianant
• Update badges (#83)
• Add instructions of minimial PyTorch version (#74)
• Rename parameters to keep consistency (#82)