TensorRT deployment Support, Pre-Processing alignment and more

This release provides TensorRT deployment support both for Python and C++ interface and supports pre-processing alignment with the official yolov5.

Highlights

Use yolort to deploy YOLOv5 on TensorRT

Unlike other pipelines that deal with YOLOv5 on TensorRT, we embed the whole post-processing into the serialized engine with TensorRT's EfficientNMS_TRT plugin. This pipeline could simplify deployment and gain in speed due to TensorRT's efficient implementation of this plugin. Check out this tutorial detailing yolort's model conversion and use cases. Here is a Python pipeline to deploy YOLOv5 on TensorRT.

import torch
from yolort.runtime import PredictorTRT

# Load the serialized TensorRT engine
engine_path = "yolov5n6.engine"
device = torch.device("cuda")
y_runtime = PredictorTRT(engine_path, device=device)

# Perform inference on an image file
predictions = y_runtime.predict("bus.jpg")

• Support exporting yolort to TensorRT serialized engine (#238, #242, #243, #252, #254, #263, #275, #288, #311, #312, #326)
• Add TensorRT Python inference pipeline (#251, #253, #256, #259, #283, #290, #307, #309)
• Add TensorRT C++ inference example (#245, #257, #258, #261, #262, #305, #321, #323, #324)

Alignment with the official yolov5 Pre-Processing

Pre-processing misalignment has been an obstacle to getting yolort to land, and we've fixed that in this release.

• Fix the consistency of pre-processing with yolov5 (#293)
• Support fixed shape inference in pre-processing (#301)
• Fix pre-processing for TVM VM inference (#302)

Backwards Incompatible Changes

• Rename relaying to relay (#313)
• Move Lightning wrapper into trainer (#314)
• Deprecate the methods run_on_image function in PredictorORT (#327)

Improvement

• Support loading the latest version of yolov5 (#235)
• Formal documentation support (#224, #225, #292)
• Cleanup AnchorGenerator and PostProcess (#203)
• Revert SPPF to SPP for downstream compatible (#240)
• Support custom image size when exporting ONNX models (#250, #264, #269)
• Upgrade with latest PyTorch version in GH Actions (#255)
• Update PyTorch to 1.10 in TVM relay VM tutorial (#287)
• Enhance PredictorORT to adapt to more scenarios (#319, #320, #327)
• Support publishing 🐍 to PyPI.org 📦 with GH Actions (#229, #241, #306, #328, #329)

Bugfixes

• Small fixes to CONTRIBUTING guide (#218)
• Fix type casting in ONNX Runtime C++ example (#221)
• Fix TorchScript exporting for custom checkpoints (#267)
• Fix non-agnostic parameter in tutorial (#284)
• Fix a bug and document RandomZoomOut (#289)
• Compatible with Windows for LibTorch deployment (#303)

Documentation improvements

• Add comparison docs for yolov5 & yolort (#300)
• Update Docs and Readme (#223, #299, #304, #308, #315)
• Add Fidan and Shiquan as co-authors for great contributions (#231, #258)

Code Quality

• Update to .clang-format (#222)
• Enable pip cache dependencies in GH Actions (#226)
• Move CONTRIBUTING and CODE_OF_CONDUCT into .github (#228)
• Cleanup unit-test (#246)
• Fixed some spelling/typos (#276)
• Set OpenCV as optional (#286)
• [pre-commit.ci] pre-commit autoupdate (#244, #249, #260, #270)
• Install yolort in GH Actions (#294)
• Replaced to_tensor() with pil_to_tensor() + convert_image_dtype() (#298)
• Remove CLI tools for detection and training temporarily (#325)

Contributors

We're grateful for our community, which helps us improving yolort by submitting issues and PRs, and providing feedback and suggestions. The following persons have contributed patches for this release:

@datumbox @lzmisscc @mattpopovich @ncnnnnn @ShiquanYu @triple-Mu @yang-gis @zhiqwang @2428513107

Special thanks to @ShiquanYu for his great contributions on TensorRT C++ example!