Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add object detection training docs #1435

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Add object detection training docs #1435

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
527a16c
Add object detection training docs
Alex-idk Sep 27, 2024
93cdcf9
change object detection docs to just converting a model
Alex-idk Oct 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 34 additions & 2 deletions docs/source/docs/objectDetection/about-object-detection.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ PhotonVision supports object detection using neural network accelerator hardware
For the 2024 season, PhotonVision ships with a **pre-trained NOTE detector** (shown above), as well as a mechanism for swapping in custom models. Future development will focus on enabling lower friction management of multiple custom models.

```{image} images/notes-ui.png

```

## Tracking Objects
Expand All @@ -32,9 +33,40 @@ Compared to other pipelines, object detection exposes very few tuning handles. T

The same area, aspect ratio, and target orientation/sort parameters from {ref}`reflective pipelines <docs/reflectiveAndShape/contour-filtering:Reflective>` are also exposed in the object detection card.

## Training Custom Models
## Converting a custom model

After training your own YOLOv5s model, you can convert it to an RKNN file for use with PhotonVision. If you have not already trained a model, you can follow the official YOLOv5 training guide [here](https://docs.ultralytics.com/yolov5/tutorials/train_custom_data/)

### Step 1: Exporting the Model to ONNX

Using your trained YOLOv5s model, you will need to export it to ONNX format using airockchip's YOLOv5 fork.

First, download airockchip's YOLOv5 fork along with the onnx to rknn conversion script.

```bash
git clone https://github.com/airockchip/yolov5.git airockchip-yolov5
curl -O link-to-onnx2rknn.py
```

#### Export Command

```bash
cd airockchip-yolov5 && python export.py --weights '/path/to/best.pt' --rknpu --include 'onnx'
```

### Step 2: Converting ONNX to RKNN

Using the `onnx2rknn.py` script, convert the ONNX model to an RKNN file. This script was downloaded in a previous step.

#### Conversion Command

Run the script, passing in the ONNX model and a folder containing images from your dataset:

```bash
python onnx2rknn.py /path/to/best.onnx /path/to/export/best.rknn /path/to/dataset/valid/images
```

Coming soon!
If you have any questions about the conversion process, ask in the PhotonVision Discord server.

## Uploading Custom Models

Expand Down
Loading