Implemented YoloV5 as a 3rd model for inference. #220

IuliuNovac · 2021-12-16T02:05:34Z

This is the second option for object detection. You can import you own custom model by replacing the public/yolov5 folder.

Disclaimer, I don't know JavaScript, i have no clue how this language works!!!

I just managed to get it work, since i need this feature. Expect code not to meet the guidelines and might be buggy.

Currently missing a way to switch between different type of models in AIObjectDetectionActions. I have no clue how to solve this, so i came up with an horrible temporary solution. I would like some help.
Missing the ability to import the labels name of the yolov5 models. Right now it's defined in ObjectDetectorYOLO.ts as in variable names.
private static names = ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light', 'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow'....
Need a nice way to upload the model via the browser, again, i don't know JavaScript at all, so i would need help to implement that.
Option for fine tuning/retraining while labeling would be very nice. Something as deploying to AWS sage maker for training or locally. I might be able to do that tho, but not the front end part.

Support all general variations of Yolov5, but missing a way to to chose between them.

After the release of YOLOv5 v6, users could export tfjs models with its main branch:

git clone https://github.com/ultralytics/yolov5.git
python export.py --weights yolov5s.pt --include tfjs

Pre-flight checklist

Unit tests for all non-trivial changes
Tested locally
Updated wiki

…option for object detection. You can import you own custom model in the public/yolov5 folder. Disclaimer, I don't know javascript. Expect code not to meet the guidelines and might be buggy

IuliuNovac · 2021-12-16T02:07:31Z

resolve #98 #156

SkalskiP · 2021-12-20T18:02:36Z

Hello @IuliuNovac 👋 ! And thank you very much for your interest in our project. I must admit that it is a pretty good starting point for further work, but as you yourself noticed it is not yet ready to be included in the production version of MS.

In my opinion we should add internal support for the YOLOv5 model trained on the COCO dataset, but allow users to upload their own model.
Because I would like to be able to freely change the code and at the same time I would like to keep a trace of the fact that it was you who originally added the changes I propose to change the target branch for that PR from develop to a specially created one dedicated to adding this feature. Your code would be added to this separate branch, as I would make the appropriate corrections and then your code and my code together would go into develop.

IuliuNovac · 2021-12-22T10:13:38Z

package.json

-    "@tensorflow/tfjs-backend-webgl": "^3.9.0",
-    "@tensorflow/tfjs-core": "^3.9.0",
-    "@tensorflow/tfjs-node": "^3.9.0",
+    "@tensorflow/tfjs-backend-cpu": "^3.8.0",


I noticed issues with different versions of tfjs, the 3.9.0 on other hand led to some issues. Could be that my project wasn't rebuild correctly.

IuliuNovac · 2021-12-22T10:14:08Z

src/ai/ObjectDetectorYOLO.ts

+    private static width = 640;
+    private static height = 640;
+    public static AIModel = AIModel.OBJECT_DETECTION;
+    private static names = ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light',


This is not ok, it should be imported from a separate json file.

IuliuNovac · 2021-12-22T10:16:47Z

src/ai/ObjectDetectorYOLO.ts

+            callback(predictions);
+        }
+    }
+}


Requires refactoring, adding a confidence threshold a user can select would be nice. I was training the model, but in the first iterations the confidence of identifying my object was low. Unfortunately i was not able to change the confidence coefficient.

IuliuNovac · 2021-12-22T10:18:32Z

src/data/enums/AIModel.ts

@@ -1,4 +1,5 @@
 export enum AIModel {
    OBJECT_DETECTION = "OBJECT_DETECTION",
-    POSE_DETECTION = "POSE_DETECTION"
+    POSE_DETECTION = "POSE_DETECTION",
+    OBJECT_DETECTION_YOLOv5 = "OBJECT_DETECTION_YOLOv5"
 }


Probably Object_Detection and Pose Detection should have a separate enum to chose between SSD, YOLO, etc.

IuliuNovac · 2021-12-22T10:22:15Z

src/logic/actions/AIObjectDetectionActions.ts

-                store.dispatch(updateActivePopupType(PopupWindowType.SUGGEST_LABEL_NAMES));
-            } else {
-                store.dispatch(updateActivePopupType(null));
+        switch (ObjectDetectorYolov5.AIModel) {


This is the horrible quick fix i made, too chunky. Also, i am storing the variable to switch between which model to use for prediction in the ObjectDetectorYolov5, that's horrible . I did not know how pass the value in AIModel to this object.
As a result, i introduced a bug that doesn't allow you to pick SSD after you pick YOLO.

IuliuNovac · 2021-12-22T10:35:53Z

Hi @SkalskiP , that sounds quite good. By no means someone should use this branch unless they just need to run YOLOv5 alone. I used it to train a yolov5 model. The steps where simple:

Label a small number of images
Export the datasets and train.
Convert the model to tfjs model
Copy it into the public/yolov5
Repeat, while the prediction from yolov5 are acceptable.

Note: You should not rely on the yolo model to assist for labeling, but rather evaluate the short comings. Since you can skip the images labeled right, while focusing on labeling the faulty predictions.

SkalskiP · 2021-12-22T23:46:55Z

Hi @IuliuNovac, that's exactly how I envision this feature. The only difference is that instead ofpublic/yolov5 I would like the user to be able to just load the model files into the editor. I'm already working on integration with YOLOv5 on a separate branch. I plan to divide the work into two phases:

Adding general support for YOLOv5. The user will be able to use a model trained on COCO dataset.
Allow the user to load a YOLOv5 model trained on any dataset.

I'll let you know the progress soon. You can see how I'm doing on feature/156_import_custom_model_yolov5 branch.

Implemented YoloV5 as a 3rd model for inference. Works as the second …

0bcb0b6

…option for object detection. You can import you own custom model in the public/yolov5 folder. Disclaimer, I don't know javascript. Expect code not to meet the guidelines and might be buggy

IuliuNovac changed the title ~~Implemented YoloV5 as a 3rd model for inference. resolve #156~~ Implemented YoloV5 as a 3rd model for inference. Dec 16, 2021

SkalskiP changed the base branch from develop to feature/156_import_custom_model December 20, 2021 18:07

SkalskiP marked this pull request as ready for review December 20, 2021 18:13

SkalskiP merged commit 184575c into SkalskiP:feature/156_import_custom_model Dec 20, 2021

IuliuNovac commented Dec 22, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implemented YoloV5 as a 3rd model for inference. #220

Implemented YoloV5 as a 3rd model for inference. #220

IuliuNovac commented Dec 16, 2021 •

edited

Loading

IuliuNovac commented Dec 16, 2021 •

edited

Loading

SkalskiP commented Dec 20, 2021

IuliuNovac Dec 22, 2021

IuliuNovac Dec 22, 2021

IuliuNovac Dec 22, 2021

IuliuNovac Dec 22, 2021

IuliuNovac Dec 22, 2021 •

edited

Loading

IuliuNovac commented Dec 22, 2021 •

edited

Loading

SkalskiP commented Dec 22, 2021 •

edited

Loading

Implemented YoloV5 as a 3rd model for inference. #220

Implemented YoloV5 as a 3rd model for inference. #220

Conversation

IuliuNovac commented Dec 16, 2021 • edited Loading

Disclaimer, I don't know JavaScript, i have no clue how this language works!!!

Pre-flight checklist

IuliuNovac commented Dec 16, 2021 • edited Loading

SkalskiP commented Dec 20, 2021

IuliuNovac Dec 22, 2021

Choose a reason for hiding this comment

IuliuNovac Dec 22, 2021

Choose a reason for hiding this comment

IuliuNovac Dec 22, 2021

Choose a reason for hiding this comment

IuliuNovac Dec 22, 2021

Choose a reason for hiding this comment

IuliuNovac Dec 22, 2021 • edited Loading

Choose a reason for hiding this comment

IuliuNovac commented Dec 22, 2021 • edited Loading

SkalskiP commented Dec 22, 2021 • edited Loading

IuliuNovac commented Dec 16, 2021 •

edited

Loading

IuliuNovac commented Dec 16, 2021 •

edited

Loading

IuliuNovac Dec 22, 2021 •

edited

Loading

IuliuNovac commented Dec 22, 2021 •

edited

Loading

SkalskiP commented Dec 22, 2021 •

edited

Loading