-
Notifications
You must be signed in to change notification settings - Fork 9.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add faster-coco-eval metric #11776
base: main
Are you sure you want to change the base?
Add faster-coco-eval metric #11776
Conversation
@BIGWangYuDong Hello! I'm waiting for you Yudong Wang |
Good library to speed up calculations |
LGTM, kindly ping @hhaAndroid have a look |
Hello @hhaAndroid, I see that you are active on the site, please take a look at this PR |
@RangiLyu Maybe you’re here and the project hasn’t been abandoned yet? |
@@ -8,6 +8,8 @@ | |||
|
|||
import numpy as np | |||
import torch | |||
from faster_coco_eval import COCO as FasterCOCO |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's best to make this dependency optional, to prevent affecting the previous usage pattern.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
requirements/runtime.txt
Outdated
@@ -1,3 +1,4 @@ | |||
faster-coco-eval |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's best to make this dependency optional, to prevent affecting the previous usage pattern.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
@MiXaiLL76 Thank you for your support |
Thanks for a good framework! |
@hhaAndroid Hi! Can you update your status? |
Co-authored-by: BigDong <yudongwang1226@gmail.com>
@hhaAndroid @ZwwWayne Is anyone else working on the project? Maybe we can take it on as support? |
Motivation
Faster COCO Eval
https://github.com/MiXaiLL76/faster_coco_eval
This package wraps a facebook C++ implementation of COCO-eval operations found in the pycocotools package. This implementation greatly speeds up the evaluation time for coco's AP metrics, especially when dealing with a high number of instances in an image.
Modification
I added a variable (use_faster_coco_eval) to the CocoMetric class that allows you to select an additional type of validation, namely COCOeval_faster
Use cases (Optional)
For our use case with a test dataset of 5000 images from the coco val dataset.
Testing was carried out using the mmdetection framework and the eval_metric.py script. The indicators are presented below.
Visualization of testing colab_example.ipynb available in directory examples/comparison
colab_example.ipynb in google collab
Tested with rtmdet model bbox + segm
Summary for 5000 imgs
Checklist