What is the future of this repository?

[AndreaPi]

I saw these scary news:

https://github.com/fizyr/keras-retinanet#warning-deprecated

What does this mean?

1. are you closing the repository to any new issues & PR? This doesn't seem the case, I can still see new commits.
2. are you planning to stop to add new features, but you will continue to support external PR, and help resolving new issues?
3. something else?

Can I also ask the reason why? Internal reasons you cannot disclose, issues with new tf.keras, you just got bored 😉 etc.?

Also, you state that this repository is deprecated in favor of torchvision. Does this mean that you plan to add new features to the RetinaNet implementation of torchvision, or is it just meant as a general suggestion, i.e. "we decided to stop working on keras_retinanet to embark on new endeavors. If you're looking for a RetinaNet implementation that will keep being updated, we saw torchvision and we're pleased with what we saw".

Last but not least, I want to say an heartfelt "thank you!" for this repository. It made a big difference for my project: I gave back in my own little way with a few PRs based on my fork of keras_retinanet, and I may continue to do so for a little while, if you choose option 1 🙂
Best of luck with your new projects!

[mlampros]

I think that it's related with the following PR
pytorch/vision#1697

[hgaiser]

are you closing the repository to any new issues & PR? This doesn't seem the case, I can still see new commits.

For the time being we still accept PRs and issues like before. There are no plans to change this, but I also can't make promises this will stay the same. What we're trying to say with the deprecation is that we are not using it internally anymore, meaning some of the support from us using it disappears. Concrete example, with new releases of tensorflow we would automatically test this project and fix any issues we encounter, that won't be the case anymore. However many people are using this project and that gives it viability. Bugs and issues should be fixed by the community now, so fixes are of course still welcome.

are you planning to stop to add new features, but you will continue to support external PR, and help resolving new issues?

This is covered by my earlier answer I think, but indeed we won't be adding new features, but we support external PRs. Issue resolving will mostly be done by the community as well.

Can I also ask the reason why? Internal reasons you cannot disclose, issues with new tf.keras, you just got bored wink etc.?

You might have seen we were working on a pure tensorflow port of keras-retinanet, but there were simply too many issues to make it functional (most of it unexplained issues in tensorflow, most noticeably difference between eager execution and non-eager execution; these should surely return the same results right?) Every release of tensorflow seemed to fix one thing but break two new things. Support for fixing these issues was not always great: tensorflow/tensorflow#33787 .

In addition, Keras sometimes makes it a bit complicated to write the model you want to write. A very concrete example is the loss function which only takes one predicted tensor and one generated output as input. In more complicated models you might want the loss to depend on multiple predicted values or multiple predicted generated values, but this isn't possible. A solution is to concatenate your tensors into one tensor, but that is both inconvenient and sometimes just doesn't work (https://github.com/fizyr/keras-maskrcnn/blob/master/keras_maskrcnn/preprocessing/generator.py#L243-L249). Maybe there are other workarounds that I haven't seen yet or maybe there are new features that allow this, but at the time of writing the code there seemed to be no other way to get the desired approach. I suppose it's possible in tensorflow, but I never got around to testing that.

Because of an unrelated reason (running a network using the Rust language) I was looking into pytorch and found torchvision. This had better Rust support and was a nice excuse to try a different framework. In my personal opinion, pytorch is more stable and has a clearer API. So far we haven't really encountered some of those strange bugs that we encountered one too many times in tensorflow. We are switching to pytorch internally for these reasons.

Also, you state that this repository is deprecated in favor of torchvision. Does this mean that you plan to add new features to the RetinaNet implementation of torchvision, or is it just meant as a general suggestion, i.e. "we decided to stop working on keras_retinanet to embark on new endeavors. If you're looking for a RetinaNet implementation that will keep being updated, we saw torchvision and we're pleased with what we saw".

Torchvision doesn't have a retinanet implementation (yet). Hopefully pytorch/vision#1697 will be merged soon. It is unlikely to have the same exact features as keras-retinanet, but the core is there and in my opinion better than in keras-retinanet. A nice example is training on COCO. The implementation in torchvision is basically the same as keras-retinanet, but for some reason the mAP in pytorch is 0.357, which is even better than the reported 0.354 of the paper. keras-retinanet would get an mAP of 0.20 if you're lucky with the same amount of training (I still haven't managed to reproduce the results in the README after many tries, maybe it was just lucky?). The exact reason for this difference will probably remain unclear, but it confirms my preferences.

Last but not least, I want to say an heartfelt "thank you!" for this repository. It made a big difference for my project: I gave back in my own little way with a few PRs based on my fork of keras_retinanet, and I may continue to do so for a little while, if you choose option 1 slightly_smiling_face
Best of luck with your new projects!

Thank you, this is very much appreciated! Likewise I would like to thank you and everyone that is contributing or using this project. It has been very beneficial for us too in multiple ways and I hope we can keep doing projects like this. For me it is a good motivation to make this project the best it could be so that it is useful for as many people as possible and I enjoy seeing all the projects that spawned from it.

[AndreaPi]

Thanks for the answers! I hope your PR to torchvision gets merged soon 😀

[hgaiser]

For the record, pytorch/vision#2784 is now the PR for retinanet in pytorch :).

[guillaumefrd]

@hgaiser Out of curiosity, is there a reason why you chose to migrate to torchvision and not mmdetection?
Thanks for all your work on this awesome repo!

[hgaiser]

@hgaiser Out of curiosity, is there a reason why you chose to migrate to torchvision and not mmdetection?
Thanks for all your work on this awesome repo!

I had looked into pytorch implementation of retinanet and didn't find mmdetection at that time, so thanks for bringing it to my attention; it looks very extensive! I did find torchvision and liked their setup so I chose that.

[bw4sz]

@hgaiser I think this might be the end for this repo, unless we want to try to talk to maintainers upstream.

reported by one of my users:
weecology/DeepForest#192 (comment)

this is why we all moved to pytorch. Thanks for all your work in this space, probably time to let this repo go? I could open an issue on keras-resnet or kapre, but I'm sure they have functionality reasons and we knew it wouldn't last forever.

https://colab.research.google.com/drive/1-J1eJux-GDVAWHvG0Aiwsmon6S7daHKR?usp=sharing

pip install keras-resnet==0.2.0 tensorflow==1.14.0
...
ERROR: kapre 0.3.5 has requirement tensorflow>=2.0.0, but you'll have tensorflow 1.14.0 which is incompatible.

import keras_resnet
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-3-93308a00f642> in <module>()
----> 1 import keras_resnet

5 frames
/usr/local/lib/python3.7/dist-packages/keras/backend.py in <module>()
     35 from tensorflow.python.distribute import distribute_coordinator as dc
     36 from tensorflow.python.distribute import distribute_coordinator_context as dc_context
---> 37 from tensorflow.python.eager.context import get_config
     38 from tensorflow.python.framework import config
     39 from keras import backend_config

ImportError: cannot import name 'get_config' from 'tensorflow.python.eager.context' (/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/context.py)

[hgaiser]

I haven't looked into this, but theoretically this repository should still work with tensorflow 2.0.0? I haven't used it since tf 2.2 I think, so can't say much for newer versions, but 2.2 definitely still worked.

[Egorundel]

@hgaiser Hi! How to use RetinaNet in PyTorch implementation? I mean, how do I start training? Do you have any guide?

Is this the right guide?
https://github.com/pytorch/vision/tree/f69eee6108cd047ac8b62a2992244e9ab3c105e1/references/detection

Thank you in advance for your answer!

[hgaiser]

@hgaiser Hi! How to use RetinaNet in PyTorch implementation? I mean, how do I start training? Do you have any guide?

Is this the right guide? https://github.com/pytorch/vision/tree/f69eee6108cd047ac8b62a2992244e9ab3c105e1/references/detection

Thank you in advance for your answer!

You could follow this guide, you just need to replace FasterRCNN for RetinaNet.

[mvizosopatino]

It would be great if someone for once in Github, instead of writing tones of lines containing useless information, write a step-by-step guide to do the installation and testing of the tools uploaded. E.g., which is the python version to be used with keras-retinanet? why is not mentioned "labelImg"? how do you get the XML files to provide the annotations? what does it mean "This repository is deprecated in favor of the [torchvision] module"? what is the MC COCO dataset? Please, simple but comprehensive step-by-step guidelines! There are tones of github tools that are impossible to run because the lack of good guidelines!

[hgaiser]

Like I mentioned in my earlier comment, this repository is no longer maintained. My advice is to use torchvision, for which you can follow this guide.