RFC: Creating SIG Models #314
Conversation
jaeyounkim
requested review from
ematejska,
ewilderj,
martinwicke and
theadactyl
as code owners
|
I would like to be part of the SIG.Thanks |
There was a problem hiding this comment.
One thing that is missing from this RFC is a set of guidelines about quality of models in the repo.
SIG-Addons has a good workflow to use as reference. There needs to be a clear statement on quality of the models in the repo and keeping up with maintenance and evolving best-practices. A big risk for a SIG like this is drive-by models that bitrot over time. There must be a process to remove models if they are not up to the standards.
rfcs/20201023-sig-models.md
Outdated
| and maintain the code quality through the review process. Each group has the | ||
| flexibility to operate differently. | ||
|
|
||
| SIG is also a place for community discussions and sharing best practices of |
There was a problem hiding this comment.
@perfinion I've changed it to SIG Models. Thank you!
rfcs/20201023-sig-models.md
Outdated
| ([models@tensorflow.org](https://groups.google.com/a/tensorflow.org/forum/#!forum/models)), | ||
| arrange a regular (e.g., monthly) video conference call rotated by the SIG | ||
| leads. We will also use a Gitter chat channel | ||
| (https://gitter.im/tensorflow/models) for discussions. |
There was a problem hiding this comment.
This should be gitter.im/tensorflow/sig-models for consistency with the other sig channels, the existing ones are eg gitter.im/tensorflow/sig-build or gitter.im/tensorflow/sig-addons
There was a problem hiding this comment.
@perfinion I've changed it. Thank you for your comment.
| the contest | ||
| * Review initial submissions and provide more support for selected high-quality | ||
| submissions | ||
| * Review final submissions and announce results |
There was a problem hiding this comment.
what is the lifecycle of such models? I can see at least: draft, initial, final
Any guarantees of maintainability? Can models be deprecated over time?
There was a problem hiding this comment.
We have already something about this in SIG-addons in the approved RFC https://github.com/tensorflow/community/blob/master/rfcs/20190308-addons-proxy-maintainership.md#repository-growth-and-review /cc @seanpmorgan
There was a problem hiding this comment.
We are also trying to figure if a bot could do some keep-alive/MIA checks tensorflow/addons#2024
There was a problem hiding this comment.
The real question is also how to implement something like tensorflow/addons#236. I don't know if this new SIG will have the resource to implement something like to solve this open topic. I think it would be really useful for all the SIGs but also for the TF ecosystem.
There was a problem hiding this comment.
@lc0 The lifecycle of community models will be one of the topics the SIG Models will discuss. We expect that we will also have a periodic review for models. Thank you for your comment.
@bhack Thank you for your suggestion. I agree with you that it would be useful to have such an analyzer. We will definitely discuss it once the TF SIG Models is created. We will ask for more inputs from you.
| submissions | ||
| * Review final submissions and announce results | ||
|
|
||
| ## What modes of communication do you intend to use? |
There was a problem hiding this comment.
I think would be helpful to also define what is expected use-case.
Do we see it as python module, that can be imported downstream, hence models are tested and maintained.
Or just a collection of scripts, that contributed by different people at different times?
There was a problem hiding this comment.
This comment is connected to the models's reusable components topic.
Some times ago we had a ticket about this at #223
There was a problem hiding this comment.
There are many dups or semi-dups like tensorflow/addons#1903 (comment) that confuse users and third party SIG maintainers.
| using TensorFlow 2 for state-of-the-art research. Furthermore, SIG provides | ||
| product feedback to help TensorFlow to be improved. | ||
|
|
||
| ## Who will be part of it? |
There was a problem hiding this comment.
What about keras-cv/keras-nlp?
There was a problem hiding this comment.
@bhack @AakashKumarNain Yes, the Keras team (keras-cv/nlp) will contribute to the SIG Models.
|
In general I think that is great and strategic to have this SIG cause Models and Hub are really one of the unique places in the ecosystem where all of the pieces converges together to compose our end2end goal. Also we often suffer more then the Pytorch ecosystem about availability of third party papers official (and unofficial) reference implementations in the github "universe". But we really need to take this SIG as an occasion to improve our ecosystem about component re-usability and ownership disambiguation. |
* Change SIG to SIG Models * Update the Gitter chat URL (https://gitter.im/tensorflow/sig-models)
theadactyl
added
RFC: Accepted
This RFC will be open for comment until Wednesday, November 11st, 2020.
Creating SIG Models
We propose to create a TensorFlow SIG Models.
What is this group for?
This group is for discussions and collaborations on enabling community
contributions to TensorFlow Model Garden
and Tensorflow Hub.
SIG Models will focus on empowering the community to contribute state-of-the-art
model implementation in TensorFlow 2. It will benefit the whole community by
providing recommended implementations and models with reproducible results.
SIG Models will have several subgroups (e.g., SIG Models Vision and SIG Models
NLP) covering different machine learning areas. There are several SIG leads for
each group to coordinate the contributions, run community events like contests,
and maintain the code quality through the review process.