Adding an example for Supervised Contrastive Learning

[ksalama]

Supervised Contrastive Learning: https://arxiv.org/abs/2004.11362

[ksalama]

@fchollet - kindly review the PR

[fchollet]

Thank you for the PR. Please only include the Python file in the PR for now.

[ksalama]

Done - Only the Python file is included in the PR.

[fchollet]

Thanks for the update!

[ksalama]

@fchollet - Have addressed your comments. Kindly review my revisions.
Thanks!

[fchollet]

Thank you for the update!

[ksalama]

@fchollet kindly find my revisions. Cheers

[fchollet]

Looking good! Just a few minor comments.

[fchollet]

Thank you! Looks good, please use preprocessing layers and address the remaining comments.

[ksalama]

@fchollet - I think I have addressed all your comments, including:

1. Using preprocessing layers for data augmentation (maintaining the same or slightly better results compared to the DataGenerator)
2. Adding a conclusion section
3. Fixing the ReLU and BatchNorm layers ordering

Please let me know if there is anything else need to be modified.

[ksalama]

I have simplified the models by removing the BatchNorm layers as they don't improve the results

[fchollet]

Using preprocessing layers for data augmentation (maintaining the same or slightly better results compared to the DataGenerator)

The code I'm seeing still uses ImageDataGenerator; did you push the changes to the PR?

[ksalama]

@fchollet - Sorry for the confusion. I've just pushed the changes and they should be now part of the PR

[ksalama]

Also made some formatting updates

[fchollet]

Looks great! Please address the remaining nits and add the autogenerated .md and .ipynb files (you can generate them with the python autogen.py add_example command)

[ksalama]

Nits addressed

[fchollet]

Looks good, please add the generated files to the PR!

[8bitmp3]

Hey @ksalama I'm a tf.keras user here and just came across the PR. This example is awesome.

Can you please check and change RestNet to ResNet (assuming you're discussing residual networks like in https://arxiv.org/abs/1512.03385).

architecture, like ResNet, and multi-class problems with many labels.

Can I also suggest to improve the introduction just a little bit to help the readers understand what they're about to learn? You could, for example, write about 1) what the user will learn about - in a bit more detail (can be summarized from the paper on arXiv (with the link included) before going into the two-phase structure that you already mentioned; 2) what the user will do - it's a classification task for X using Y dataset. Both of these could be just 1-2 short sentences.

Also, I'd advise to copy-paste the Markdown text into Google Docs just to check that all grammar and spelling is 👍 - but I've not spotted anything out of the ordinary here.

Anyway, these are just suggestions from experience. I really enjoyed the paper, thanks for sharing the work!

[8bitmp3]

Can you please change RestNet to ResNet (assuming you're discussing residual networks like in https://arxiv.org/abs/1512.03385)?

Suggested change

	architecture, like RestNet, and multi-class problems with many labels.
	architecture, like ResNet, and multi-class problems with many labels.

[ksalama]

Thanks @8bitmp3 for spotting this. I have applied the fixes 👍

[fchollet]

Thanks @8bitmp3 for the feedback. @ksalama, I believe the example is ready, so you can add the .md and .ipynb files to the PR.

[ksalama]

@fchollet I have added the generated .md and .ipynb files to the PR

[fchollet]

Excellent, thank you for the great contribution! I'll take it from there 👍