Implements YogiOptimizer

[manzilz]

Implement Yogi optimizer in paper Adaptive Methods for Nonconvex Optimization as a tf.keras.optimizers.Optimizer

[jkr26]

+1 to this PR!!

[WindQAQ]

Hi @manzilz, thanks for the contribution! Could you run make code-format to format codes?

[manzilz]

@WindQAQ Finished code formatting. PTAL

[manzilz]

Fixed a small error.

[avinava-o]

+1 for this PR!

[manzilz]

Any update @WindQAQ @facaiy @Squadrick ?

[WindQAQ]

LGTM. Thanks for the PR!

[relaxation82]

It seems that the Yogi optimiser was released as part of tensorflow-addons-0.7.0, but I am unable to find any documentation about it. In the case I was just blind, could someone provide a link to a page with the documentation, or in case it is actually missing for the documentation, could you provide a short example describing how it can be used?

[seanpmorgan]

It seems that the Yogi optimiser was released as part of tensorflow-addons-0.7.0, but I am unable to find any documentation about it. In the case I was just blind, could someone provide a link to a page with the documentation, or in case it is actually missing for the documentation, could you provide a short example describing how it can be used?

@relaxation82 thank you very much for this callout. Seems the import got missed and thus the API docs were not generated.

I've submitted a PR to fix this (#875) and we will likely do a patch release in the next couple of weeks. In the mean time it can still be used on 0.7.0 with:

from tensorflow_addons.optimizers.yogi import Yogi

The documentation will be whats present in the doc strings of the module:
https://github.com/tensorflow/addons/blob/master/tensorflow_addons/optimizers/yogi.py