Add Fast Decoding Support For Relative Dot Product

[aeloyq]

Add Fast Decoding Support For And Only For Relative Dot Product by enabling caching mechanism.
This pr have been tested on "En-Zh" case and got the same results as expected.

[googlebot]

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

[aeloyq]

Check CLA again.

[googlebot]

CLAs look good, thanks!

[afrozenator]

thanks a lot @aeloyq !

[aeloyq]

There's a typo in

tensor2tensor/models/transformer.py line-276

if self._hparams.self_attention_type not in ["dot_product", "dot_product_relative1"]:

should be modified to:

if self._hparams.self_attention_type not in ["dot_product", "dot_product_relative"]:

[afrozenator]

Thanks - yes @lukaszkaiser pointed that out, we fixed it on our end.