ML Engine integration broken with 1.11

[connectdotz]

Description

Trying to train translate_ende_wmt32k demo with ml-engine, got tensorflow version mismatch error.

looks like t2t 1.11 introduced tensorflow_probability, which added an dependency of tensorflow 1.12. Based on the ML Engine runtime versions: it currently supports up to tensorflow 1.10. Thus t2t 1.11 can no longer submit ML Engine training job.

Environment information

OS: <your answer here>

$ pip freeze | grep tensor
# your output here

mesh-tensorflow==0.0.4
tensor2tensor==1.11.0
tensorboard==1.12.0
tensorflow==1.12.0
tensorflow-hub==0.1.1
tensorflow-metadata==0.9.0
tensorflow-probability==0.5.0

$ python -V
# your output here
Python 3.6.6

For bugs: reproduction and error logs

# Steps to reproduce:

just follow ML Engine example

# Error logs: From ML Engine log:

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/root/.local/lib/python3.5/site-packages/tensor2tensor/bin/t2t_trainer.py", line 25, in <module>
    from tensor2tensor import models  # pylint: disable=unused-import
  File "/root/.local/lib/python3.5/site-packages/tensor2tensor/models/__init__.py", line 22, in <module>
    from tensor2tensor.layers import modalities  # pylint: disable=g-import-not-at-top
  File "/root/.local/lib/python3.5/site-packages/tensor2tensor/layers/modalities.py", line 22, in <module>
    from tensor2tensor.layers import common_attention
  File "/root/.local/lib/python3.5/site-packages/tensor2tensor/layers/common_attention.py", line 31, in <module>
    from tensor2tensor.layers import common_layers
  File "/root/.local/lib/python3.5/site-packages/tensor2tensor/layers/common_layers.py", line 32, in <module>
    import tensorflow_probability as tfp
  File "/root/.local/lib/python3.5/site-packages/tensorflow_probability/__init__.py", line 68, in <module>
    _ensure_tf_install()
  File "/root/.local/lib/python3.5/site-packages/tensorflow_probability/__init__.py", line 65, in _ensure_tf_install
    present=tf.__version__))
ImportError: This version of TensorFlow Probability requires TensorFlow version >= 1.12.0; Detected an installation of version 1.9.0. Please upgrade TensorFlow to proceed.

[dcompgriff]

I'm having the same issues.