Fix pai training with optimizer config

[typhoonzero]

When training on PAI, we generate the train code calls runtime.pai.tensorflow.train which will call oss.save_oss_model to save the model and model meta-data to OSS, yet we generate the code using tfSaveModelTmplText to save the model meta-data again:

sqlflow/go/codegen/pai/template_tf.go

Lines 95 to 127 in 6974de2

	const tfSaveModelTmplText = tfImportsText + `
	import types
	estimator = import_model('''{{.Estimator}}''')
	is_estimator = is_tf_estimator(estimator)
	# Keras single node is using h5 format to save the model, no need to deal with export model format.
	# Keras distributed mode will use estimator, so this is also needed.
	FLAGS = tf.app.flags.FLAGS
	if is_estimator:
	if FLAGS.task_index == 0:
	with open("exported_path", "r") as fn:
	saved_model_path = fn.read()
	oss.save_dir("{{.OSSModelDir}}", saved_model_path)
	oss.save_file("{{.OSSModelDir}}", "exported_path")
	else:
	if len(FLAGS.worker_hosts.split(",")) > 1:
	if FLAGS.task_index == 0:
	oss.save_file("{{.OSSModelDir}}", "exported_path")
	else:
	oss.save_file("{{.OSSModelDir}}", "model_save")
	oss.save_metas("{{.OSSModelDir}}",
	{{.NumWorkers}},
	"tensorflow_model_desc",
	"{{.Estimator}}",
	feature_column_names,
	feature_column_names_map,
	feature_metas,
	label_meta,
	model_params,
	feature_columns_code)
	`

We do not need to save the meta-data twice.

And, we need to save the model meta-data as some Python code to save feature column, optimizer info. Yet runtime.pai.tensorflow.train tries to save the model_params as objects which will cause an error, we need to save the model_params before it is converted to objects.

File "/tensorflow_jobs/runtime/pai/tensorflow/train.py", line 123, in train
feature_columns_code, num_workers)
File "/tensorflow_jobs/runtime/model/oss.py", line 292, in save_oss_model
label_meta, model_params, feature_columns_code)
File "/tensorflow_jobs/runtime/model/oss.py", line 232, in save_metas
serialized = pickle.dumps(list(meta))
...
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle SwigPyObject objects

We should polish more after this fix was merged.

[sneaxiy]

Not quite understand what this PR does. Would you like to explain why this PR?

[sneaxiy]

LGTM.