Init exporting and loading

[rnett]

This adds proper initializer exporting to SavedModelBundle, and fixes #365.

Won't be so big once #375 is merged.

[rnett]

Currently having issues since python does it's variable checkpoint restore, then runs init ops, which makes no sense.

I can manually run restore afterwards here, but that will run it twice, and doesn't help if you load the model in python.

[rnett]

I'm leaning towards not exporting our init ops at all (except maybe as bookkeeping). Any state initialized or changed within them will be handled by variable restoration anyways. And the only solution I can think of is removing Assign ops from the init ops before exporting or something similar, which is not great.

[karllessard]

@rnett , can you please rebase your PR since we have just upgraded to TF2.6.0 and all ops were updated as well?

[karllessard]

Currently having issues since python does it's variable checkpoint restore, then runs init ops, which makes no sense.

I can manually run restore afterwards here, but that will run it twice, and doesn't help if you load the model in python.

That is strange behavior if this is what is happening, indeed. Can you please provide more details on the purpose of this PR and summarize which what is the new logic flow in the init sequence when exporting/importing a model? I'm not sure to totally understand what are the impacts of these changes, thanks

[rnett]

Currently having issues since python does it's variable checkpoint restore, then runs init ops, which makes no sense.
I can manually run restore afterwards here, but that will run it twice, and doesn't help if you load the model in python.

That is strange behavior if this is what is happening, indeed. Can you please provide more details on the purpose of this PR and summarize which what is the new logic flow in the init sequence when exporting/importing a model? I'm not sure to totally understand what are the impacts of these changes, thanks

Yeah, I'm not sure why it's done that way. Nothing really changes on our end for initialization, things are just remembered better when using SavedModels. I.e. init ops will be correctly added to the init scope when loading a SavedModel, regardless of whether they were added from Python or Java (they won't be ran, since variables will already have been loaded). It also adds table init ops from some hub models as init ops, so that they are ran (see #365).