Skip to content

Update to TensorFlow 2.10.1 #282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jan 24, 2023
Merged

Conversation

Minnozz
Copy link
Contributor

@Minnozz Minnozz commented Jan 12, 2023

  • Update submodule to 2.10.1
  • Update Dockerfiles in docker/ and ci_build/ to tensorflow/tensorflow:2.10.1
    • Verify that stack --docker test succeeds
  • Update Nix integration to nixpkgs that contains libtensorflow 2.10.1
    • Verify that stack --nix test succeeds
  • Update protobuf file list
  • Use new byte layout for string tensors
    (breaking change; fixes QueueTest in tensorflow-ops)

@google-cla
Copy link

google-cla bot commented Jan 12, 2023

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@Minnozz Minnozz force-pushed the tensorflow-2.10.1 branch 4 times, most recently from bdaa6e1 to 12ac796 Compare January 13, 2023 20:40
@Minnozz Minnozz marked this pull request as ready for review January 13, 2023 20:42
@Minnozz
Copy link
Contributor Author

Minnozz commented Jan 13, 2023

I'm not a user of TensorFlow; I updated these bindings for a colleague who needed the SavedModel functionality.
Feedback or further contributions are welcome.

@fkm3
Copy link
Contributor

fkm3 commented Jan 14, 2023

Thanks for the PR, I'll try to review soon.

@fkm3
Copy link
Contributor

fkm3 commented Jan 17, 2023

I'm seeing some failures in tensorflow-ops:GradientTest. Running the CI to confirm. Are you seeing the same? Haven't had a chance to try debugging yet.

This PR does a couple things

  • Updates to new TF version
  • Updates to new GHC and Stack version
  • Switches from TF_DeprecatedSession to TF_Session
  • Adds support for SavedModel

I know sometimes things needs to happen together, but could you separate out at least the new feature (SavedModel support) if possible.

@Minnozz
Copy link
Contributor Author

Minnozz commented Jan 17, 2023

I'm seeing some failures in tensorflow-ops:GradientTest. Running the CI to confirm. Are you seeing the same? Haven't had a chance to try debugging yet.

Yes, I see those too. I can't find any test output that indicates what's going wrong, so I'm not sure how to debug this further.

This PR does a couple things

* Updates to new TF version

* Updates to new GHC and Stack version

* Switches from TF_DeprecatedSession to TF_Session

* Adds support for SavedModel

I know sometimes things needs to happen together, but could you separate out at least the new feature (SavedModel support) if possible.

Yes, now that I have a (mostly) working version I'll try to separate things out some more. SavedModel support depends on the new TF version (for TF_LoadSessionFromSavedModel) and on TF_Session (which is used by TF_LoadSessionFromSavedModel). I'll try to see if the new GHC and Stack versions are still required (it was one of the first things I tried to solve a compilation error).

@Minnozz
Copy link
Contributor Author

Minnozz commented Jan 17, 2023

@fkm3 Is there a way you would like to receive sequential PRs or should I wait with submitting the next one until the previous one is merged?

@Minnozz Minnozz marked this pull request as draft January 17, 2023 10:52
@Minnozz Minnozz changed the title Update to TensorFlow 2.10.1 and support SavedModel Update to TensorFlow 2.10.1 Jan 17, 2023
@Minnozz
Copy link
Contributor Author

Minnozz commented Jan 17, 2023

@fkm3 I split out the original PR into multiple PRs (in merge order):

Updating the Stackage / GHC version turns out to not be necessary.

I could use some help with updating the byte layout of string tensors, which is the reason the QueueTest fails.

@fkm3
Copy link
Contributor

fkm3 commented Jan 17, 2023

Thanks!

I could use some help with updating the byte layout of string tensors, which is the reason the QueueTest fails.

I'll try to figure it out. Thanks for narrowing it down

@Minnozz
Copy link
Contributor Author

Minnozz commented Jan 17, 2023

@fkm3
Copy link
Contributor

fkm3 commented Jan 22, 2023

I think I've got a working solution at 63753cc. Can you cherry pick that CL into this PR? (I either don't have permissions to push to this or I am doing it wrong)

@Minnozz
Copy link
Contributor Author

Minnozz commented Jan 22, 2023

@fkm3 Thanks for implementing the new TF_STRING format! I've replaced my placeholder commit with yours.

@Minnozz Minnozz marked this pull request as ready for review January 22, 2023 11:00
@Minnozz
Copy link
Contributor Author

Minnozz commented Jan 22, 2023

My remaining question is whether all the protos I added should be generated or if some of them are considered internal.

@fkm3
Copy link
Contributor

fkm3 commented Jan 23, 2023

I'm not sure. We could look at the BUILD files and check the visibility for the various proto files, but I'm not sure if it is worth the effort.

For the sake of compile times, it would be best to only add what we need. Not a blocker though, we can always trim it later and build time wise other parts of the library are much worse (I can't believe how slow this project builds, not sure if it was always this bad or if something regressed).

@fkm3
Copy link
Contributor

fkm3 commented Jan 23, 2023

There is a build error. It is my fault, forgot to test my commit with --pedantic

tensorflow                 > /[tfhs/tensorflow/src/TensorFlow/Types.hs:82](https://cs.corp.google.com/piper///depot/google3/tfhs/tensorflow/src/TensorFlow/Types.hs?l=82):1: error: [-Wunused-imports, -Werror=unused-imports]
tensorflow                 >     The qualified import of ‘Data.Attoparsec.ByteString’ is redundant
tensorflow                 >       except perhaps to import instances from ‘Data.Attoparsec.ByteString’
tensorflow                 >     To import instances alone, use: import Data.Attoparsec.ByteString()
tensorflow                 >    |
tensorflow                 > 82 | import qualified Data.Attoparsec.ByteString as Atto
tensorflow                 >    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tensorflow                 > 
tensorflow                 > /[tfhs/tensorflow/src/TensorFlow/Types.hs:131](https://cs.corp.google.com/piper///depot/google3/tfhs/tensorflow/src/TensorFlow/Types.hs?l=131):1: error: [-Wunused-imports, -Werror=unused-imports]
tensorflow                 >     The import of ‘TensorFlow.Internal.VarInt’ is redundant
tensorflow                 >       except perhaps to import instances from ‘TensorFlow.Internal.VarInt’
tensorflow                 >     To import instances alone, use: import TensorFlow.Internal.VarInt()
tensorflow                 >     |
tensorflow                 > 131 | import TensorFlow.Internal.VarInt (getVarInt, putVarInt)
tensorflow                 >     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Do you mind amending it to delete those two imports?

@fkm3 fkm3 merged commit 199f1c7 into tensorflow:master Jan 24, 2023
@fkm3
Copy link
Contributor

fkm3 commented Jan 24, 2023

Thanks for the PR, first TF version update in over 2 years!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants