Tensorflow 2.3.0 Support #267

jcmartin · 2020-11-05T10:24:02Z

I have verified that this patch builds through Docker on an Ubuntu machine. The list of changes is as follows:

Support for Tensorflow 2.3.0
Support for ByteString as an argument. Needed for functions such as einsum.
Support for additional types for Adam optimizer.

I have verified all of these new features and have been using internally for about a month.

google-cla · 2020-11-05T10:24:12Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-11-05T14:21:00Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

blackgnezdo

Great, thank you! Minor nits below.

I'll check what's up with the build and report back.

tensorflow-ops/src/TensorFlow/Minimize.hs

tensorflow-ops/tests/GradientTest.hs

tensorflow-ops/tests/OpsTest.hs

blackgnezdo · 2020-11-05T15:26:54Z

Build errors:

tensorflow-proto           > Installing library in /tfhs/.stack-work/install/x86_64-linux/a9c8c207ad3f6fbee05cf5280a4ba381dab715c23f35841cd40ab2d92a2ed93e/8.8.4/lib/x86_64-linux-ghc-8.8.4/tensorflow-proto-0.2.0.1-CW9xNdT4HVhw5p4FGDmg5
tensorflow-proto           > Registering library for tensorflow-proto-0.2.0.1..
tensorflow                 > configure (lib + test)
tensorflow                 > Configuring tensorflow-0.2.0.1...
tensorflow-opgen           > configure (lib)
tensorflow                 > build (lib + test)
tensorflow                 > Preprocessing library for tensorflow-0.2.0.1..
tensorflow                 > /tfhs/tensorflow/In file included from .stack-work/dist/x86_64-linux/Cabal-3.0.1.0/build/TensorFlow/Internal/Raw.chs.h:2:0:
tensorflow                 > /tfhs/third_party/tensorflow/tensorflow/c/c_api.h:23:10: fatal error: tensorflow/c/tf_datatype.h: No such file or directory
tensorflow                 >  #include "tensorflow/c/tf_datatype.h"
tensorflow                 >           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
tensorflow                 > compilation terminated.
tensorflow                 > c2hs: Error during preprocessing custom header file
tensorflow-opgen           > Configuring tensorflow-opgen-0.2.0.1...
tensorflow-opgen           > build (lib)
tensorflow-opgen           > Preprocessing library for tensorflow-opgen-0.2.0.1..
tensorflow-opgen           > Building library for tensorflow-opgen-0.2.0.1..
tensorflow-opgen           > [1 of 2] Compiling TensorFlow.OpGen.ParsedOp
tensorflow-opgen           > [2 of 2] Compiling TensorFlow.OpGen
tensorflow-opgen           > copy/register
tensorflow-opgen           > Installing library in /tfhs/.stack-work/install/x86_64-linux/a9c8c207ad3f6fbee05cf5280a4ba381dab715c23f35841cd40ab2d92a2ed93e/8.8.4/lib/x86_64-linux-ghc-8.8.4/tensorflow-opgen-0.2.0.1-ABE0vjcjW1e9iXzVsnDDe8
tensorflow-opgen           > Registering library for tensorflow-opgen-0.2.0.1..

--  While building package tensorflow-0.2.0.1 using:
      /root/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_3.0.1.0_ghc-8.8.4 --builddir=.stack-work/dist/x86_64-linux/Cabal-3.0.1.0 build lib:tensorflow test:FFITest test:VarIntTest --ghc-options ""
    Process exited with code: ExitFailure 1

jcmartin · 2020-11-05T15:34:05Z

The nitpicks are easy to take care of. I'll look into why it is not building.

jcmartin · 2020-11-05T16:00:44Z

I did a fresh build on my computer and was unable to recreate the build error.

git clone git@github.com:jcmartin/haskell.git
cd haskell
git submodule init
git submodule update
cd docker
docker build -t test .
cd ..
stack --docker --docker-image=test:latest test

Perhaps the submodule are not getting updated?

google-cla · 2020-11-05T17:21:31Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-11-05T17:45:32Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

jcmartin · 2020-11-05T17:45:47Z

I believe I have identified the build issue. It is due to ci_build/Dockerfile not being updated. I have addressed this with the latest commit. Everything should hopefully build correctly now.

jcmartin · 2020-11-05T18:12:17Z

@googlebot I signed it!

blackgnezdo

Thank you so much for wrapping the stringly-typed parameters!

Another small batch of nits while the build is running...

tensorflow-ops/src/TensorFlow/Minimize.hs

tensorflow-ops/src/TensorFlow/Convolution.hs

blackgnezdo

Awesome! Thanks a lot for the long overdue upgrade and for bearing with my nitpicks.

I'll merge this as soon as I get a green build.

jcmartin · 2020-11-06T14:35:05Z

@blackgnezdo Do you mind running kokoro again? I discovered that the build was failing due to a deprecated import line that has since been fixed. Running ci_build/outer_launch_tests.sh passes on my end.

blackgnezdo · 2020-11-06T17:18:12Z

CI failed due to

curl: (56) OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 104

I'll restart it.

blackgnezdo · 2020-11-06T19:31:38Z

Cool!
tensorflow-ops > testEinsum: [OK]

jcmartin · 2020-11-07T02:50:08Z

@blackgnezdo Thanks for handling this so expediently! I'd recommend bumping the package version numbers and updating the related documentation to use a later version of stack. The stack version currently recommended will issue warnings about it not supporting GHC 8.8.

It would also be nice if this could somehow make it onto hackage and stackage.

In addition, there is some minor cruft related to old pull requests and issues related to 2.3.0 that should probably be garbage collected.

jcmartin added 5 commits August 28, 2020 01:54

Tensorflow 2.3.0 building and passing tests.

9cc48c3

Added einsum and test.

b30d9a5

Added ByteString as a possible argument to a function.

43eebd2

Support more data types for Adam.

0c95543

Move to later version of LTS on stackage.

92bf575

Revert Dockerfile debugging.

03164c0

blackgnezdo added the kokoro:run label Nov 5, 2020

kokoro-team removed the kokoro:run label Nov 5, 2020

blackgnezdo requested changes Nov 5, 2020

View reviewed changes

tensorflow-ops/src/TensorFlow/Minimize.hs Outdated Show resolved Hide resolved

tensorflow-ops/tests/GradientTest.hs Outdated Show resolved Hide resolved

tensorflow-ops/tests/OpsTest.hs Outdated Show resolved Hide resolved

jcmartin added 2 commits November 5, 2020 17:17

Addresses blackgnezdo nitpicks related to comments.

550ed59

Added a wrapper module for convolution functions.

4bca19d

Update Dockerfile for ci.

f947768

jcmartin requested a review from blackgnezdo November 5, 2020 18:36

blackgnezdo requested changes Nov 5, 2020

View reviewed changes

Further nitpicks by blackgnezdo addressed.

5258303

jcmartin requested a review from blackgnezdo November 5, 2020 20:54

blackgnezdo added the kokoro:run label Nov 6, 2020

kokoro-team removed the kokoro:run label Nov 6, 2020

blackgnezdo approved these changes Nov 6, 2020

View reviewed changes

jcmartin added 2 commits November 6, 2020 05:56

Update ci build to use a later version of stack.

ad9ae14

Removed a deprecated import in GradientTest.

4824ca9

blackgnezdo added the kokoro:run label Nov 6, 2020

kokoro-team removed the kokoro:run label Nov 6, 2020

blackgnezdo merged commit c66c912 into tensorflow:master Nov 6, 2020

This was referenced Nov 7, 2020

Upgraded to tensorflow 2.3 #265

Closed

Preliminary support for TensorFlow 2 #242

Closed

Package won't install. #264

Closed

Fail on Debian Build #262

Closed

blackgnezdo mentioned this pull request Sep 13, 2022

Running a loaded TF2 SavedModel gives TensorfFlowException TF_INVALID_ARGUMENT #279

Closed

Tensorflow 2.3.0 Support #267

Tensorflow 2.3.0 Support #267

Uh oh!

Conversation

jcmartin commented Nov 5, 2020

Uh oh!

google-cla bot commented Nov 5, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

Uh oh!

google-cla bot commented Nov 5, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

Uh oh!

blackgnezdo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

blackgnezdo commented Nov 5, 2020

Uh oh!

jcmartin commented Nov 5, 2020

Uh oh!

jcmartin commented Nov 5, 2020

Uh oh!

google-cla bot commented Nov 5, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

Uh oh!

google-cla bot commented Nov 5, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

Uh oh!

jcmartin commented Nov 5, 2020

Uh oh!

jcmartin commented Nov 5, 2020

Uh oh!

blackgnezdo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

blackgnezdo left a comment

Choose a reason for hiding this comment

Uh oh!

jcmartin commented Nov 6, 2020

Uh oh!

blackgnezdo commented Nov 6, 2020

Uh oh!

blackgnezdo commented Nov 6, 2020

Uh oh!

jcmartin commented Nov 7, 2020

Uh oh!

Uh oh!