Skip to content

Updated the redistributed version of Tensorflow to 1.14 #3929

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Jul 8, 2019
Merged

Updated the redistributed version of Tensorflow to 1.14 #3929

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
8749a10
Merge pull request #1 from dotnet/master
harishsk Jun 20, 2019
fe25bf6
Fixed build errors resulting from upgrade to VS2019 compilers
harishsk Jun 21, 2019
cb446be
Added additional message describing the previous fix
harishsk Jun 21, 2019
b5ee220
Merge pull request #2 from dotnet/master
harishsk Jun 21, 2019
b9a7471
Merge pull request #3 from dotnet/master
harishsk Jun 24, 2019
80e238d
Merge pull request #4 from dotnet/master
harishsk Jun 27, 2019
8bc8c0f
Updated Tensorflow to version 1.14
harishsk Jun 27, 2019
a1dbdf5
Removed rename rules for Mac
harishsk Jun 27, 2019
5084a8d
Updated names of files for Mac
harishsk Jun 27, 2019
28eaf1c
Attempting to fix test errors on Mac
harishsk Jun 28, 2019
c94d209
Attempting to fix Mac test errors
harishsk Jun 28, 2019
2fe4404
Changed list of files to copy from archive for Linux and Mac
harishsk Jun 30, 2019
c59d5fc
Fixed CopyNativeAssemblies to copy the tensorflow binaries that have …
harishsk Jul 1, 2019
f62491f
Fixed windows build
harishsk Jul 1, 2019
414b9b3
Another attempt at fixing both Windows and Linux builds
harishsk Jul 1, 2019
2b19fe3
Changed copying version-suffixed binaries to occur only for non-Windo…
harishsk Jul 2, 2019
cdf4b26
Changed copying of versioned assemblies to use HasMetadata
harishsk Jul 2, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion Directory.Build.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,14 @@
</NativeAssemblyReference>
</ItemGroup>

<ItemGroup>
<NativeAssemblyReferenceWithMajorVersion Include="@(NativeAssemblyReference->HasMetadata('MajorVersion'))">
<!-- Tensorflow has a different naming scheme for v1.14.0. Those binaries need to be copied along with the standard names -->
<AssemblyPathWithMajorVersion Condition="'$(OS)' != 'Windows_NT'">$(NativeOutputPath)$(LibPrefix)%(NativeAssemblyReference.Identity)$(LibExtension).%(NativeAssemblyReference.MajorVersion)</AssemblyPathWithMajorVersion>
<AssemblyPathWithMajorVersion Condition="$([MSBuild]::IsOSPlatform('osx'))">$(NativeOutputPath)$(LibPrefix)%(NativeAssemblyReference.Identity).%(NativeAssemblyReference.MajorVersion)$(LibExtension)</AssemblyPathWithMajorVersion>
</NativeAssemblyReferenceWithMajorVersion>
</ItemGroup>

<Copy SourceFiles = "@(NativeAssemblyReference->'%(FullAssemblyPath)')"
DestinationFolder="$(OutputPath)"
OverwriteReadOnlyFiles="$(OverwriteReadOnlyFiles)"
Expand All @@ -30,7 +38,19 @@
UseSymboliclinksIfPossible="$(CreateSymbolicLinksForPublishFilesIfPossible)">
<Output TaskParameter="DestinationFiles" ItemName="FileWrites"/>
</Copy>


<!-- Optionally copy the native binaries that have a version number attended (Only tensorflow right now) -->
<Copy Condition="'$(OS)' != 'Windows_NT'"
SourceFiles = "@(NativeAssemblyReferenceWithMajorVersion->'%(AssemblyPathWithMajorVersion)')"
DestinationFolder="$(OutputPath)"
OverwriteReadOnlyFiles="$(OverwriteReadOnlyFiles)"
Retries="$(CopyRetryCount)"
RetryDelayMilliseconds="$(CopyRetryDelayMilliseconds)"
UseHardlinksIfPossible="$(CreateHardLinksForPublishFilesIfPossible)"
UseSymboliclinksIfPossible="$(CreateSymbolicLinksForPublishFilesIfPossible)">
<Output TaskParameter="DestinationFiles" ItemName="FileWrites"/>
</Copy>

</Target>

<Import Project="$(ToolsDir)/versioning.targets" Condition="Exists('$(ToolsDir)/versioning.targets')" />
Expand Down
3 changes: 2 additions & 1 deletion build/Dependencies.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@
<SystemDrawingCommonPackageVersion>4.5.0</SystemDrawingCommonPackageVersion>
<SystemIOFileSystemAccessControl>4.5.0</SystemIOFileSystemAccessControl>
<SystemSecurityPrincipalWindows>4.5.0</SystemSecurityPrincipalWindows>
<TensorFlowVersion>1.13.1</TensorFlowVersion>
<TensorFlowVersion>1.14.0</TensorFlowVersion>
<TensorFlowMajorVersion>1</TensorFlowMajorVersion>
</PropertyGroup>

<!-- Code Analyzer Dependencies -->
Expand Down
8 changes: 6 additions & 2 deletions src/Microsoft.ML.DnnAnalyzer/Microsoft.ML.DnnAnalyzer/Microsoft.ML.DnnAnalyzer.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,12 @@

<!-- TensorFlow is 64-bit only -->
<ItemGroup Condition="'$(NativeTargetArchitecture)' == 'x64'">
<NativeAssemblyReference Include="tensorflow" />
<NativeAssemblyReference Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework" />
<NativeAssemblyReference Include="tensorflow">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
<NativeAssemblyReference Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
</ItemGroup>

</Project>
30 changes: 21 additions & 9 deletions src/Redist/Microsoft.ML.TensorFlow.Redist/Microsoft.ML.TensorFlow.Redist.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,27 @@

<ItemGroup>
<TensorFlowConfig Include="windows" FileExtension=".zip" FilesFromArchive="lib\tensorflow.dll;include\tensorflow\c\LICENSE" Runtime="win-x64"/>
<TensorFlowConfig Include="linux" FileExtension=".tar.gz" FilesFromArchive="lib\libtensorflow.so;lib\libtensorflow_framework.so;include\tensorflow\c\LICENSE" Runtime="linux-x64" />
<TensorFlowConfig Include="darwin" FileExtension=".tar.gz" FilesFromArchive="lib\libtensorflow.so;lib\libtensorflow_framework.so;include\tensorflow\c\LICENSE" Runtime="osx-x64" />

<AdditionalDownloadFile Include="https://raw.githubusercontent.com/tensorflow/tensorflow/master/LICENSE" DownloadFile="$(IntermediateOutputPath)LICENSE" />
<TensorFlowConfig Include="linux" FileExtension=".tar.gz"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just an FYI -

The tests in the repo do not run against the produced NuGet packages. Instead, you need to manually test that the NuGet packages are created correctly.

Can you build the packages locally, and then manually smoke test them to ensure they are created correctly?

FilesFromArchive="lib\libtensorflow.so;
lib\libtensorflow.so.$(TensorFlowMajorVersion);
lib\libtensorflow.so.$(TensorFlowVersion);
lib\libtensorflow_framework.so;
lib\libtensorflow_framework.so.$(TensorFlowMajorVersion);
lib\libtensorflow_framework.so.$(TensorFlowVersion);
include\tensorflow\c\LICENSE"
Runtime="linux-x64" />
<TensorFlowConfig Include="darwin" FileExtension=".tar.gz"
FilesFromArchive="lib\libtensorflow.dylib;
lib\libtensorflow.$(TensorFlowMajorVersion).dylib;
lib\libtensorflow.$(TensorFlowVersion).dylib;
lib\libtensorflow_framework.dylib;
lib\libtensorflow_framework.$(TensorFlowMajorVersion).dylib;
lib\libtensorflow_framework.$(TensorFlowVersion).dylib;
include\tensorflow\c\LICENSE"
Runtime="osx-x64" />


<AdditionalDownloadFile Include="https://raw.githubusercontent.com/tensorflow/tensorflow/master/LICENSE" DownloadFile="$(IntermediateOutputPath)LICENSE" />
</ItemGroup>

<Target Name="PrepareArchives">
Expand Down Expand Up @@ -93,8 +110,6 @@
<!-- batch rather than transform so that we can split FilesFromArchive metadata -->
<_fileFromArchive Include="%(TensorFlowArchive.FilesFromArchive)" ExtractDirectory="%(TensorFlowArchive.ExtractDirectory)" Runtime="%(TensorFlowArchive.Runtime)" />
<_fileFromArchive DestinationFile="%(FileName)%(Extension)"/>
<!-- rename the .so to .dylib since CORECLR's DLLImport convention requires dylib -->
<_fileFromArchive Condition="'%(Runtime)' == 'osx-x64' AND '%(Extension)' == '.so'" DestinationFile="%(FileName).dylib" />
<_fileFromArchive PackagePath="runtimes\%(_fileFromArchive.Runtime)\native\%(_fileFromArchive.DestinationFile)" />

<!-- LICENSE from the package is actually THIRD_PARTY_NOTICES-->
Expand All @@ -120,9 +135,6 @@
<Copy SourceFiles="@(FilesFromArchive)"
DestinationFiles="@(FilesFromArchive->'%(TargetPath)')" />

<!-- fix up the renamed libtensorflow, this can only run on mac -->
<Exec Condition="'%(FilesFromArchive.Runtime)' == 'osx-x64' AND '%(FilesFromArchive.FileName)' == 'libtensorflow'"
Command="install_name_tool -change @rpath/libtensorflow_framework.so @rpath/libtensorflow_framework.dylib &quot;%(FilesFromArchive.TargetPath)&quot;" />
</Target>

<!-- Define Build to do the work of downloading and copying assets -->
Expand Down
1 change: 0 additions & 1 deletion src/Redist/Microsoft.ML.TensorFlow.Redist/libtensorflow-cpu-darwin-x86_64-1.13.1.tar.gz.sha
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions src/Redist/Microsoft.ML.TensorFlow.Redist/libtensorflow-cpu-darwin-x86_64-1.14.0.tar.gz.sha
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
7002EF701BD23C5EF5FF94192E935F0DDF960A21BE2531CEE158586830C00E0BA889900F7F6E8AB568BEE0ACF1F5A6A246BB43D11C4109E9DC782B46377D8142
1 change: 0 additions & 1 deletion src/Redist/Microsoft.ML.TensorFlow.Redist/libtensorflow-cpu-linux-x86_64-1.13.1.tar.gz.sha
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions src/Redist/Microsoft.ML.TensorFlow.Redist/libtensorflow-cpu-linux-x86_64-1.14.0.tar.gz.sha
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
E3F6D0309117E9E45780ECF8BC4D0268B3FC9F12E3E38FFE58496789330A4ACD2DC8FF721F3B8900357F6155F8A54000E45B99495F823486B558E8B42532392D
1 change: 0 additions & 1 deletion src/Redist/Microsoft.ML.TensorFlow.Redist/libtensorflow-cpu-windows-x86_64-1.13.1.zip.sha
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions src/Redist/Microsoft.ML.TensorFlow.Redist/libtensorflow-cpu-windows-x86_64-1.14.0.zip.sha
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
59A2B80B441439B851202358CE4A65BA0DDDB319A8A29E87B135DCD9954BC5B0628F2C0C8E72D6942EA3CDCE172805C2BD5421815B3D0210B62BC0936DC59A08
8 changes: 6 additions & 2 deletions test/Microsoft.ML.Core.Tests/Microsoft.ML.Core.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,12 @@

<!-- TensorFlow is 64-bit only -->
<ItemGroup Condition="'$(NativeTargetArchitecture)' == 'x64'">
<NativeAssemblyReference Include="tensorflow" />
<NativeAssemblyReference Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework" />
<NativeAssemblyReference Include="tensorflow">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
<NativeAssemblyReference Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
</ItemGroup>

</Project>
8 changes: 6 additions & 2 deletions test/Microsoft.ML.Functional.Tests/Microsoft.ML.Functional.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,12 @@

<!-- TensorFlow is 64-bit only -->
<ItemGroup Condition="'$(NativeTargetArchitecture)' == 'x64'">
<NativeAssemblyReference Include="tensorflow" />
<NativeAssemblyReference Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework" />
<NativeAssemblyReference Include="tensorflow">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
<NativeAssemblyReference Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.ML.TensorFlow.TestModels" Version="$(MicrosoftMLTensorFlowTestModelsVersion)" />
Expand Down
8 changes: 6 additions & 2 deletions test/Microsoft.ML.Tests/Microsoft.ML.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,12 @@

<!-- TensorFlow is 64-bit only -->
<ItemGroup Condition="'$(NativeTargetArchitecture)' == 'x64'">
<NativeAssemblyReference Include="tensorflow" />
<NativeAssemblyReference Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework" />
<NativeAssemblyReference Include="tensorflow">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
<NativeAssemblyReference Condition="'$(OS)' != 'Windows_NT'" Include="tensorflow_framework">
<MajorVersion>$(TensorFlowMajorVersion)</MajorVersion>
</NativeAssemblyReference>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.ML.TensorFlow.TestModels" Version="$(MicrosoftMLTensorFlowTestModelsVersion)" />
Expand Down