Skip to content

Nightlybuild fix #4935

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Mar 12, 2020
Merged

Nightlybuild fix #4935

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
9ab4bd0
remove unnecessary for nightlybuild
frank-dong-ms-zz Mar 11, 2020
10c83dd
add missing dependency and fix delete unnecessary file
frank-dong-ms-zz Mar 12, 2020
332ff14
fix library path setting for centos
frank-dong-ms-zz Mar 12, 2020
bc882b1
fix target framework issue
frank-dong-ms-zz Mar 12, 2020
7a72b02
add debug step
frank-dong-ms-zz Mar 12, 2020
e76b1d3
fix path
frank-dong-ms-zz Mar 12, 2020
7d3f61c
remove debug step
frank-dong-ms-zz Mar 12, 2020
797a364
fix comments
frank-dong-ms-zz Mar 12, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions build/.night-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,12 @@ jobs:
Debug_Build:
_configuration: Debug-netcoreapp3_0
_config_short: DI
_targetFramework: netcoreapp3.0
_includeBenchmarkData: false
Release_Build:
_configuration: Release-netcoreapp3_0
_config_short: RI
_targetFramework: netcoreapp3.0
_includeBenchmarkData: true
nightlyBuild: true
pool:
Expand Down
15 changes: 12 additions & 3 deletions build/ci/job-template.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ parameters:
jobs:
- job: ${{ parameters.name }}
${{ if eq(parameters.nightlyBuild, 'true') }}:
timeoutInMinutes: 20
timeoutInMinutes: 30
${{ if and(eq(parameters.nightlyBuild, 'false'), eq(parameters.codeCoverage, 'false')) }}:
timeoutInMinutes: 75
${{ if eq(parameters.codeCoverage, 'true') }}:
Expand All @@ -23,7 +23,7 @@ jobs:
dotnetPath: $(Build.SourcesDirectory)/Tools/dotnetcli/dotnet
nugetFeed: https://pkgs.dev.azure.com/dnceng/public/_packaging/MachineLearning/nuget/v3/index.json
nightlyBuildProjPath: $(Build.SourcesDirectory)/test/Microsoft.ML.NightlyBuild.Tests/Microsoft.ML.NightlyBuild.Tests.csproj
nightlyBuildRunPath: (Build.SourcesDirectory)/bin/AnyCPU.(_configuration)/Microsoft.ML.NightlyBuild.Tests/(_targetFramework)
nightlyBuildRunPath: $(Build.SourcesDirectory)/bin/AnyCPU.$(_configuration)/Microsoft.ML.NightlyBuild.Tests/$(_targetFramework)
packageUpdaterProjPath: $(Build.SourcesDirectory)/test/Microsoft.ML.NugetPackageVersionUpdater/Microsoft.ML.NugetPackageVersionUpdater.csproj
versionFilePath: $(Build.SourcesDirectory)/test/Microsoft.ML.NugetPackageVersionUpdater/latest_versions.txt
strategy:
Expand Down Expand Up @@ -52,7 +52,7 @@ jobs:
displayName: Install build dependencies
- ${{ if and( eq(parameters.nightlyBuild, 'true'), eq(parameters.pool.name, 'Hosted Ubuntu 1604')) }}:
- bash: echo "##vso[task.setvariable variable=LD_LIBRARY_PATH]$(nightlyBuildRunPath):$LD_LIBRARY_PATH"
displayName: Set LD_LIBRARY_PATH for Ubuntu to locate Native shared library in current running path
displayName: Set LD_LIBRARY_PATH for Ubuntu and CentOS to locate Native shared library in current running path
- script: ${{ parameters.buildScript }} -$(_configuration) -buildArch=${{ parameters.architecture }}
displayName: Build
- ${{ if eq(parameters.nightlyBuild, 'true') }}:
Expand All @@ -62,6 +62,15 @@ jobs:
displayName: List latest package versions
- script: $(dotnetPath) run --project $(packageUpdaterProjPath)
displayName: Update package versions for nightly build
- ${{ if eq(parameters.buildScript, 'build.cmd') }}:
- powershell: |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please explain why this is useless? If this useless, why do we have it in our tree and have it build?
If the reason is different, can you please add the actual reason? (Also, small typo: useless)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, for NightlyBuild pipeline only NightlyBuild test project is necessary, other test projects are built from previous "Build" step. This "Build" are doing multiple things thatNightlyBuild pipeline also requires, like initialize tools, build native components. We are reusing the "Build" step from CI build pipeline but with side effect that we are also building too much test projects. We can separate the build step into smaller steps and only include necessary steps in nightly build pipeline but that can be quite large effort and I'm not seeing much value of this.

And thanks for pointing out the typo, I will fix the typo

In reply to: 391391849 [](ancestors = 391391849)

Get-ChildItem -Path '.\bin\AnyCPU.*' -Recurse |
Select -ExpandProperty FullName |
Where {$_ -notlike '*\Microsoft.ML.NightlyBuild.Tests*'} |
sort length -Descending |
Remove-Item -force
Write-Output "Done cleaning up usless project..."
displayName: Clean up useless project
- script: $(dotnetPath) msbuild -restore $(nightlyBuildProjPath) /p:ReferenceTypeForTestFramework="Nuget" /p:Configuration=$(_configuration) /p:TargetArchitecture=${{ parameters.architecture }}
displayName: Build Nightly-Build Project with latest package versions
- script: ${{ parameters.buildScript }} -$(_configuration) -runnightlybuildtests
Expand Down
4 changes: 4 additions & 0 deletions test/Microsoft.ML.NightlyBuild.Tests/Microsoft.ML.NightlyBuild.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,8 @@
<ProjectReference Include="..\Microsoft.ML.TestFrameworkCommon\Microsoft.ML.TestFrameworkCommon.csproj" />
</ItemGroup>

<ItemGroup>
<PackageReference Include="Microsoft.ML.OnnxRuntime" Version="$(MicrosoftMLOnnxRuntimePackageVersion)" />
</ItemGroup>

</Project>