Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Helix Integration. #5837

Merged
merged 14 commits into from
Jun 23, 2021
Merged

Helix Integration. #5837

merged 14 commits into from
Jun 23, 2021

Conversation

michaelgsharp
Copy link
Member

@michaelgsharp michaelgsharp commented Jun 8, 2021

This PR changes the CI testing process to use Helix instead of testing on the build machines (except for the code coverage tests currently). This lets us take advantage of more powerful machines for our testing.

I disabled 6 tests tracked here #5845 due to slight differences between the results in the helix machines and the build machines.

This PR also fixes a bug in the MatrixFactorizationNative where we were compiling the native code to be compatible with Intel CPU's only.

@justinormont
Copy link
Contributor

Description of helix: (I had look up helix)

Helix is the distributed test platform that we use to run tests. We build a helix payload that contains the publish directory of every test project that we want to test send a job with with this payload to a set of queues for the various combinations of OS that we want to test for example: Windows.10.Amd64.ClientRS4.VS2017.Open, OSX.1012.Amd64.Open, Ubuntu.1804.Amd64.Open. Helix takes care of unzipping, running the job, and reporting results.

via: https://github.com/dotnet/aspnetcore/blob/83f7970b8cf3d87299d564e713caa0310eaf326b/docs/Helix.md

@michaelgsharp
Copy link
Member Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@codecov
Copy link

codecov bot commented Jun 17, 2021

Codecov Report

Merging #5837 (bc11aae) into main (6114b4f) will decrease coverage by 0.26%.
The diff coverage is 46.77%.

@@            Coverage Diff             @@
##             main    #5837      +/-   ##
==========================================
- Coverage   68.46%   68.20%   -0.27%     
==========================================
  Files        1134     1134              
  Lines      241911   241939      +28     
  Branches    25109    25298     +189     
==========================================
- Hits       165628   165018     -610     
- Misses      69785    70267     +482     
- Partials     6498     6654     +156     
Flag Coverage Δ
Debug 68.20% <46.77%> (-0.27%) ⬇️
production 62.86% <0.00%> (-0.21%) ⬇️
test 88.82% <49.15%> (-0.49%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...icrosoft.ML.Core/Utilities/ResourceManagerUtils.cs 59.15% <0.00%> (-2.46%) ⬇️
.../Microsoft.ML.AutoML.Tests/ColumnInferenceTests.cs 96.29% <0.00%> (-2.44%) ⬇️
...r.Tests/ApprovalTests/ConsoleCodeGeneratorTests.cs 99.54% <0.00%> (-0.46%) ⬇️
....CodeGenerator.Tests/ApprovalTests/TemplateTest.cs 95.74% <0.00%> (-4.26%) ⬇️
...ft.ML.Core.Tests/UnitTests/TestResourceDownload.cs 0.00% <ø> (-75.52%) ⬇️
...est/Microsoft.ML.Predictor.Tests/TestPredictors.cs 69.62% <ø> (-0.50%) ⬇️
...ests/TrainerEstimators/MatrixFactorizationTests.cs 83.59% <0.00%> (-13.48%) ⬇️
test/Microsoft.ML.FSharp.Tests/SmokeTests.fs 77.77% <33.33%> (-10.46%) ⬇️
...est/Microsoft.ML.TestFrameworkCommon/TestCommon.cs 77.83% <40.00%> (-1.12%) ⬇️
test/Microsoft.ML.TestFramework/TestCommandBase.cs 39.41% <85.71%> (-5.55%) ⬇️
... and 68 more

Copy link
Member

@tarekgh tarekgh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I'll let @safern and @ericstj take a look as they more familiar with such changes :-)

Directory.Build.targets Outdated Show resolved Hide resolved
build/ci/send-to-helix.yml Outdated Show resolved Hide resolved
build/ci/send-to-helix.yml Outdated Show resolved Hide resolved
eng/helix.proj Outdated Show resolved Hide resolved
eng/helix.proj Outdated Show resolved Hide resolved
eng/helix.proj Outdated Show resolved Hide resolved
eng/helix.proj Outdated Show resolved Hide resolved
eng/helix.proj Outdated Show resolved Hide resolved
@michaelgsharp michaelgsharp merged commit ce7f91a into dotnet:main Jun 23, 2021
@michaelgsharp michaelgsharp deleted the helix branch June 23, 2021 02:44
darth-vader-lg added a commit to darth-vader-lg/ML-NET that referenced this pull request Jun 26, 2021
* official/main:
  Helix Integration. (dotnet#5837)

# Conflicts:
#	src/Microsoft.ML.Mkl.Redist/Microsoft.ML.Mkl.Redist.csproj

Signed-off-by: darth-vader-lg <luigi.generale@gmail.com>
@ghost ghost locked as resolved and limited conversation to collaborators Mar 17, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants