Add SBRPTests #689

v-chayan · 2023-06-01T06:27:52Z

Implement some basic tests. Resolves dotnet/source-build#3089

MichaelSimons

Thanks for the PR that introduces tests. This will be helpful in preventing regressions.

Friendly reminder to link PRs to the issues they are addressing.

This new test should be integrated into CI. Builds/PR validation should fail if the tests don't pass.

run-tests.sh

tests/SBRPTests/SBRPTests.cs

tests/SBRPTests/PackageData.cs

tests/SBRPTests/SBRPFixture.cs

tests/SBRPTests/SBRPTests.cs

tests/SBRPTests/PackageData.cs

tests/SBRPTests/CommandHelper.cs

tests/SBRPTests/SBRPTests.cs

eng/Versions.props

tests/SBRPTests/SBRPTests.csproj

ViktorHofer

Please change the test project to use Arcade's defaults to avoid all the boilerplate code. Also, consider using the default build scripts for running tests and test their behavior not just for Unix but also for Windows.

tests/GenerateScriptRegressionTests/GenerateScriptRegressionTests.Tests.csproj

tests/GenerateScriptRegressionTests/ExecuteHelper.cs

MichaelSimons

I like the utilizing of the arcade testing infra versus adding a new run test script.

tests/GenerateScriptRegressionTests/GenerateScriptRegressionTestsFixture.cs

tests/GenerateScriptRegressionTests/GenerateScriptRegressionTests.cs

tests/GenerateScriptRegressionTests/CompareDirs.cs

tests/GenerateScriptTests/GenerateScriptTests.Tests.csproj

tests/GenerateScriptTests/CompareDirs.cs

MichaelSimons

Friendly reminder to integrate the tests into the pipeline so that they run in PR validiation.

tests/GenerateScriptTests/GenerateScriptTests.cs

tests/GenerateScriptRegressionTests/GenerateScriptRegressionTests.cs

tests/GenerateScriptTests/GenerateScriptTests.cs

test.sh

azure-pipelines/builds/ci.yml

test.cmd

MichaelSimons · 2023-08-01T16:04:08Z

tests/GenerateScriptTests/GenerateScriptTests.cs

+        new object[] { "System.Xml.ReaderWriter", "4.0.11", PackageType.Reference },
+        new object[] { "Microsoft.Extensions.Logging.Abstractions", "7.0.1", PackageType.Reference },
+        new object[] { "Microsoft.CodeAnalysis.CSharp", "3.11.0", PackageType.Reference },
+        new object[] { "System.Security.Cryptography.Pkcs", "7.0.2", PackageType.Reference },


We should be validating text only packages as well. All the test infrastructure was defined but no test data for them were added.

Suggested change

new object[] { "System.Security.Cryptography.Pkcs", "7.0.2", PackageType.Reference },

new object[] { "System.Security.Cryptography.Pkcs", "7.0.2", PackageType.Reference },

new object[] { "Microsoft.Build.NoTargets", "3.7.0", PackageType.Text },

test.cmd

ViktorHofer

You checked in the project file which I think you didn't want to? Also, the file name casing of the project file seems wrong.

MichaelSimons · 2023-08-02T15:01:05Z

You checked in the project file which I think you didn't want to? Also, the file name casing of the project file seems wrong.

The package was regenerated with the latest tooling which is necessary if we are going to use it as a baseline in the tests. The casing change is because the tooling uses the package name's casing.

MichaelSimons · 2023-08-02T14:56:21Z

src/textOnlyPackages/src/microsoft.build.notargets/3.7.0/microsoft.build.notargets.nuspec

 <package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
  <metadata>
    <id>Microsoft.Build.NoTargets</id>
    <version>3.7.0</version>
    <authors>Microsoft</authors>
    <license type="expression">MIT</license>
    <licenseUrl>https://licenses.nuget.org/MIT</licenseUrl>
-    <icon>MSBuild-NuGet-Icon.png</icon>


This icon file should be deleted.

I tried deleting the icon file, but the test failed. Because the icon file is still generated by the latest tooling. So I reverted the change and kept the icon file.

ViktorHofer · 2023-08-02T17:06:24Z

The package was regenerated with the latest tooling which is necessary if we are going to use it as a baseline in the tests. The casing change is because the tooling uses the package name's casing.

I see. So the old checked-in project file had an incorrect casing. Sounds good then.

This reverts commit 86c602f.

Add SBRPTests

9cc191b

v-chayan requested review from MichaelSimons, mthalman and oleksandr-didyk as code owners June 1, 2023 06:27

MichaelSimons reviewed Jun 1, 2023

View reviewed changes

ViktorHofer reviewed Jun 2, 2023

View reviewed changes

eng/Versions.props Outdated Show resolved Hide resolved

ViktorHofer reviewed Jun 2, 2023

View reviewed changes

tests/SBRPTests/SBRPTests.csproj Outdated Show resolved Hide resolved

ViktorHofer reviewed Jun 2, 2023

View reviewed changes

tests/SBRPTests/SBRPTests.csproj Outdated Show resolved Hide resolved

ViktorHofer requested changes Jun 2, 2023

View reviewed changes

v-chayan and others added 2 commits June 5, 2023 14:54

Update

341b5c0

Generate package to sandbox instead of repo src

a44a595

ViktorHofer reviewed Jun 5, 2023

View reviewed changes

tests/GenerateScriptRegressionTests/GenerateScriptRegressionTests.Tests.csproj Outdated Show resolved Hide resolved

ViktorHofer reviewed Jun 5, 2023

View reviewed changes

tests/GenerateScriptRegressionTests/ExecuteHelper.cs Outdated Show resolved Hide resolved

MichaelSimons reviewed Jun 5, 2023

View reviewed changes

Optimize according to review comments

3317b3e

ViktorHofer reviewed Jun 6, 2023

View reviewed changes

tests/GenerateScriptTests/GenerateScriptTests.Tests.csproj Outdated Show resolved Hide resolved

ViktorHofer reviewed Jun 6, 2023

View reviewed changes

tests/GenerateScriptTests/CompareDirs.cs Outdated Show resolved Hide resolved

ViktorHofer reviewed Jun 6, 2023

View reviewed changes

tests/GenerateScriptTests/CompareDirs.cs Outdated Show resolved Hide resolved

ViktorHofer reviewed Jun 6, 2023

View reviewed changes

tests/GenerateScriptTests/CompareDirs.cs Outdated Show resolved Hide resolved

MichaelSimons reviewed Jun 6, 2023

View reviewed changes

root and others added 10 commits June 7, 2023 14:50

Update

1d35990

Merge branch 'main' into AddSBRPTests

c059ae8

Integrate tests into pipline

a02a301

Fix running tests on Windows

71b62ef

Test in Pipeline attemption

53e47a4

update

58b7d1f

update

538c64e

Fix job name

cea8a98

update

2a1f106

update

40fc1ef

v-chayan and others added 4 commits June 20, 2023 10:54

Merge branch 'main' into AddSBRPTests

87aaf62

Use git diff for comparison

eaaf94a

Merge branch 'main' into AddSBRPTests

638f270

update

33c0c0d

v-chayan requested a review from ViktorHofer June 29, 2023 08:23

root and others added 3 commits June 29, 2023 16:32

update

17f7633

undo incorrect change

e93b6ef

Merge branch 'main' into AddSBRPTests

d1dfd1c

ViktorHofer reviewed Jul 10, 2023

View reviewed changes

test.sh Outdated Show resolved Hide resolved

ViktorHofer reviewed Jul 10, 2023

View reviewed changes

azure-pipelines/builds/ci.yml Outdated Show resolved Hide resolved

ViktorHofer reviewed Jul 10, 2023

View reviewed changes

test.cmd Show resolved Hide resolved

Winniexu01 added 2 commits July 25, 2023 14:15

Update

9af641e

Update

4515e0f

ViktorHofer approved these changes Jul 31, 2023

View reviewed changes

ViktorHofer requested a review from MichaelSimons July 31, 2023 10:40

MichaelSimons reviewed Aug 1, 2023

View reviewed changes

Winniexu01 added 2 commits August 2, 2023 17:27

Update

3fb54d4

Generate latest text package

3695786

MichaelSimons approved these changes Aug 2, 2023

View reviewed changes

ViktorHofer requested changes Aug 2, 2023

View reviewed changes

MichaelSimons requested changes Aug 2, 2023

View reviewed changes

Winniexu01 and others added 3 commits August 3, 2023 09:22

Delete icon file

86c602f

Revert "Delete icon file"

e88cc70

This reverts commit 86c602f.

Merge branch 'main' into AddSBRPTests

78d36af

ViktorHofer approved these changes Aug 3, 2023

View reviewed changes

ViktorHofer enabled auto-merge (squash) August 3, 2023 13:59

MichaelSimons approved these changes Aug 3, 2023

View reviewed changes

ViktorHofer merged commit 21584ce into dotnet:main Aug 3, 2023

	new object[] { "System.Security.Cryptography.Pkcs", "7.0.2", PackageType.Reference },
	new object[] { "System.Security.Cryptography.Pkcs", "7.0.2", PackageType.Reference },
	new object[] { "Microsoft.Build.NoTargets", "3.7.0", PackageType.Text },

Add SBRPTests #689

Add SBRPTests #689

Uh oh!

Conversation

v-chayan commented Jun 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MichaelSimons left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ViktorHofer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

MichaelSimons left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MichaelSimons left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MichaelSimons Aug 1, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ViktorHofer left a comment

Choose a reason for hiding this comment

Uh oh!

MichaelSimons commented Aug 2, 2023

Uh oh!

MichaelSimons Aug 2, 2023

Choose a reason for hiding this comment

Uh oh!

Winniexu01 Aug 3, 2023

Choose a reason for hiding this comment

Uh oh!

ViktorHofer commented Aug 2, 2023

Uh oh!

Uh oh!

v-chayan commented Jun 1, 2023 •

edited

Loading