-
Notifications
You must be signed in to change notification settings - Fork 552
New unit test runner #1075
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
New unit test runner #1075
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| <?xml version="1.0" encoding="utf-8"?> | ||
| <Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> | ||
| <PropertyGroup> | ||
| <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration> | ||
| <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform> | ||
| <ProjectGuid>{910DCD88-D50A-4AAD-BA7A-CD51AB8532BF}</ProjectGuid> | ||
| <ProjectTypeGuids>{EFBA0AD7-5A72-4C68-AF49-83D382785DCF};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids> | ||
| <OutputType>Library</OutputType> | ||
| <RootNamespace>LocalTests.NUnit</RootNamespace> | ||
| <AssemblyName>LocalTests.NUnit</AssemblyName> | ||
| <TargetFrameworkVersion>v8.1</TargetFrameworkVersion> | ||
| <AndroidResgenFile>Resources\Resource.designer.cs</AndroidResgenFile> | ||
| <AndroidResgenClass>Resource</AndroidResgenClass> | ||
| <MonoAndroidResourcePrefix>Resources</MonoAndroidResourcePrefix> | ||
| <MonoAndroidAssetsPrefix>Assets</MonoAndroidAssetsPrefix> | ||
| <AndroidUseLatestPlatformSdk>true</AndroidUseLatestPlatformSdk> | ||
| </PropertyGroup> | ||
| <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' "> | ||
| <DebugSymbols>true</DebugSymbols> | ||
| <DebugType>full</DebugType> | ||
| <Optimize>false</Optimize> | ||
| <OutputPath>bin\Debug</OutputPath> | ||
| <DefineConstants>DEBUG;</DefineConstants> | ||
| <ErrorReport>prompt</ErrorReport> | ||
| <WarningLevel>4</WarningLevel> | ||
| <AndroidLinkMode>None</AndroidLinkMode> | ||
| </PropertyGroup> | ||
| <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' "> | ||
| <DebugSymbols>true</DebugSymbols> | ||
| <DebugType>pdbonly</DebugType> | ||
| <Optimize>true</Optimize> | ||
| <OutputPath>bin\Release</OutputPath> | ||
| <ErrorReport>prompt</ErrorReport> | ||
| <WarningLevel>4</WarningLevel> | ||
| <AndroidManagedSymbols>true</AndroidManagedSymbols> | ||
| <AndroidUseSharedRuntime>false</AndroidUseSharedRuntime> | ||
| </PropertyGroup> | ||
| <ItemGroup> | ||
| <Reference Include="System" /> | ||
| <Reference Include="System.Xml" /> | ||
| <Reference Include="System.Core" /> | ||
| <Reference Include="Mono.Android" /> | ||
| <Reference Include="Xamarin.Android.NUnitLite" /> | ||
| <Reference Include="System.Net" /> | ||
| <Reference Include="System.Net.Http" /> | ||
| </ItemGroup> | ||
| <ItemGroup> | ||
| <Compile Include="Resources\Resource.designer.cs" /> | ||
| <Compile Include="Properties\AssemblyInfo.cs" /> | ||
| <Compile Include="System.Net\NetworkChangeTest.cs" /> | ||
| <Compile Include="System.Net.Http\HttpClientTest.cs" /> | ||
| </ItemGroup> | ||
| <ItemGroup> | ||
| <None Include="Resources\AboutResources.txt" /> | ||
| </ItemGroup> | ||
| <ItemGroup> | ||
| <AndroidResource Include="Resources\values\Strings.xml" /> | ||
| </ItemGroup> | ||
| <Import Project="$(MSBuildExtensionsPath)\Xamarin\Android\Xamarin.Android.CSharp.targets" /> | ||
| </Project> |
7 changes: 3 additions & 4 deletions
7
...roid.Bcl-Tests/Properties/AssemblyInfo.cs → ...calTests.NUnit/Properties/AssemblyInfo.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
4 changes: 4 additions & 0 deletions
4
tests/BCL-Tests/LocalTests.NUnit/Resources/values/Strings.xml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| <?xml version="1.0" encoding="utf-8"?> | ||
| <resources> | ||
| <string name="library_name">LocalTests.NUnit</string> | ||
| </resources> |
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| using Android.App; | ||
| using Android.Widget; | ||
| using Android.OS; | ||
|
|
||
| namespace UnitTestRunner | ||
| { | ||
| [Activity(Label = "Xamarin.Android UnitTestRunner", MainLauncher = true, Icon = "@mipmap/icon")] | ||
| public class MainActivity : Activity | ||
| { | ||
| int count = 1; | ||
|
|
||
| protected override void OnCreate(Bundle savedInstanceState) | ||
| { | ||
| base.OnCreate(savedInstanceState); | ||
|
|
||
| // Set our view from the "main" layout resource | ||
| SetContentView(Resource.Layout.Main); | ||
|
|
||
| // Get our button from the layout resource, | ||
| // and attach an event to it | ||
| Button button = FindViewById<Button>(Resource.Id.myButton); | ||
|
|
||
| button.Click += delegate { button.Text = $"{count++} clicks!"; }; | ||
| } | ||
| } | ||
| } | ||
|
|
87 changes: 87 additions & 0 deletions
87
tests/BCL-Tests/Xamarin.Android.Bcl-Tests/NUnitInstrumentation.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| using System; | ||
| using System.Collections.Generic; | ||
| using System.IO; | ||
| using System.Reflection; | ||
| using System.Text; | ||
|
|
||
| using Android.App; | ||
| using Android.Runtime; | ||
| using Xamarin.Android.UnitTests; | ||
| using Xamarin.Android.UnitTests.NUnit; | ||
|
|
||
| namespace UnitTestRunner | ||
| { | ||
| [Instrumentation (Name = "xamarin.android.bcltests.NUnitInstrumentation")] | ||
| public class NUnitInstrumentation : NUnitTestInstrumentation | ||
| { | ||
| const string DefaultLogTag = "NUnit"; | ||
|
|
||
| string logTag = DefaultLogTag; | ||
|
|
||
| protected override string LogTag { | ||
| get { return logTag; } | ||
| set { logTag = value ?? DefaultLogTag; } | ||
| } | ||
|
|
||
| protected NUnitInstrumentation (IntPtr handle, JniHandleOwnership transfer) : base (handle, transfer) | ||
| { | ||
| CommonInit (); | ||
| } | ||
|
|
||
| public NUnitInstrumentation () | ||
| { | ||
| CommonInit (); | ||
| } | ||
|
|
||
| void CommonInit () | ||
| { | ||
| TestAssembliesGlobPattern = "*_test.dll"; | ||
| string cacheDir = Path.Combine (Application.Context.CacheDir.AbsolutePath, DefaultLogTag); | ||
| using (var files = typeof (NUnitInstrumentation).Assembly.GetManifestResourceStream ("bcl-tests.zip")) { | ||
| ExtractAssemblies (cacheDir, files); | ||
| } | ||
|
|
||
| TestsDirectory = cacheDir; | ||
| Environment.CurrentDirectory = cacheDir; | ||
| var excluded = new List<string> { | ||
| "AndroidNotWorking", | ||
| "CAS", | ||
| "InetAccess", | ||
| "MobileNotWorking", | ||
| "NotWorking", | ||
| }; | ||
|
|
||
| if (!Environment.Is64BitOperatingSystem) | ||
| excluded.Add ("LargeFileSupport"); | ||
|
|
||
| ExcludedCategories = excluded; | ||
| } | ||
|
|
||
| protected override void ConfigureFilters(NUnitTestRunner runner) | ||
| { | ||
| HashSet<string> excludedTestNames = null; | ||
| using (var s = typeof (NUnitInstrumentation).Assembly.GetManifestResourceStream ("nunit-excluded-tests.txt")) { | ||
| using (var sr = new StreamReader (s, Encoding.UTF8)) { | ||
| excludedTestNames = LoadExcludedTests (sr); | ||
| } | ||
| } | ||
|
|
||
| ExcludedTestNames = excludedTestNames; | ||
|
|
||
| base.ConfigureFilters(runner); | ||
| } | ||
|
|
||
| protected override IList<TestAssemblyInfo> GetTestAssemblies() | ||
| { | ||
| IList<TestAssemblyInfo> ret = base.GetTestAssemblies(); | ||
|
|
||
| if (ret == null) | ||
| ret = new List<TestAssemblyInfo> (); | ||
|
|
||
| Assembly asm = typeof (BclTests.HttpClientTest).Assembly; | ||
| ret.Add (new TestAssemblyInfo (asm, asm.Location ?? String.Empty)); | ||
|
|
||
| return ret; | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
27 changes: 27 additions & 0 deletions
27
tests/BCL-Tests/Xamarin.Android.Bcl-Tests/Properties/AssemblyInfo.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| using System.Reflection; | ||
| using System.Runtime.CompilerServices; | ||
| using Android.App; | ||
|
|
||
| // Information about this assembly is defined by the following attributes. | ||
| // Change them to the values specific to your project. | ||
|
|
||
| [assembly: AssemblyTitle("xUnitTestRunner")] | ||
| [assembly: AssemblyDescription("")] | ||
| [assembly: AssemblyConfiguration("")] | ||
| [assembly: AssemblyCompany("")] | ||
| [assembly: AssemblyProduct("")] | ||
| [assembly: AssemblyCopyright("Microsoft, Inc")] | ||
| [assembly: AssemblyTrademark("")] | ||
| [assembly: AssemblyCulture("")] | ||
|
|
||
| // The assembly version has the format "{Major}.{Minor}.{Build}.{Revision}". | ||
| // The form "{Major}.{Minor}.*" will automatically update the build and revision, | ||
| // and "{Major}.{Minor}.{Build}.*" will update just the revision. | ||
|
|
||
| [assembly: AssemblyVersion("1.0.0")] | ||
|
|
||
| // The following attributes are used to specify the signing key for the assembly, | ||
| // if desired. See the Mono documentation for more information about signing. | ||
|
|
||
| //[assembly: AssemblyDelaySign(false)] | ||
| //[assembly: AssemblyKeyFile("")] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need
Packagehere? It looks like it is duplicated in all the cases.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we do - each instrumentation is assigned to a single package, think of it as a relation between tables in SQL. It's also easier to write the batched task execution this way. It's cleaner (and yeasier) this way than to create a comma or semicolon separated list in the metadata item for
TestApkUh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see it only used in the https://github.com/xamarin/xamarin-android/pull/1075/files#diff-6969d1e0d6b6392b34ee6cdcf770c84f and the Package is the same.
Wouldn't it be at least nice to have the
Packageset in a property, so that it can be set at single place?Like:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Package name in property name would be nice but it doesn't work in xbuild - been there done that :) It is also used here. It's the only sensible mechanism to support multiple instrumentations per package. We can't put the information in
TestApkelement and there's also no sane way to "link"<TestApkInstrumetation>to a particularTestApkinstance.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know why would it not work on xbuild. We did it before and it worked, like here: https://github.com/grendello/xamarin-android/blob/385699a58635d87a1cc65b60995d3420995f21d7/src/Mono.Android/Test/Mono.Android-Tests.projitems
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried that approach and, unfortunately, this is still broken. Using a property instead of the string makes
UndeployApkTeststarget fail to process half of them when batching - this is the issue I had sometime ago while working on mono runtime builds. So, as much as it would be a nice change, we can't do it right now :(There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is weird, it worked not so long time ago. What error do you get?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No error at all... Half of the items using that property are simply ignored (in my case in
UndeployApksandDeployApkswhich iterate over@(TestApk)) - this is the behavior I saw with the runtime stuff. It appears that sometimes xbuild fails to process properties inside item metadata.