Skip to content

[iOS][libraries] Skip failing iOS tests with ActiveIssues and proj level skips #51491

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 78 commits into from
May 12, 2021
Merged

[iOS][libraries] Skip failing iOS tests with ActiveIssues and proj level skips #51491

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
af145aa
[iOS][tvOS][libraries] Add Microsoft.Extensions.Hosting.Unit.Tests cl…
Apr 28, 2021
40e7e09
[iOS][tvOS][libraries] Add Microsoft.Extensions.Logging.Console.Tests…
Apr 16, 2021
4d73eef
[iOS][tvOS][libraries] Add Microsoft.VisualBasic.Core.Tests ActiveIss…
Apr 16, 2021
ef1b757
[iOS][tvOS][libraries] Add System.Collections.Immutable.Tests ActiveI…
Apr 13, 2021
2aa718e
[iOS][tvOS][libraries] Add System.Console.Tests PNSE ActiveIssues 36878
Apr 14, 2021
21e9d29
[iOS][tvOS][libraries] Add System.Data.Common.Tests ActiveIssues 36879
Apr 14, 2021
0246404
[iOS][tvOS][libraries] Add System.Diagnostics.Debug.Tests ActiveIssue…
Apr 14, 2021
19f342a
[iOS][tvOS][libraries] Add System.Diagnostics.Process.Tests ActiveIss…
Apr 16, 2021
9d880b6
[iOS][tvOS][libraries] Add System.Diagnostics.StackTrace.Tests Active…
Apr 16, 2021
0818061
[iOS][tvOS][libraries] Add System.Diagnostics.Tracing.Tests ActiveIss…
Apr 16, 2021
9ec775f
[iOS][tvOS][libraries] Add System.IO.Compression.Brotli.Tests ActiveI…
Apr 14, 2021
6e95dc4
[iOS][tvOS][libraries] Add System.IO.Compression.Tests test project skip
Apr 16, 2021
520dee3
[iOS][tvOS][libraries] Add System.IO.Compression.Tests ActiveIssue 51376
Apr 16, 2021
b75b57b
[iOS][tvOS][libraries] Add System.IO.FileSystem.Watcher.Tests ActiveI…
Apr 16, 2021
3355723
[iOS][tvOS][libraries] Add System.IO.MemoryMappedFiles.Tests ActiveIs…
Apr 16, 2021
0f72ebc
[iOS][tvOS][libraries] Add System.IO.Tests ActiveIssue 51390
Apr 16, 2021
e9e55eb
[iOS][tvOS][libraries] Add System.Net.NameResolution.Functional.Tests…
Apr 16, 2021
4a798e9
[iOS][tvOS][libraries] Add System.Net.NetworkInformation.Functional.T…
Apr 14, 2021
bf8d939
[iOS][tvOS][libraries] Add System.Reflection.Tests ActiveIssues 36892
Apr 15, 2021
e87df9b
[iOS][tvOS][libraries] Add System.Resources.ResourceManager.Tests Act…
Apr 15, 2021
bb2894b
[iOS][tvOS][libraries] Add System.Runtime.Extensions.Tests ActiveIssu…
Apr 15, 2021
300f063
[iOS][tvOS][libraries] Add System.Security.Cryptography.Encoding.Test…
Apr 16, 2021
0c7f4e1
[iOS][libraries] Add System.Security.Cryptograph.OpenSsl.Tests test p…
Apr 16, 2021
0040180
[iOS][tvOS][libraries] Add System.Text.Encoding.Tests ActiveIssue 51394
Apr 16, 2021
2b332e3
[iOs][tvOS][libraries] Add System.Text.RegularExpressions.Tests Activ…
Apr 15, 2021
476be16
[iOS][tvOS][libraries] Add System.Threading.Tests ActiveIssue 51400
Apr 16, 2021
b8bdc81
[iOS][tvOS][libraries] Remove dated test exclusions
Apr 16, 2021
4bce18f
[iOS][tvOS][libraries] Skip test suites that crash or hang in iOS/tvOS
Apr 19, 2021
ba9614e
[tvOS] Add System.Linq.Expressions.Tests hangs ActiveIssue
Apr 28, 2021
a22fc0b
[tvOS] Add System.Memory.Tests hangs ActiveIssue
Apr 28, 2021
aee8f62
[iOS][tvOS] Add System.Net.Sockets.Tests hangs ActiveIssue
Apr 28, 2021
aa69c73
[iOS][libraries] Remove ios target from System.IO.Ports
Apr 22, 2021
09be804
[iOS][tvOS] Add System.Runtime.Tests ActiveIssue 52072
Apr 29, 2021
757023e
[iOS][tvOS] Add System.Threading.Thread.Tests PNSE ActiveIssue
Apr 29, 2021
263f836
[iOS][tvOS] Refactor test suite skips
Apr 29, 2021
754d6fc
[iOS][tvOS] Add System.Net.Quic.Functional.Tests test proj skip
Apr 30, 2021
77692f2
[iOS][tvOS] Add issues for hanging and crashed test suites
Apr 30, 2021
23bf1c2
[iOS][tvOS] Add System.Net.Sockets.Tests ActiveIssue 52124
Apr 30, 2021
854b0b1
[iOS][tvOS] Update System.Security.Cryptography.Csp.Tests ActiveIssue
Apr 30, 2021
4c1bace
[iOS][tvOS] Update System.Cryptography.Algorithms ActiveIssue
Apr 30, 2021
c73da54
[iOS][tvOS] Add System.Console PNSE tests to SkipPlatform
tqiu8 Apr 30, 2021
e023e03
[iOS][tvOS] Add System.Security.Cryptography RountripEmptyArray flake…
May 3, 2021
fa253f8
[iOS][tvOS] Add Microsoft.Extensions.Hosting remaining ActiveIssue
May 3, 2021
88e4012
[iOS][tvOS] Add Microsoft.Extensions.Hosting.Unit.Tests SkipOnPlatform
May 3, 2021
a55f286
[iOS][tvOS] Update System.Diagnostics.Process ActiveIssue to SkipOnPl…
tqiu8 May 4, 2021
efadd89
[iOS][tvOS] Convert ActiveIssue to SkipOnPlatform for PNSE related te…
tqiu8 May 4, 2021
72e5e27
[iOS][tvOS] Move SkipOnPlatform attribute for System.Console SyncText…
tqiu8 May 4, 2021
af9677e
[iOS][tvOS] Add System.Net.Sockets.Tests PNSE SkipOnPlatform
May 3, 2021
610058d
[iOS][tvOS] Add System.Threading.Thread.Tests PNSE SkipOnPlatform
May 3, 2021
660a03f
[iOS][tvOS] Add System.Runtime.Extensions.Tests PNSE SkipOnPlatform
May 3, 2021
d3fdbe9
[iOS][tvOS] Add System.Security.Cryptography.Xml.Tests PNSE SkipOnPla…
May 3, 2021
5bb9c58
[iOS][tvOS] Add System.Security.Cryptography.Xml.Tests ActiveIssue 51370
May 3, 2021
9eb7e60
[iOS][tvOS] System.IO.FileSystem.Tests crashes with process terminated
May 4, 2021
1d422e9
[iOS][tvOS] Add System.IO.FileSystem.Tests SkipOnPlatform
May 4, 2021
a0ff63e
[iOS][tvOS] Add System.IO.FileSystem.Tests ActiveIssue
May 4, 2021
580308d
[iOS][tvOS] Update test suites skipped due to PNSE
May 5, 2021
e7a4d5d
[iOS][tvOS] Remove Microsoft.Extensions.HostFactoryResolver and Syste…
May 5, 2021
5f65e18
[iOS][tvOS] Reenable System.Diagnostics.Process.Tests ActiveIssue and…
May 5, 2021
5c48370
[tvOS] Skip tvOS.Simulator.Aot.Test due to linking issue
May 6, 2021
3f044b9
[iOS][tvOS] Skip ipv6 System.Net.Sockets.Tests PNSE
May 7, 2021
9a3155a
[iOS][tvOS] Update System.Threading.Thread.Tests PNSE SkipOnPlatform
May 7, 2021
662e4cd
[iOS][tvOS] Add System.Runtime.Extensions.Tests UserName_Valid Active…
May 7, 2021
093db84
[iOS][tvOS][Catalyst] Update System.IO.FileSystem SkipOnPlatform
May 7, 2021
80fcf69
[iOS][tvOS] Remove System.Xml.XmlSchemaSet.Tests test proj skip
May 7, 2021
acac640
[iOS][tvOS] Remove System.IO.Compression.Tests test proj skip
May 7, 2021
efe09f4
[MacCatalyst] Amend all iOS tvOS ActiveIssues to include MacCatalyst
May 7, 2021
b0b1a8c
[iOS][tvOS] Remove System.Xml.RW.XmlReader.Tests ActiveIssue 49187
May 10, 2021
e5d9708
[libraries] Undo incorrect SkipOnPlatform change
May 10, 2021
287b6aa
Temporarily skip iOS Simulator AOT functional test
May 11, 2021
755f1f9
[iOS][tvOS][libraries] Add System.Threading.Tasks.Tests ActiveIssue 5…
May 11, 2021
0dbecd9
[iOS][tvOS] Skip suites crashing on CI that pass locally
May 11, 2021
ccd9398
[iOS][tvOS] Add System.IO.Compression.ZipFile.Tests ActiveIssue faili…
May 11, 2021
e28bbde
[iOS][tvOS] Add System.Net.Ping.Functional.Tests ActiveIssue failing …
May 11, 2021
2aa52ed
Fix typo
May 11, 2021
f01f3cf
[iOS][tvOS][macCatalyst] Temporary test suite skips to get checks pas…
May 12, 2021
31ac47c
[iOS][tvOS] Add System.IO.Packaging and System.Security.Cryptography.…
May 12, 2021
488a6f7
Fix typo in SkipOnPlatform in System.Runtime.Extensions
May 12, 2021
fe031a4
[iOS][tvOS] Add GitHub issue to System.Linq.Expressions.Tests tvOS Ac…
May 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions src/libraries/Common/tests/StreamConformanceTests/System/IO/StreamConformanceTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -711,6 +711,7 @@ public abstract class StandaloneStreamConformanceTests : StreamConformanceTests
}

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ArgumentValidation_ThrowsExpectedException()
{
await foreach (Stream? stream in GetStreamsForValidation())
Expand All @@ -724,6 +725,7 @@ public virtual async Task ArgumentValidation_ThrowsExpectedException()
}

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task Disposed_ThrowsObjectDisposedException()
{
await foreach (Stream? stream in GetStreamsForValidation())
Expand Down Expand Up @@ -813,6 +815,7 @@ public virtual async Task Read_DataStoredAtDesiredOffset(ReadWriteMode mode)
[InlineData(ReadWriteMode.SyncArray)]
[InlineData(ReadWriteMode.AsyncArray)]
[InlineData(ReadWriteMode.AsyncAPM)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task Write_DataReadFromDesiredOffset(ReadWriteMode mode)
{
using Stream? stream = await CreateReadWriteStream();
Expand Down Expand Up @@ -1601,6 +1604,7 @@ protected static bool Bidirectional(StreamPair streams) =>
streams.Stream2.CanRead && streams.Stream2.CanWrite;

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ArgumentValidation_ThrowsExpectedException()
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand All @@ -1612,6 +1616,7 @@ public virtual async Task ArgumentValidation_ThrowsExpectedException()
}

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task Disposed_ThrowsObjectDisposedException()
{
StreamPair streams = await CreateConnectedStreamsAsync();
Expand All @@ -1624,6 +1629,7 @@ public virtual async Task Disposed_ThrowsObjectDisposedException()
}

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ReadWriteAsync_PrecanceledOperations_ThrowsCancellationException()
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand All @@ -1635,6 +1641,7 @@ public virtual async Task ReadWriteAsync_PrecanceledOperations_ThrowsCancellatio
}

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ReadAsync_CancelPendingTask_ThrowsCancellationException()
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand All @@ -1644,6 +1651,7 @@ public virtual async Task ReadAsync_CancelPendingTask_ThrowsCancellationExceptio
}

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ReadAsync_CancelPendingValueTask_ThrowsCancellationException()
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand All @@ -1653,6 +1661,7 @@ public virtual async Task ReadAsync_CancelPendingValueTask_ThrowsCancellationExc
}

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ReadWriteByte_Success()
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand Down Expand Up @@ -1721,6 +1730,7 @@ public virtual async Task ReadWrite_Success_Large(ReadWriteMode mode, int writeS

[Theory]
[MemberData(nameof(ReadWrite_Success_MemberData))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ReadWrite_Success(ReadWriteMode mode, int writeSize, bool startWithFlush)
{
foreach (CancellationToken nonCanceledToken in new[] { CancellationToken.None, new CancellationTokenSource().Token })
Expand Down Expand Up @@ -1778,6 +1788,7 @@ public virtual async Task ReadWrite_Success(ReadWriteMode mode, int writeSize, b
[Theory]
[MemberData(nameof(AllReadWriteModesAndValue), false)]
[MemberData(nameof(AllReadWriteModesAndValue), true)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task Read_Eof_Returns0(ReadWriteMode mode, bool dataAvailableFirst)
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand Down Expand Up @@ -1816,6 +1827,7 @@ public virtual async Task Read_Eof_Returns0(ReadWriteMode mode, bool dataAvailab
[InlineData(ReadWriteMode.SyncArray)]
[InlineData(ReadWriteMode.AsyncArray)]
[InlineData(ReadWriteMode.AsyncAPM)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task Read_DataStoredAtDesiredOffset(ReadWriteMode mode)
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand Down Expand Up @@ -1845,6 +1857,7 @@ public virtual async Task Read_DataStoredAtDesiredOffset(ReadWriteMode mode)
[InlineData(ReadWriteMode.SyncArray)]
[InlineData(ReadWriteMode.AsyncArray)]
[InlineData(ReadWriteMode.AsyncAPM)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task Write_DataReadFromDesiredOffset(ReadWriteMode mode)
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand Down Expand Up @@ -1943,6 +1956,7 @@ public static IEnumerable<object[]> ReadAsync_ContinuesOnCurrentContextIfDesired

[Theory]
[MemberData(nameof(ReadAsync_ContinuesOnCurrentContextIfDesired_MemberData))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ReadAsync_ContinuesOnCurrentSynchronizationContextIfDesired(bool flowExecutionContext, bool? continueOnCapturedContext)
{
await default(JumpToThreadPoolAwaiter); // escape xunit sync ctx
Expand Down Expand Up @@ -2025,6 +2039,7 @@ public virtual async Task ReadAsync_ContinuesOnCurrentSynchronizationContextIfDe

[Theory]
[MemberData(nameof(ReadAsync_ContinuesOnCurrentContextIfDesired_MemberData))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ReadAsync_ContinuesOnCurrentTaskSchedulerIfDesired(bool flowExecutionContext, bool? continueOnCapturedContext)
{
await default(JumpToThreadPoolAwaiter); // escape xunit sync ctx
Expand Down Expand Up @@ -2114,6 +2129,7 @@ await Task.Factory.StartNew(() =>
[InlineData(ReadWriteMode.AsyncMemory)]
[InlineData(ReadWriteMode.SyncAPM)]
[InlineData(ReadWriteMode.AsyncAPM)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ZeroByteRead_BlocksUntilDataAvailableOrNops(ReadWriteMode mode)
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand Down Expand Up @@ -2176,6 +2192,7 @@ public virtual async Task ZeroByteRead_BlocksUntilDataAvailableOrNops(ReadWriteM
[InlineData(ReadWriteMode.AsyncMemory)]
[InlineData(ReadWriteMode.SyncAPM)]
[InlineData(ReadWriteMode.AsyncAPM)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ZeroByteWrite_OtherDataReceivedSuccessfully(ReadWriteMode mode)
{
byte[][] buffers = new[] { Array.Empty<byte>(), Encoding.UTF8.GetBytes("hello"), Array.Empty<byte>(), Encoding.UTF8.GetBytes("world") };
Expand Down Expand Up @@ -2229,6 +2246,7 @@ public virtual async Task ZeroByteWrite_OtherDataReceivedSuccessfully(ReadWriteM
[Theory]
[InlineData(false)]
[InlineData(true)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ReadWrite_CustomMemoryManager_Success(bool useAsync)
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand Down Expand Up @@ -2349,6 +2367,7 @@ public virtual async Task CopyToAsync_AllDataCopied_Large(bool useAsync) =>

[Theory]
[MemberData(nameof(CopyToAsync_AllDataCopied_MemberData))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task CopyToAsync_AllDataCopied(int byteCount, bool useAsync)
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand Down Expand Up @@ -2386,6 +2405,7 @@ await Task.WhenAll(Enumerable.Range(0, 20).Select(_ => Task.Run(async () =>
}

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task Timeout_Roundtrips()
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand Down Expand Up @@ -2420,6 +2440,7 @@ public virtual async Task Timeout_Roundtrips()
}

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task ReadTimeout_Expires_Throws()
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand Down Expand Up @@ -2554,6 +2575,7 @@ public virtual async Task ReadAsync_DuringReadAsync_ThrowsIfUnsupported()
}

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task Flush_ValidOnWriteableStreamWithNoData_Success()
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand All @@ -2568,6 +2590,7 @@ public virtual async Task Flush_ValidOnWriteableStreamWithNoData_Success()
}

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task Flush_ValidOnReadableStream_Success()
{
using StreamPair streams = await CreateConnectedStreamsAsync();
Expand All @@ -2585,6 +2608,7 @@ public virtual async Task Flush_ValidOnReadableStream_Success()
[InlineData(0)]
[InlineData(1)]
[InlineData(2)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51371", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task Dispose_ClosesStream(int disposeMode)
{
if (!CansReturnFalseAfterDispose)
Expand Down
10 changes: 10 additions & 0 deletions src/libraries/Common/tests/System/IO/Compression/CompressionStreamUnitTestBase.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ public virtual void FlushAsync_DuringWriteAsync()

[ConditionalFact(typeof(PlatformDetection), nameof(PlatformDetection.IsThreadingSupported))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36845", TestPlatforms.Android)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36884", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public async Task FlushAsync_DuringReadAsync()
{
byte[] buffer = new byte[32];
Expand Down Expand Up @@ -76,6 +77,7 @@ public async Task FlushAsync_DuringReadAsync()

[ConditionalFact(typeof(PlatformDetection), nameof(PlatformDetection.IsThreadingSupported))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36845", TestPlatforms.Android)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36884", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public async Task FlushAsync_DuringFlushAsync()
{
byte[] buffer = null;
Expand Down Expand Up @@ -118,6 +120,7 @@ public async Task FlushAsync_DuringFlushAsync()

[ConditionalFact(typeof(PlatformDetection), nameof(PlatformDetection.IsThreadingSupported))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36845", TestPlatforms.Android)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36884", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public virtual async Task Dispose_WithUnfinishedReadAsync()
{
string compressedPath = CompressedTestFile(UncompressedTestFile());
Expand All @@ -137,6 +140,7 @@ public virtual async Task Dispose_WithUnfinishedReadAsync()
[Theory]
[MemberData(nameof(UncompressedTestFiles))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36845", TestPlatforms.Android)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36884", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public async Task Read(string testFile)
{
var uncompressedStream = await LocalMemoryStream.readAppFileAsync(testFile);
Expand Down Expand Up @@ -173,6 +177,7 @@ public async Task Read(string testFile)

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36845", TestPlatforms.Android)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36884", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public async Task Read_EndOfStreamPosition()
{
var compressedStream = await LocalMemoryStream.readAppFileAsync(CompressedTestFile(UncompressedTestFile()));
Expand All @@ -193,6 +198,7 @@ public async Task Read_EndOfStreamPosition()

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36845", TestPlatforms.Android)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36884", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public async Task Read_BaseStreamSlowly()
{
string testFile = UncompressedTestFile();
Expand Down Expand Up @@ -328,6 +334,7 @@ IEnumerable<Func<Stream, Stream>> CtorFunctions()

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36845", TestPlatforms.Android)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36884", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public async Task TestLeaveOpenAfterValidDecompress()
{
//Create the Stream
Expand Down Expand Up @@ -403,6 +410,7 @@ public void BaseStreamTest(CompressionMode mode)
[InlineData(CompressionMode.Compress)]
[InlineData(CompressionMode.Decompress)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36845", TestPlatforms.Android)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36884", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public async Task BaseStream_Modify(CompressionMode mode)
{
using (var baseStream = await LocalMemoryStream.readAppFileAsync(CompressedTestFile(UncompressedTestFile())))
Expand Down Expand Up @@ -435,6 +443,7 @@ public void BaseStream_NullAfterDisposeWithFalseLeaveOpen(CompressionMode mode)
[InlineData(CompressionMode.Compress)]
[InlineData(CompressionMode.Decompress)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36845", TestPlatforms.Android)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36884", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public async Task BaseStream_ValidAfterDisposeWithTrueLeaveOpen(CompressionMode mode)
{
var ms = await LocalMemoryStream.readAppFileAsync(CompressedTestFile(UncompressedTestFile()));
Expand All @@ -453,6 +462,7 @@ public async Task BaseStream_ValidAfterDisposeWithTrueLeaveOpen(CompressionMode

[Theory]
[MemberData(nameof(UncompressedTestFiles))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/36884", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public async Task CompressionLevel_SizeInOrder(string testFile)
{
using var uncompressedStream = await LocalMemoryStream.readAppFileAsync(testFile);
Expand Down
1 change: 1 addition & 0 deletions ...ests/System/Security/Cryptography/AlgorithmImplementations/ECDsa/ECDsaTests.netcoreapp.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ public void KeySizeProp()
}

[Theory, MemberData(nameof(TestNewCurves))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/51332", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not be necessary anymore, fixed by AppleAppBuilder change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@filipnavara I just ran the test suite locally (based on 835f7c9) and it seems to still fail with what I think are the same failures and messages.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latest CI run also saw consistent crashes of System.Security.Cryptography.Algorithms.Tests across iOSSimulator/tvOSSimulator and MacCatalyst, we should probably disable it for now too so we can get this PR in.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akoeplinger Are they the crashes from the AesGcm, AesCcm and ChaCha20Poly1305 tests that fail to load OpenSSL? I'd very much prefer to disable just the three test classes. #52598

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The AesGcm, AesCcm, ChaChaPoly1305 shouldn't even by trying to load the "real" implementation on non-macOS platforms since they are backed by OpenSSL. If these are indeed the cause then we should fix the project file so that we end up coming AesGcm.NotSupported.cs instead of AesGcm.Unix.cs for these platforms.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vcsjones Yes, it should totally be done. At the moment iOS/tvOS/MacCatalyst consume the same implemention as macOS. I didn't want to touch it since the code was still very much in flux for the past weeks due to introduction of ChaChaPoly.

Until that is done properly I would still prefer not to disable the whole test suite but just the three affected classes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@filipnavara https://gist.github.com/mdh1418/7ad77945032249078a8d438fe9321c25 I've been seeing problems with ChaCha20Poly1305 issues when running System.Security.Cryptography.Algorithms.Tests locally

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mdh1418 The AesCcm, AesGcm and ChaCha20Poly1305 classes are supposed to throw PNSE on iOS-like platforms. The main reason why I didn't submit PR for it is that ChaChaPoly is new code and the same PR introduced AesCcm.IsSupported and AesGcm.IsSupported properties. The tests are now guarded by the new properties but the implementation still incorrectly returns true on iOS and/or attempts to load the OpenSSL library. In some cases it results in loading the native OpenSSL shim library which in turn asserts in native code because real OpenSSL is not available. The assert causes the whole test suite to crash.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviously the correct fix is to fix all the .IsSupported methods to return false but since there are still open issues with the new code and @vcsjones attacks the same API surface I didn't want to interfere with the in-flight PRs.

We can do surgical workaround by disabling just the three test classes and re-enabling it once the dust settles and the implementation is fixed properly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup sounds good, but we'll still go ahead and merge this PR for now to get to a green state in runtime-staging :)

public void TestRegenKeyExplicit(CurveDef curveDef)
{
ECParameters param, param2;
Expand Down
1 change: 1 addition & 0 deletions .../Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -338,6 +338,7 @@ private void RsaCryptRoundtrip(RSAEncryptionPadding paddingMode, bool expectSucc

[Fact]
[SkipOnTargetFramework(TargetFrameworkMonikers.NetFramework)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/52199", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public void RoundtripEmptyArray()
{
using (RSA rsa = RSAFactory.Create(TestData.RSA2048Params))
Expand Down
1 change: 1 addition & 0 deletions src/libraries/Common/tests/Tests/System/StringTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7300,6 +7300,7 @@ public static void InternalTestAotSubset()

[ConditionalFact(typeof(PlatformDetection), nameof(PlatformDetection.IsNotInvariantGlobalization))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/34577", TestPlatforms.Windows, TargetFrameworkMonikers.Netcoreapp, TestRuntimes.Mono)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/52072", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public static unsafe void NormalizationTest() // basic test; more tests in globalization tests
{
// U+0063 LATIN SMALL LETTER C
Expand Down
1 change: 1 addition & 0 deletions src/libraries/Microsoft.Extensions.Hosting/tests/UnitTests/HostBuilderTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,7 @@ public void HostConfigParametersReadCorrectly()
}

[Fact]
[ActiveIssue("https://github.com/dotnet/runtime/issues/52114", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public void RelativeContentRootIsResolved()
{
using (var host = new HostBuilder()
Expand Down
1 change: 1 addition & 0 deletions src/libraries/Microsoft.Extensions.Hosting/tests/UnitTests/HostTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ namespace Microsoft.Extensions.Hosting.Tests
public partial class HostTests
{
[Fact]
[SkipOnPlatform(TestPlatforms.iOS | TestPlatforms.tvOS, "Not supported on iOS or tvOS.")]
public async Task StopAsyncWithCancellation()
{
var builder = new HostBuilder();
Expand Down
1 change: 1 addition & 0 deletions src/libraries/Microsoft.Extensions.Hosting/tests/UnitTests/Internal/HostTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

namespace Microsoft.Extensions.Hosting.Internal
{
[ActiveIssue("https://github.com/dotnet/runtime/issues/52114", TestPlatforms.iOS | TestPlatforms.tvOS | TestPlatforms.MacCatalyst)]
public class HostTests
{
[Fact]
Expand Down
Loading