[mono][tvos] DataSet Xml serialization tests failing on tvos lanes

[ivanpovazan]

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=896707
Affected pipeline: runtime-extra-platforms
Job name: tvos-arm64 Release AllSubsets_Mono
Build error leg or test failing: System.Data.Common.Tests

Crash details

[FAIL] System.Data.Tests.DataSetTest.DeserializeModifiedDataSet
[16:49:48.6707670] System.InvalidOperationException : There was an error generating the XML document.
[16:49:48.6726660] 2024-12-16 21:49:48.709 System.Data.Common.Tests[47911:134275559] ---- System.NotSupportedException : DataSet implementation of IXmlSerializable is not trim compatible and has been disabled in the app configuration.
[16:49:48.6727080]    at System.Xml.Serialization.XmlSerializer.Serialize(XmlWriter xmlWriter, Object o, XmlSerializerNamespaces namespaces, String encodingStyle, String id)
[16:49:48.6727580] 2024-12-16 21:49:48.710 System.Data.Common.Tests[47911:134275559]    at System.Xml.Serialization.XmlSerializer.Serialize(XmlWriter xmlWriter, Object o)
[16:49:48.6727850]    at System.Data.Tests.DataSetTest.DeserializeModifiedDataSet()
[16:49:48.6727960]    at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
[16:49:48.6728360] 2024-12-16 21:49:48.710 System.Data.Common.Tests[47911:134275559]    at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
[16:49:48.6728680] ----- Inner Stack Trace -----
[16:49:48.6728810]    at System.Data.DataSet.System.Xml.Serialization.IXmlSerializable.WriteXml(XmlWriter writer)
[16:49:48.6729520] 2024-12-16 21:49:48.710 System.Data.Common.Tests[47911:134275559]    at System.Xml.Serialization.XmlSerializationWriter.WriteSerializable(IXmlSerializable serializable, String name, String ns, Boolean isNullable, Boolean wrapped)
[16:49:48.6729820]    at System.Xml.Serialization.ReflectionXmlSerializationWriter.WriteElement(Object o, ElementAccessor element, Boolean writeAccessor)
[16:49:48.6731190] 2024-12-16 21:49:48.710 System.Data.Common.Tests[47911:134275559]    at System.Xml.Serialization.ReflectionXmlSerializationWriter.WriteElements(Object o, ElementAccessor[] elements, TextAccessor text, ChoiceIdentifierAccessor choice, Boolean writeAccessors, Boolean isNullable)
[16:49:48.6732140] 2024-12-16 21:49:48.710 System.Data.Common.Tests[47911:134275559]    at System.Xml.Serialization.ReflectionXmlSerializationWriter.WriteMember(Object o, Object choiceSource, ElementAccessor[] elements, TextAccessor text, ChoiceIdentifierAccessor choice, TypeDesc memberTypeDesc, Boolean writeAccessors)
[16:49:48.6732380]    at System.Xml.Serialization.ReflectionXmlSerializationWriter.GenerateTypeElement(Object o, XmlTypeMapping xmlMapping)
[16:49:48.6732480] 2024-12-16 21:49:48.710 System.Data.Common.Tests[47911:134275559]    at System.Xml.Serialization.ReflectionXmlSerializationWriter.WriteObject(Object o)
[16:49:48.6732590]    at System.Xml.Serialization.XmlSerializer.SerializeUsingReflection(XmlWriter xmlWriter, Object o, XmlSerializerNamespaces namespaces, String encodingStyle, String id)
[16:49:48.6732710] 2024-12-16 21:49:48.711 System.Data.Common.Tests[47911:134275559]    at System.Xml.Serialization.XmlSerializer.Serialize(XmlWriter xmlWriter, Object o, XmlSerializerNamespaces namespaces, String encodingStyle, String id)
[16:49:48.6791230] 2024-12-16 21:49:48.716 System.Data.Common.Tests[47911:134275559]

ConsoleLog
CrashLog

Root cause

This seems to be introduced with #110477 more specifically: a149938

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorPattern": ["Failed: [1-9]+[0-9]*", "net.dot.System.Data.Common.Tests"],
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=896707
Error message validated: [Failed: [1-9]\d*.*(net\.dot\.System\.Data\.Common\.Tests)+]
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 12/17/2024 9:01:33 AM UTC

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=896707
Error message validated: [(Failed: )+[1-9]\d*.*(net\.dot\.System\.Data\.Common\.Tests)+]
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 12/17/2024 9:27:50 AM UTC

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=896707
Error message validated: [Failed: [1-9]\d*.*net\.dot\.System\.Data\.Common\.Tests]
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 12/17/2024 9:34:35 AM UTC

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=896707
Error message validated: [Failed: [1-9]+[0-9]* net.dot.System.Data.Common.Tests]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 12/17/2024 9:43:23 AM UTC

Report

Build	Definition	Test	Pull Request
896707	dotnet/runtime	System.Data.Common.Tests.WorkItemExecution

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	1

[ivanpovazan]

/cc: @akoeplinger

[akoeplinger]

Hmm this shouldn't have happened because these tests are marked with ConditionalFact that uses the same feature switch as the NotSupportedException:

runtime/src/libraries/System.Data.Common/tests/System/Data/DataTableTest5.cs

Line 787 in 9482c1e

[ConditionalFact(typeof(PlatformDetection), nameof(PlatformDetection.DataSetXmlSerializationIsSupported))]

Can you check whether the AppConfig gets generated correctly?

[akoeplinger]

I wonder if we need to add the msbuild property to:

runtime/eng/testing/tests.mobile.targets

Lines 43 to 48 in 69d8076

	<!-- When tests are built on Helix, override default trimming switches -->
	<PropertyGroup Condition="'$(EnableAggressiveTrimming)' == 'true' and '$(BuildTestsOnHelix)' == 'true'">
	<DynamicCodeSupport Condition="'$(DynamicCodeSupport)' == '' and '$(MonoForceInterpreter)' != 'true'">false</DynamicCodeSupport>
	<_DefaultValueAttributeSupport Condition="'$(OverrideDefaultValueAndDesignerHostSupport)' == 'true'">true</_DefaultValueAttributeSupport>
	<_DesignerHostSupport Condition="'$(OverrideDefaultValueAndDesignerHostSupport)' == 'true'">true</_DesignerHostSupport>
	</PropertyGroup>

but I'm not 100% sure why we get this mismatch between the feature switch used inside of the BCL and the test code...

[ivanpovazan]

I also noticed trimming tests have this configuration:

runtime/src/libraries/System.Data.Common/tests/TrimmingTests/System.Data.Common.TrimmingTests.proj

Lines 7 to 9 in e78ee77

	<TestConsoleAppSourceFiles Include="DataSetXmlSerialization.cs" >
	<DisabledFeatureSwitches>System.Data.DataSet.XmlSerializationIsSupported</DisabledFeatureSwitches>
	</TestConsoleAppSourceFiles>

[ivanpovazan]

I think the problem is related to how we split trimming+AOT builds between Helix and build machines where we have the following:

1. On build machines we only build and don't have PublishTrimmed=true. Therefore RuntimeHostConfiguration gets configured by only what is specified in test project files. This means that we will have only a few features configured that end up in RuntimeHostConfiguration and in runtimeconfig.json. For example System.Data.Common.Tests.runtimeconfig.json contains:

  {
    "runtimeOptions": {
      "tfm": "net10.0",
      "includedFrameworks": [
        {
          "name": "Microsoft.NETCore.App",
          "version": "10.0.0"
        }
      ],
      "configProperties": {
        "System.Reflection.Metadata.MetadataUpdater.IsSupported": false,
        "System.Runtime.CompilerServices.RuntimeFeature.IsDynamicCodeSupported": false,
        "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization": true
      }
    }
  }

2. We transfer both RuntimeHostConfiguration (via _AppleUsedRuntimeHostConfigurationOption) and runtimeconfig.json to Helix through our "AOT on Helix" logic
3. On Helix we perform trimming and AOT compilation and we have PublishTrimmed=true. Therefore, Microsoft.NET.ILLink.targets set defaults for feature switches, along with

   runtime/src/tools/illink/src/ILLink.Tasks/build/Microsoft.NET.ILLink.targets

   Line 60 in 8a25506

   <_DataSetXmlSerializationSupport Condition="'$(_DataSetXmlSerializationSupport)' == ''">false</_DataSetXmlSerializationSupport>

   We overwrite only the feature switch values we collected during build via _AppleUsedRuntimeHostConfigurationOption:

   runtime/src/mono/msbuild/apple/data/ProxyProjectForAOTOnHelix.proj

   Lines 102 to 104 in 6d6f2bd

   	<!-- To recreate the state of feature flags (RuntimeHostConfigurationOption) during app build-time we need to discard the default setting, coming from SDK, and used the values passed in ProxyProjectForAOTOnHelix.props -->
   	<RuntimeHostConfigurationOption Remove="@(_AppleUsedRuntimeHostConfigurationOption)" />
   	<RuntimeHostConfigurationOption Include="@(_AppleUsedRuntimeHostConfigurationOption)" />

   
   and run the linker. Since we didn't specify a value for _DataSetXmlSerializationSupport during build, trimmer uses the default and wires XmlSerializationIsSupported to false and removes deadcode.
4. During runtime, PlatformDetection determines which tests need to be included by reading XmlSerializationIsSupported from AppContext, but since runtimeconfig.json does not set it, the value will be resolved as true and we will run the tests which don't have XmlSerialization support - causing tests to crash

[akoeplinger]

Yeah that sounds like a reasonable explanation and is also pretty scary :)
It's why I mentioned when we talked with @kotlarmilos that I don't like this setup.

I wonder if we could only do the AOT+app build on Helix and do the publish+trim on AzDO, that one should be fast enough.

[kotlarmilos]

I wonder if we could only do the AOT+app build on Helix and do the publish+trim on AzDO, that one should be fast enough.

Do you mean publish+trim on build machine? I remember there was a bottleneck, but let's try again. I agree that the current can "mask" errors.

[akoeplinger]

yes, publish+trim. afaik the bottleneck was on AOTing the assemblies.