Failure in test XML parsing

[joshpeterson]

I'm trying to sync the Unity fork of the dotnet/runtime repo with upstream changes made last week and we're seeing failures on our CI, trying to parse the test results. We're running this:

src\tests\run.cmd x64 checked

And I see this error:

Analyzing C:\build\output\Unity-Technologies\runtime\artifacts\log\TestRun.xml

Traceback (most recent call last):
  File "C:\build\output\Unity-Technologies\runtime\src\tests\..\..\src\tests\run.py", line 1387, in <module>
    sys.exit(main(args))
  File "C:\build\output\Unity-Technologies\runtime\src\tests\..\..\src\tests\run.py", line 1375, in main
    parse_test_results(args, tests, assemblies)
  File "C:\build\output\Unity-Technologies\runtime\src\tests\..\..\src\tests\run.py", line 1174, in parse_test_results
    parse_test_results_xml_file(args, item, item_name, tests, assemblies)
  File "C:\build\output\Unity-Technologies\runtime\src\tests\..\..\src\tests\run.py", line 1194, in parse_test_results_xml_file
    assembly_name = assembly.attrib["name"]
KeyError: 'name'

Not surprisingly, I see in that XML file some results that have an element like this: <assembly \>, so I think that causing the issue. Should the parsing code handle an element like this? Or does the existence of this element indicate a different problem?

It looks like these changes came in with #61942 last week. Are we doing something wrong on our CI, that this should not happen? Or is there a known issue around this?

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[joshpeterson]

I suspect the proper label for this issue is area-Infrastructure-coreclr.

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

I'm trying to sync the Unity fork of the dotnet/runtime repo with upstream changes made last week and we're seeing failures on our CI, trying to parse the test results. We're running this:

src\tests\run.cmd x64 checked

And I see this error:

Analyzing C:\build\output\Unity-Technologies\runtime\artifacts\log\TestRun.xml

Traceback (most recent call last):
  File "C:\build\output\Unity-Technologies\runtime\src\tests\..\..\src\tests\run.py", line 1387, in <module>
    sys.exit(main(args))
  File "C:\build\output\Unity-Technologies\runtime\src\tests\..\..\src\tests\run.py", line 1375, in main
    parse_test_results(args, tests, assemblies)
  File "C:\build\output\Unity-Technologies\runtime\src\tests\..\..\src\tests\run.py", line 1174, in parse_test_results
    parse_test_results_xml_file(args, item, item_name, tests, assemblies)
  File "C:\build\output\Unity-Technologies\runtime\src\tests\..\..\src\tests\run.py", line 1194, in parse_test_results_xml_file
    assembly_name = assembly.attrib["name"]
KeyError: 'name'

Not surprisingly, I see in that XML file some results that have an element like this: <assembly \>, so I think that causing the issue. Should the parsing code handle an element like this? Or does the existence of this element indicate a different problem?

It looks like these changes came in with #61942 last week. Are we doing something wrong on our CI, that this should not happen? Or is there a known issue around this?

Author:	joshpeterson
Assignees:	-
Labels:	area-Infrastructure, untriaged
Milestone:	-

[jkoritzinsky]

cc: @trylek

[ericstj]

Failure introduced in fc6fa22#diff-bf9472fd7a5011d9392cd97e8edc287f0150708d08eb8007a740ac9fb2e77b32R1194

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

I'm trying to sync the Unity fork of the dotnet/runtime repo with upstream changes made last week and we're seeing failures on our CI, trying to parse the test results. We're running this:

src\tests\run.cmd x64 checked

And I see this error:

Analyzing C:\build\output\Unity-Technologies\runtime\artifacts\log\TestRun.xml

Traceback (most recent call last):
  File "C:\build\output\Unity-Technologies\runtime\src\tests\..\..\src\tests\run.py", line 1387, in <module>
    sys.exit(main(args))
  File "C:\build\output\Unity-Technologies\runtime\src\tests\..\..\src\tests\run.py", line 1375, in main
    parse_test_results(args, tests, assemblies)
  File "C:\build\output\Unity-Technologies\runtime\src\tests\..\..\src\tests\run.py", line 1174, in parse_test_results
    parse_test_results_xml_file(args, item, item_name, tests, assemblies)
  File "C:\build\output\Unity-Technologies\runtime\src\tests\..\..\src\tests\run.py", line 1194, in parse_test_results_xml_file
    assembly_name = assembly.attrib["name"]
KeyError: 'name'

Not surprisingly, I see in that XML file some results that have an element like this: <assembly \>, so I think that causing the issue. Should the parsing code handle an element like this? Or does the existence of this element indicate a different problem?

It looks like these changes came in with #61942 last week. Are we doing something wrong on our CI, that this should not happen? Or is there a known issue around this?

Author:	joshpeterson
Assignees:	-
Labels:	area-Infrastructure-coreclr, untriaged
Milestone:	-

[trylek]

I'm taking a look now. @joshpeterson, would it be possible for you to share a sample XML hitting the run.py failure, perhaps as a Gist, so that I can use it to validate the fix?

[trylek]

OK, I was somewhat surprised that the line numbers you quoted in the stack trace were slightly off w.r.t. my local fresh clone of the source from the runtime repo. I noticed this bug myself and I fixed it about two days ago with the PR #62683 and commit bf54733#diff-bf9472fd7a5011d9392cd97e8edc287f0150708d08eb8007a740ac9fb2e77b32. @joshpeterson - would it be possible for you to retry with a newer version of the runtime repo including this fix?

[joshpeterson]

@trylek: Yes, I'll try that, thanks! I suspect that will correct the issue we see.

[joshpeterson]

@trylek: The latest code works. Thanks!