Segmentation fault in System.Text.RegularExpressions.Tests

[akoeplinger]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=431489
Build error leg or test failing: Libraries Test Run release coreclr osx x64 Release
Pull request: N/A

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "Segmentation fault.*System.Text.RegularExpressions.Tests",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=431489
Error message validated: Segmentation fault.*System.Text.RegularExpressions.Tests
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 10/9/2023 11:24:45 AM UTC

Report

Build	Definition	Test	Pull Request
643217	dotnet/runtime	System.Text.RegularExpressions.Tests.WorkItemExecution	#100446
631279	dotnet/runtime	System.Text.RegularExpressions.Tests.WorkItemExecution	#100658
2422814	dotnet-runtime	System.Text.RegularExpressions.Tests.WorkItemExecution	#38744
626423	dotnet/runtime	System.Text.RegularExpressions.Tests.WorkItemExecution	#100503
624173	dotnet/runtime	System.Text.RegularExpressions.Tests.WorkItemExecution	#100446
617751	dotnet/runtime	System.Text.RegularExpressions.Tests.WorkItemExecution	#100169

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
1	1	6

Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=431489
Build error leg or test failing: Libraries Test Run release coreclr osx x64 Release
Pull request: N/A

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "Segmentation fault.*System.Text.RegularExpressions.Tests",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Author:	akoeplinger
Assignees:	-
Labels:	area-System.Text.RegularExpressions, blocking-clean-ci, Known Build Error
Milestone:	-

[akoeplinger]

Looks similar to #85046

[steveharter]

Some detail from the logs; no dump:

  Discovering: System.Text.RegularExpressions.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Text.RegularExpressions.Tests (found 329 of 357 test cases)
  Starting:    System.Text.RegularExpressions.Tests (parallel test collections = on, max threads = 4)
./RunTests.sh: line 204: 48718 Segmentation fault: 11  "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Text.RegularExpressions.Tests.runtimeconfig.json --depsfile System.Text.RegularExpressions.Tests.deps.json xunit.console.dll System.Text.RegularExpressions.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/private/tmp/helix/working/B0D30970/w/B6A909D4/e
----- end Mon Oct 9 04:57:38 EDT 2023 ----- exit code 139 ----------------------------------------------------------
exit code 139 means SIGSEGV Illegal memory access. Deref invalid pointer, overrunning buffer, stack overflow etc. Core dumped.
ulimit -c value: 0
+ export _commandExitCode=139
+ _commandExitCode=139
+ /usr/local/bin/python3 /tmp/helix/working/B0D30970/p/reporter/run.py https://dev.azure.com/dnceng-public/ public 9545048 eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6Im9PdmN6NU1fN3AtSGpJS2xGWHo5M3VfVjBabyJ9.eyJuYW1laWQiOiJjNzczZjJjMi01MTIwLTQyMDctYWZlMi1hZmFmMzVhOGJjMGEiLCJzY3AiOiJhcHBfdG9rZW4iLCJhdWkiOiJmYjRjZWM4Ny00MzBlLTQ5MjctYWY5Zi0wODEyMDY0ZjliNTEiLCJzaWQiOiJkNDQ0NDU2ZS0wMGFkLTQxODktOTZiZi03NTc5YzhkNDFhMzIiLCJCdWlsZElkIjoiY2JiMTgyNjEtYzQ4Zi00YWJiLTg2NTEtOGNkY2I1NDc0NjQ5OzQzMTQ4OSIsImpvYnJlZiI6IjRlNzhkMDdhLTU0MDQtNDA0Yy04NWQ0LWYwMGYwMjU0OTc1Zjo3ZTk4NjRhNi03ZmVjLTVjZDQtZmEwMS01OGQ3NmEwMzU4ZjkiLCJwcGlkIjoidnN0ZnM6Ly8vQnVpbGQvQnVpbGQvNDMxNDg5Iiwib3JjaGlkIjoiNGU3OGQwN2EtNTQwNC00MDRjLTg1ZDQtZjAwZjAyNTQ5NzVmLmJ1aWxkLmxpYnJhcmllc190ZXN0X3J1bl9yZWxlYXNlX2NvcmVjbHJfb3N4X3g2NF9yZWxlYXNlLl9fZGVmYXVsdCIsInJlcG9JZHMiOiIiLCJpc3MiOiJhcHAudnN0b2tlbi52aXN1YWxzdHVkaW8uY29tIiwiYXVkIjoiYXBwLnZzdG9rZW4udmlzdWFsc3R1ZGlvLmNvbXx2c286NmZjYzkyZTUtNzNhNy00Zjg4LThkMTMtZDkwNDViNDVmYjI3IiwibmJmIjoxNjk2ODQwNDc2LCJleHAiOjE2OTY4NTA2NzZ9.IVBKJCxDEmXAR-ltcWk6a08SCMG-owMAPvaBhq_6BAtitZNAJPwChwcwtCqIv5sOeUCkQYfSmU6UD1OWMSxeUeY_nFlukS-q4eD9X_HJaz5rICRMFcmO3u884rtGgHef6YAaojCw894W-rBndNvV2mZ-cso9BmfPOYZMoYpO3pQfeCZoaRq3Im1VsJY26_W0rExBk8asLIjuEKDlk7LjYA7kOF61uh3Qy5fQRJtQkDDaJY9PEExvCBneeRf5cnoARNncRi9z3mpHiqsIZ6W2gyMP6uff5v0OG5GFu-AJkVHXgREL3pw935WTLMKFMORsc8lD5MqYryve3RCOTw0CZA
2023-10-09T08:57:46.052Z	INFO   	run.py	run(48)	main	Beginning reading of test results.
2023-10-09T08:57:46.054Z	INFO   	run.py	__init__(42)	read_results	Searching '/private/tmp/helix/working/B0D30970/w/B6A909D4/e' for test results files
2023-10-09T08:57:46.056Z	INFO   	run.py	__init__(42)	read_results	Searching '/tmp/helix/working/B0D30970/w/B6A909D4/uploads' for test results files
2023-10-09T08:57:46.057Z	WARNING	run.py	__init__(55)	read_results	No results file found in any of the following formats: xunit, junit, trx
2023-10-09T08:57:46.058Z	INFO   	run.py	packing_test_reporter(30)	report_results	Packing 0 test reports to '/tmp/helix/working/B0D30970/w/B6A909D4/e/__test_report.json'
2023-10-09T08:57:46.058Z	INFO   	run.py	packing_test_reporter(33)	report_results	Packed 1553 bytes
+ /usr/local/bin/python3 /tmp/helix/working/B0D30970/p/gen-debug-dump-docs.py -buildid 431489 -workitem System.Text.RegularExpressions.Tests -jobid 5472e3d3-beb1-49db-af99-5d3100d2a736 -outdir /tmp/helix/working/B0D30970/w/B6A909D4/uploads -templatedir /tmp/helix/working/B0D30970/p -dumpdir /cores -productver 9.0.0
Did not find dumps, skipping dump docs generation.
+ exit 139
['System.Text.RegularExpressions.Tests' END OF WORK ITEM LOG: Command exited with 139]

[steveharter]

Both this and #85046 occurred on OSX.

[stephentoub]

Without a dump it'll be impossible to make progress on this. It's also very unlikely to be in regex itself, and much more likely to be an issue either in the span-related functionality regex sits on top of, or in codegen / the runtime.

[akoeplinger]

I was able to capture a core dump on my local Mac using the Helix artifacts, but it is 8GB so uploading will take a while :)