[wasm][aot] `RangeError: Maximum call stack size exceeded` in `System.Text.RegularExpressions.Tests`

[radical]

Seen on main rolling build:
Build
Log

[21:39:13] fail: RangeError: Maximum call stack size exceeded
[21:39:13] fail: RangeError: Maximum call stack size exceeded
[21:39:13] fail:     at jit_info_table_index (<anonymous>:wasm-function[121904]:0x18e0147)
[21:39:13] fail:     at jit_info_table_find (<anonymous>:wasm-function[121903]:0x18e0078)
[21:39:13] fail:     at mono_jit_info_table_find_internal (<anonymous>:wasm-function[121902]:0x18dffb9)
[21:39:13] fail:     at mini_jit_info_table_find_ext (<anonymous>:wasm-function[134483]:0x1a4b0f0)
[21:39:13] fail:     at mono_find_jit_info_ext (<anonymous>:wasm-function[134482]:0x1a4af06)
[21:39:13] fail:     at unwinder_unwind_frame (<anonymous>:wasm-function[134500]:0x1a4b969)
[21:39:13] fail:     at mono_walk_stack_full (<anonymous>:wasm-function[134464]:0x1a4a85e)
[21:39:13] fail:     at mono_walk_stack_with_ctx (<anonymous>:wasm-function[134462]:0x1a4a6ec)
[21:39:13] fail:     at mono_runtime_walk_stack_with_ctx (<anonymous>:wasm-function[134448]:0x1a4a462)
[21:39:13] fail:     at mono_stack_walk (<anonymous>:wasm-function[121987]:0x18e30ab)
[21:39:13] fail:     at is_running_protected_wrapper (<anonymous>:wasm-function[123237]:0x1901809)
[21:39:13] fail:     at mono_thread_interruption_checkpoint_request (<anonymous>:wasm-function[123251]:0x1901b4a)
[21:39:13] fail:     at mono_thread_interruption_checkpoint (<anonymous>:wasm-function[123250]:0x1901b1c)
[21:39:13] fail:     at aot_wrapper_icall_mono_thread_interruption_checkpoint (<anonymous>:wasm-function[66920]:0xf05864)
[21:39:13] fail:     at aot_wrapper_corlib_System_dot_Runtime_dot_CompilerServices_System_dot_Runtime_dot_CompilerServices_dot_RuntimeHelpers__SufficientExecutionStack_pinvoke_bool_bool_ (<anonymous>:wasm-function[75022]:0x106220a)
[21:39:13] fail:     at corlib_System_Runtime_CompilerServices_RuntimeHelpers_TryEnsureSufficientExecutionStack (<anonymous>:wasm-function[74477]:0x10517fb)
[21:39:13] fail:     at System_Text_RegularExpressions_System_Threading_StackHelper_TryEnsureSufficientExecutionStack (<anonymous>:wasm-function[96157]:0x138dd1d)
[21:39:13] fail:     at System_Text_RegularExpressions_System_Text_RegularExpressions_RegexPrefixAnalyzer__FindCaseSensitivePrefixg__Process_17_0_System_Text_RegularExpressions_RegexNode_System_Text_ValueStringBuilder_ (<anonymous>:wasm-function[96329]:0x139ba31)
[21:39:13] fail:     at System_Text_RegularExpressions_System_Text_RegularExpressions_RegexPrefixAnalyzer__FindCaseSensitivePrefixg__Process_17_0_System_Text_RegularExpressions_RegexNode_System_Text_ValueStringBuilder_ (<anonymous>:wasm-function[96329]:0x139be7a)
[21:39:13] fail:     at System_Text_RegularExpressions_System_Text_RegularExpressions_RegexPrefixAnalyzer__FindCaseSensitivePrefixg__Process_17_0_System_Text_RegularExpressions_RegexNode_System_Text_ValueStringBuilder_ (<anonymous>:wasm-function[96329]:0x139be7a)

... <frames repeat>

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Seen on main rolling build:
Build
Log

[21:39:13] fail: RangeError: Maximum call stack size exceeded
[21:39:13] fail: RangeError: Maximum call stack size exceeded
[21:39:13] fail:     at jit_info_table_index (<anonymous>:wasm-function[121904]:0x18e0147)
[21:39:13] fail:     at jit_info_table_find (<anonymous>:wasm-function[121903]:0x18e0078)
[21:39:13] fail:     at mono_jit_info_table_find_internal (<anonymous>:wasm-function[121902]:0x18dffb9)
[21:39:13] fail:     at mini_jit_info_table_find_ext (<anonymous>:wasm-function[134483]:0x1a4b0f0)
[21:39:13] fail:     at mono_find_jit_info_ext (<anonymous>:wasm-function[134482]:0x1a4af06)
[21:39:13] fail:     at unwinder_unwind_frame (<anonymous>:wasm-function[134500]:0x1a4b969)
[21:39:13] fail:     at mono_walk_stack_full (<anonymous>:wasm-function[134464]:0x1a4a85e)
[21:39:13] fail:     at mono_walk_stack_with_ctx (<anonymous>:wasm-function[134462]:0x1a4a6ec)
[21:39:13] fail:     at mono_runtime_walk_stack_with_ctx (<anonymous>:wasm-function[134448]:0x1a4a462)
[21:39:13] fail:     at mono_stack_walk (<anonymous>:wasm-function[121987]:0x18e30ab)
[21:39:13] fail:     at is_running_protected_wrapper (<anonymous>:wasm-function[123237]:0x1901809)
[21:39:13] fail:     at mono_thread_interruption_checkpoint_request (<anonymous>:wasm-function[123251]:0x1901b4a)
[21:39:13] fail:     at mono_thread_interruption_checkpoint (<anonymous>:wasm-function[123250]:0x1901b1c)
[21:39:13] fail:     at aot_wrapper_icall_mono_thread_interruption_checkpoint (<anonymous>:wasm-function[66920]:0xf05864)
[21:39:13] fail:     at aot_wrapper_corlib_System_dot_Runtime_dot_CompilerServices_System_dot_Runtime_dot_CompilerServices_dot_RuntimeHelpers__SufficientExecutionStack_pinvoke_bool_bool_ (<anonymous>:wasm-function[75022]:0x106220a)
[21:39:13] fail:     at corlib_System_Runtime_CompilerServices_RuntimeHelpers_TryEnsureSufficientExecutionStack (<anonymous>:wasm-function[74477]:0x10517fb)
[21:39:13] fail:     at System_Text_RegularExpressions_System_Threading_StackHelper_TryEnsureSufficientExecutionStack (<anonymous>:wasm-function[96157]:0x138dd1d)
[21:39:13] fail:     at System_Text_RegularExpressions_System_Text_RegularExpressions_RegexPrefixAnalyzer__FindCaseSensitivePrefixg__Process_17_0_System_Text_RegularExpressions_RegexNode_System_Text_ValueStringBuilder_ (<anonymous>:wasm-function[96329]:0x139ba31)
[21:39:13] fail:     at System_Text_RegularExpressions_System_Text_RegularExpressions_RegexPrefixAnalyzer__FindCaseSensitivePrefixg__Process_17_0_System_Text_RegularExpressions_RegexNode_System_Text_ValueStringBuilder_ (<anonymous>:wasm-function[96329]:0x139be7a)
[21:39:13] fail:     at System_Text_RegularExpressions_System_Text_RegularExpressions_RegexPrefixAnalyzer__FindCaseSensitivePrefixg__Process_17_0_System_Text_RegularExpressions_RegexNode_System_Text_ValueStringBuilder_ (<anonymous>:wasm-function[96329]:0x139be7a)

Author:	radical
Assignees:	-
Labels:	arch-wasm, blocking-clean-ci, area-Codegen-AOT-mono
Milestone:	-

[radical]

cc @vargaz

[lewing]

cc @radekdoulik

[lewing]

runtime/src/libraries/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexPrefixAnalyzer.cs

Line 54 in 25237fa

static bool Process(RegexNode node, ref ValueStringBuilder vsb)

looks like the problem. It calls TryEnsureSufficientExecutionStack and the mono icall may need some improvement

[lewing]

cc @lambdageek

[lambdageek]

@lewing @radical Is this test actually expected to recurse deeply? There's not a bug where the test is unintentionally getting into an infinite recursion?

It looks like in this case we overflowed in a method called from the wrapper to do a stack walk. Maybe those are expensive on wasm? In any case, maybe we can double MONO_MIN_EXECUTION_STACK_SIZE on Wasm to 128 as if it was a 64-bit platform:

runtime/src/mono/mono/metadata/icall.c

Lines 118 to 120 in c159108

	#define MONO_MIN_EXECUTION_STACK_SIZE (128 * 1024)
	#else
	#define MONO_MIN_EXECUTION_STACK_SIZE (64 * 1024)

(I checked and it seems like the stack bounds computation will get non-fake values on Emscripten - so doing this stack check should work)