Stack overflow should abort the process normally, not segfault and dump core

[brson]

Today, when you overflow the stack, Rust traps the SIGSEGV and prints an error message, then the program exits with a segfault, exit code 135 on linux, and dumps core.

When we used segstacks for stack overflow the runtime just abort!ed, exiting with an illegal instruction.

That stack overflow protection is implemented by handling a segfault is an implementation detail. Rust should not be exposing it. Instead, when it detects a segfault because of stack overflow it should abort like any other fatal error.

cc https://users.rust-lang.org/t/rust-guarantees-no-segfaults-with-only-safe-code-but-it-segfaults-stack-overflow/4305/20

[jethrogb]

Considering Rust already traps the SIGSEGV, modifying the signal handler to exit at that point should be easy.

[nagisa]

SIGSEGV is the default behaviour on stack overflow in C/C++ programs compiled both by gcc and clang.

I’m not sure SIGILL is any better end-result compared to SIGSEGV. They both will dump the core and they both are pretty opaque. Fixing this issue will only serve to make “prevents segfaults” (should be changed to something else regardless of the outcome here) on the front page a little bit true-r.

[lambda]

I think that SIGABRT is fairly reasonable for a stack overflow; it's still treated as a crash, so will dump core or send crash reports on most platforms, but it doesn't cause the confusion that a SIGSEGV would cause by making people think that they need to track down some piece of unsafe code that's chasing a dangling pointer.

I've sent a PR that implements this.

[lambda]

This ticket is related to #30963 (about updating the documentation to mention stack overflow segfaults, which would not be necessary if this issue is fixed) and #26458 (which contains discussion of the most recent change to the current stack overflow behavior).

[nagisa]

@lambda there’s no (portable) way to produce SIGABRT in a no-stack condition, I think. SIGSEGV, SIGILL and SIGBUS, I think are an exhaustive set of signals you can make happen without stack.

[lambda]

@nagisa Do you have any reference for that? According to POSIX:

The following table defines a set of functions that shall be either reentrant or non-interruptible by signals and shall be async-signal-safe. Therefore applications may invoke them, without restriction, from signal-catching functions:

• ...
• abort()
• ...
• raise()

Remember, when we're in our SIGBUS/SIGSEGV handler, we're operating on a small alternative stack, so we actually do have enough stack space to run our signal handler and print out an error message. I believe that should be sufficient for raising SIGABRT as well.

[nagisa]

we're operating on a small alternative stack

Yep, I wrote the comment under the assumption we’re restricted to not using any stack, but I had totally forgotten we have signal specific stack for SIGSEGV handler.