Issue python#11393: Add the new faulthandler module

Author: Victor Stinner

Doc/library/debug.rst

@@ -10,7 +10,8 @@ allowing you to identify bottlenecks in your programs.
 .. toctree::
 
    bdb.rst
+   faulthandler.rst
    pdb.rst
    profile.rst
    timeit.rst
-   trace.rst
+   trace.rst

Doc/library/faulthandler.rst

@@ -0,0 +1,129 @@
+:mod:`faulthandler` --- Dump the Python traceback
+=================================================
+
+.. module:: faulthandler
+   :synopsis: Dump the Python traceback.
+
+This module contains functions to dump the Python traceback explicitly, on a
+fault, after a timeout or on a user signal. Call :func:`faulthandler.enable` to
+install fault handlers for :const:`SIGSEGV`, :const:`SIGFPE`, :const:`SIGBUS`
+and :const:`SIGILL` signals. You can also enable them at startup by setting the
+:envvar:`PYTHONFAULTHANDLER` environment variable or by using :option:`-X`
+``faulthandler`` command line option.
+
+The fault handler is compatible with system fault handlers like Apport or
+the Windows fault handler. The module uses an alternative stack for signal
+handlers, if the :c:func:`sigaltstack` function is available, to be able to
+dump the traceback even on a stack overflow.
+
+The fault handler is called on catastrophic cases and so can only use
+signal-safe functions (e.g. it cannot allocate memory on the heap). That's why
+the traceback is limited: only support ASCII encoding (use the
+``backslashreplace`` error handler), limit each string to 100 characters, don't
+print the source code (only the filename, the function name and the line
+number), limit to 100 frames and 100 threads.
+
+By default, the Python traceback is written to :data:`sys.stderr`. Start your
+graphical applications in a terminal and run your server in foreground to see
+the traceback, or specify a log file to :func:`faulthandler.enable()`.
+
+The module is implemented in C to be able to dump a traceback on a crash or
+when Python is blocked (e.g. deadlock).
+
+.. versionadded:: 3.3
+
+
+Dump the traceback
+------------------
+
+.. function:: dump_traceback(file=sys.stderr, all_threads=False)
+
+   Dump the traceback of the current thread, or of all threads if *all_threads*
+   is ``True``, into *file*.
+
+
+Fault handler state
+-------------------
+
+.. function:: enable(file=sys.stderr, all_threads=False)
+
+   Enable the fault handler: install handlers for :const:`SIGSEGV`,
+   :const:`SIGFPE`, :const:`SIGBUS` and :const:`SIGILL` signals to dump the
+   Python traceback. It dumps the traceback of the current thread, or all
+   threads if *all_threads* is ``True``, into *file*.
+
+.. function:: disable()
+
+   Disable the fault handler: uninstall the signal handlers installed by
+   :func:`enable`.
+
+.. function:: is_enabled()
+
+   Check if the fault handler is enabled.
+
+
+Dump the tracebacks after a timeout
+-----------------------------------
+
+.. function:: dump_tracebacks_later(timeout, repeat=False, file=sys.stderr, exit=False)
+
+   Dump the tracebacks of all threads, after a timeout of *timeout* seconds, or
+   each *timeout* seconds if *repeat* is ``True``.  If *exit* is True, call
+   :cfunc:`_exit` with status=1 after dumping the tracebacks to terminate
+   immediatly the process, which is not safe.  For example, :cfunc:`_exit`
+   doesn't flush file buffers.  If the function is called twice, the new call
+   replaces previous parameters (resets the timeout). The timer has a
+   sub-second resolution.
+
+   This function is implemented using a watchdog thread, and therefore is
+   not available if Python is compiled with threads disabled.
+
+.. function:: cancel_dump_traceback_later()
+
+   Cancel the last call to :func:`dump_traceback_later`.
+
+
+Dump the traceback on a user signal
+-----------------------------------
+
+.. function:: register(signum, file=sys.stderr, all_threads=False)
+
+   Register a user signal: install a handler for the *signum* signal to dump
+   the traceback of the current thread, or of all threads if *all_threads* is
+   ``True``, into *file*.
+
+   Not available on Windows.
+
+.. function:: unregister(signum)
+
+   Unregister a user signal: uninstall the handler of the *signum* signal
+   installed by :func:`register`.
+
+   Not available on Windows.
+
+
+File descriptor issue
+---------------------
+
+:func:`enable`, :func:`dump_traceback_later` and :func:`register` keep the
+file descriptor of their *file* argument. If the file is closed and its file
+descriptor is reused by a new file, or if :func:`os.dup2` is used to replace
+the file descriptor, the traceback will be written into a different file. Call
+these functions again each time that the file is replaced.
+
+
+Example
+-------
+
+Example of a segmentation fault on Linux: ::
+
+    $ python -q -X faulthandler
+    >>> import ctypes
+    >>> ctypes.string_at(0)
+    Fatal Python error: Segmentation fault
+
+    Traceback (most recent call first):
+      File "/home/python/cpython/Lib/ctypes/__init__.py", line 486 in string_at
+      File "<stdin>", line 1 in <module>
+    Segmentation fault
+

Doc/using/cmdline.rst

@@ -498,6 +498,13 @@ These environment variables influence Python's behavior.
    separated string, it is equivalent to specifying :option:`-W` multiple
    times.
 
+.. envvar:: PYTHONFAULTHANDLER
+
+   If this environment variable is set, :func:`faulthandler.enable` is called
+   at startup: install a handler for :const:`SIGSEGV`, :const:`SIGFPE`,
+   :const:`SIGBUS` and :const:`SIGILL` signals to dump the Python traceback.
+   This is equivalent to :option:`-X` ``faulthandler`` option.
+
 
 Debug-mode variables
 ~~~~~~~~~~~~~~~~~~~~

Doc/whatsnew/3.3.rst

@@ -68,6 +68,14 @@ New, Improved, and Deprecated Modules
 
 * Stub
 
+faulthandler
+------------
+
+New module: :mod:`faulthandler`.
+
+ * :envvar:`PYTHONFAULTHANDLER`
+ * :option:`-X` ``faulthandler``
+
 os
 --
 

Include/traceback.h

@@ -5,6 +5,8 @@
 extern "C" {
 #endif
 
+#include "pystate.h"
+
 struct _frame;
 
 /* Traceback interface */
@@ -28,6 +30,44 @@ PyAPI_FUNC(int) _Py_DisplaySourceLine(PyObject *, PyObject *, int, int);
 PyAPI_DATA(PyTypeObject) PyTraceBack_Type;
 #define PyTraceBack_Check(v) (Py_TYPE(v) == &PyTraceBack_Type)
 
+/* Write the Python traceback into the file 'fd'. For example:
+
+       Traceback (most recent call first):
+         File "xxx", line xxx in <xxx>
+         File "xxx", line xxx in <xxx>
+         ...
+         File "xxx", line xxx in <xxx>
+
+   Return 0 on success, -1 on error.
+
+   This function is written for debug purpose only, to dump the traceback in
+   the worst case: after a segmentation fault, at fatal error, etc. That's why,
+   it is very limited. Strings are truncated to 100 characters and encoded to
+   ASCII with backslashreplace. It doesn't write the source code, only the
+   function name, filename and line number of each frame. Write only the first
+   100 frames: if the traceback is truncated, write the line " ...".
+
+   This function is signal safe. */
+
+PyAPI_DATA(int) _Py_DumpTraceback(
+    int fd,
+    PyThreadState *tstate);
+
+/* Write the traceback of all threads into the file 'fd'. current_thread can be
+   NULL. Return NULL on success, or an error message on error.
+
+   This function is written for debug purpose only. It calls
+   _Py_DumpTraceback() for each thread, and so has the same limitations. It
+   only write the traceback of the first 100 threads: write "..." if there are
+   more threads.
+
+   This function is signal safe. */
+
+PyAPI_DATA(const char*) _Py_DumpTracebackThreads(
+    int fd, PyInterpreterState *interp,
+    PyThreadState *current_thread);
+
+
 #ifdef __cplusplus
 }
 #endif

Lib/test/regrtest.py

@@ -157,6 +157,7 @@
 """
 
 import builtins
+import faulthandler
 import getopt
 import json
 import os
@@ -490,6 +491,7 @@ def main(tests=None, testdir=None, verbose=0, quiet=False,
             next_single_test = alltests[alltests.index(selected[0])+1]
         except IndexError:
             next_single_test = None
+    selected = ['test_faulthandler']
     # Remove all the tests that precede start if it's set.
     if start:
         try:
@@ -1551,6 +1553,9 @@ def _make_temp_dir_for_build(TEMPDIR):
     return TEMPDIR, TESTCWD
 
 if __name__ == '__main__':
+    # Display the Python traceback on segfault and division by zero
+    faulthandler.enable()
+
     # Remove regrtest.py's own directory from the module search path. Despite
     # the elimination of implicit relative imports, this is still needed to
     # ensure that submodules of the test package do not inappropriately appear

Lib/test/script_helper.py

@@ -56,11 +56,12 @@ def assert_python_failure(*args, **env_vars):
     """
     return _assert_python(False, *args, **env_vars)
 
-def spawn_python(*args):
+def spawn_python(*args, **kw):
     cmd_line = [sys.executable, '-E']
     cmd_line.extend(args)
     return subprocess.Popen(cmd_line, stdin=subprocess.PIPE,
-                            stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+                            stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
+                            **kw)
 
 def kill_python(p):
     p.stdin.close()