Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error_already_set::what() is now constructed lazily #1895

Merged
merged 136 commits into from
Jun 2, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
136 commits
Select commit Hold shift + click to select a range
87d1f6b
error_already_set::what() is now constructed lazily
superbobry Aug 27, 2019
e289e0e
Do not attempt to normalize if no exception occurred
superbobry Aug 28, 2019
f1435c7
Extract exception name via tp_name
superbobry Aug 28, 2019
60df143
Reverted error_already_set to subclass std::runtime_error
superbobry Aug 28, 2019
c935b73
Revert "Extract exception name via tp_name"
superbobry Aug 28, 2019
b48bc72
Cosmit following @YannickJadoul's comments
superbobry Aug 29, 2019
115a757
Fixed PyPy build
superbobry Aug 29, 2019
db14dd9
Moved normalization to error_already_set ctor
superbobry Aug 29, 2019
9635e88
Merge branch 'master' of https://github.com/pybind/pybind11 into lazy…
Skylion007 Feb 13, 2022
dbb3e6b
Fix merge bugs
Skylion007 Feb 13, 2022
a692362
Fix more merge errors
Skylion007 Feb 13, 2022
ff3185e
Improve formatting
Skylion007 Feb 13, 2022
95eeaef
Improve error message in rare case
Skylion007 Feb 13, 2022
00e4852
Revert back if statements
Skylion007 Feb 13, 2022
648e670
Fix clang-tidy
Skylion007 Feb 13, 2022
244aa52
Try removing mutable
Skylion007 Feb 13, 2022
b09f14d
Does build_mode release fix it
Skylion007 Feb 13, 2022
1d1ec9e
Set to Debug to expose segfault
Skylion007 Feb 13, 2022
6d0354b
Fix remove set error string
Skylion007 Feb 13, 2022
2c5ad0b
Do not run error_string() more than once
Skylion007 Feb 14, 2022
68b349a
Trying setting the tracebackk to the value
Skylion007 Feb 14, 2022
91bf33f
guard if m_type is null
Skylion007 Feb 15, 2022
63dfbb4
Merge branch 'master' of https://github.com/pybind/pybind11 into lazy…
Skylion007 Feb 15, 2022
d231f15
Try to debug PGI
Skylion007 Feb 15, 2022
df6cb30
One last try for PGI
Skylion007 Feb 15, 2022
bdcd95a
Does reverting this fix PyPy
Skylion007 Feb 16, 2022
225dbae
Reviewer suggestions
Skylion007 Feb 20, 2022
f8d3ed2
Remove unnecessary initialization
Skylion007 Feb 20, 2022
b90bd78
Add noexcept move and explicit fail throw
Skylion007 Feb 22, 2022
3286964
Optimize error_string creation
Skylion007 Feb 22, 2022
978bf2a
Fix typo
Skylion007 Feb 22, 2022
6c97e68
Revert noexcept
Skylion007 Feb 22, 2022
6523bc2
Merge branch 'master' of https://github.com/pybind/pybind11 into lazy…
Skylion007 Mar 2, 2022
6c6971f
Fix merge conflict error
Skylion007 Mar 2, 2022
98aff18
t pushMerge branch 'master' of https://github.com/pybind/pybind11 int…
Skylion007 Apr 23, 2022
e71d9b9
Abuse assignment operator
Skylion007 Apr 24, 2022
e9a2e6d
Revert operator abuse
Skylion007 May 3, 2022
a83028c
Merge branch 'master' of https://github.com/pybind/pybind11 into lazy…
Skylion007 May 4, 2022
e0d8d0e
See if we still need debug
Skylion007 May 4, 2022
d32bc91
Remove unnecessary mutable
Skylion007 May 4, 2022
226cdde
Merge branch 'master' into lazy-error-string
May 5, 2022
f4a440a
Report "FATAL failure building pybind11::error_already_set error_stri…
May 5, 2022
8f6ab3f
Try specifying noexcept again
Skylion007 May 6, 2022
4da9968
Try explicit ctor
Skylion007 May 6, 2022
f181dac
default ctor is noexcept too
Skylion007 May 6, 2022
6e781a0
Apply reviewer suggestions, simplify code, and make helper method pri…
Skylion007 May 8, 2022
1f8f0ab
Remove unnecessary include
Skylion007 May 8, 2022
2031225
Merge branch 'master' of https://github.com/pybind/pybind11 into lazy…
Skylion007 May 8, 2022
0aee425
Clang-Tidy fix
Skylion007 May 8, 2022
1c8398f
Merge branch 'master' into lazy-error-string
May 10, 2022
a8f7a97
Merge branch 'lazy-error-string' of https://github.com/superbobry/pyb…
May 10, 2022
0691d5f
detail::obj_class_name(), fprintf with [STDERR], [STDOUT] tags, polis…
May 10, 2022
28de959
consistently check m_lazy_what.empty() also in production builds
May 11, 2022
c7a7146
Make a comment slightly less ambiguous.
May 11, 2022
de84a27
Bug fix: Remove `what();` from `restore()`.
May 11, 2022
498195a
Replace extremely opaque (unhelpful) error message with a truthful re…
May 11, 2022
80b05ba
Fix clang-tidy error [performance-move-constructor-init].
May 11, 2022
acdf332
Make expected error message less specific.
May 11, 2022
58ad49b
Various changes.
May 11, 2022
90b2453
bug fix: error_string(PyObject **, ...)
May 11, 2022
dded024
Putting back the two eager PyErr_NormalizeException() calls.
May 12, 2022
4193375
Change error_already_set() to call pybind11_fail() if the Python erro…
May 12, 2022
b020e04
Remove mutable (fixes oversight in the previous commit).
May 12, 2022
02df6c0
Normalize the exception only locally in error_string(). Python 3.6 & …
May 12, 2022
d28c155
clang-tidy: use auto
May 12, 2022
7578de9
Use `gil_scoped_acquire_local` in `error_already_set` destructor. See…
May 12, 2022
bab6f66
For Python < 3.8: `PyErr_NormalizeException` before `PyErr_WriteUnrai…
May 12, 2022
2ad2285
Go back to replacing the held Python exception with then normalized e…
May 13, 2022
e3ebb0d
Slightly rewording comment. (There were also other failures.)
May 13, 2022
8dff51d
Add 1-line comment for obj_class_name()
May 13, 2022
923725a
Benchmark code, with results in this commit message.
May 13, 2022
53f40a0
Changing call_repetitions_target_elapsed_secs to 0.1 for regular unit…
May 13, 2022
b77e703
Adding in `recursion_depth`
May 14, 2022
493d751
Optimized ctor
Skylion007 May 14, 2022
6d0ec49
Merge branch 'lazy-error-string' of https://github.com/superbobry/pyb…
Skylion007 May 14, 2022
6d9188f
Fix silly bug in recurse_first_then_call()
May 15, 2022
8f3d3a6
Add tests that have equivalent PyErr_Fetch(), PyErr_Restore() but no …
May 15, 2022
724ee3d
Add call_error_string to tests. Sample only recursion_depth 0, 100.
May 15, 2022
467ab00
Show lazy-what speed-up in percent.
May 15, 2022
eaa1a48
Include real_work in benchmarks.
May 15, 2022
dbed20a
Replace all PyErr_SetString() with generate_python_exception_with_tra…
May 16, 2022
f021543
Better organization of test loops.
May 16, 2022
125e2d0
Add test_error_already_set_copy_move
May 16, 2022
b037958
Fix bug in newly added test (discovered by clang-tidy): actually use …
May 16, 2022
2baa3bc
MSVC detects the unreachable return
May 16, 2022
4c479f4
change test_perf_error_already_set.py back to quick mode
May 17, 2022
b4326f9
Inherit from std::exception (instead of std::runtime_error, which doe…
May 17, 2022
aaec34e
Special handling under Windows.
May 17, 2022
99af318
print with leading newline
May 17, 2022
e5f028f
Merge branch 'master' into lazy-error-string
May 17, 2022
8c76360
Removing test_perf_error_already_set (copies are under https://github…
May 18, 2022
c7149d8
Avoid gil and scope overhead if there is nothing to release.
May 18, 2022
47f5445
Restore default move ctor. "member function" instead of "function" (n…
May 18, 2022
7c08e7f
Delete error_already_set copy ctor.
May 18, 2022
f746a3d
Make restore() non-const again to resolve clang-tidy failure (still e…
May 18, 2022
25e4cc9
Bring back error_already_set copy ctor, to see if that resolves the 4…
May 18, 2022
70b870a
Add noexcept to error_already_set copy & move ctors (as suggested by …
May 18, 2022
d425bb8
Trying one-by-one noexcept copy ctor for old compilers.
May 18, 2022
65aaa4c
Add back test covering copy ctor. Add another simple test that exerci…
May 18, 2022
19bb595
Exclude more older compilers from using the noexcept = default ctors.…
May 19, 2022
d07b5ba
Factor out & reuse gil_scoped_acquire_local as gil_scoped_acquire_simple
May 19, 2022
f6bf5aa
Guard gil_scoped_acquire_simple by _Py_IsFinalizing() check.
May 19, 2022
cc5d76a
what() GIL safety
May 19, 2022
2b31e4d
clang-tidy & Python 3.6 fixes
May 19, 2022
73cffe4
Merge branch 'master' into lazy-error-string
May 20, 2022
2c5dd19
Use `gil_scoped_acquire` in dtor, copy ctor, `what()`. Remove `_Py_Is…
May 20, 2022
14be38c
Remove error_scope from copy ctor.
May 21, 2022
db97ce7
Add `error_scope` to `get_internals()`, to cover the situation that `…
May 21, 2022
42d2e1a
Add `FlakyException` tests with failure triggers in `__init__` and `_…
May 23, 2022
6417a76
Tweaks to resolve Py 3.6 and PyPy CI failures.
May 24, 2022
a62b3d7
Normalize Python exception immediately in error_already_set ctor.
May 24, 2022
c54309c
Merge branch 'master' into lazy-error-string
May 24, 2022
c786796
Fix oversights based on CI failures (copy & move ctor initialization).
May 24, 2022
61bb513
Move @pytest.mark.xfail("env.PYPY") after @pytest.mark.parametrize(...)
May 24, 2022
bb3f24c
Use @pytest.mark.skipif (xfail does not work for segfaults, of course).
May 24, 2022
087ef29
Merge branch 'master' into lazy-error-string
May 31, 2022
bee5fb4
Remove unused obj_class_name_or() function (it was added only under t…
Jun 1, 2022
bea0c9f
Remove already obsolete C++ comments and code that were added only un…
Jun 1, 2022
d6d371a
Slightly better (newly added) comments.
Jun 1, 2022
eec0547
Factor out detail::error_fetch_and_normalize. Preparation for produci…
Jun 1, 2022
ec1a2c0
Copy most of error_string() code to new error_fetch_and_normalize::co…
Jun 1, 2022
c39ebd5
Remove all error_string() code from detail/type_caster_base.h. Note t…
Jun 1, 2022
286023b
Return const std::string& instead of const char * and move error_stri…
Jun 1, 2022
156b57b
Remove gil_scope_acquire from error_fetch_and_normalize, add back to …
Jun 1, 2022
6f175ac
Better handling of FlakyException __str__ failure.
Jun 1, 2022
71f33a8
Move error_fetch_and_normalize::complete_lazy_error_string() implemen…
Jun 1, 2022
0739d4c
Add error_fetch_and_normalize::release_py_object_references() and use…
Jun 1, 2022
80dfaea
Use shared_ptr for m_fetched_error => 1. non-racy, copy ctor that doe…
Jun 1, 2022
a7ba693
Add comments.
Jun 1, 2022
9854589
Trivial renaming of a newly introduced member function.
Jun 1, 2022
d2e78b0
Merge branch 'master' into lazy-error-string
Jun 1, 2022
0e50c45
Workaround for PyPy
Jun 1, 2022
4e06fb1
Bug fix (oversight). Only valgrind got this one.
Jun 1, 2022
f892cce
Use shared_ptr custom deleter for m_fetched_error in error_already_se…
Jun 2, 2022
53e29f4
Further small simplification. With the GIL held, simply deleting the …
Jun 2, 2022
8e145c0
IWYU cleanup
Jun 2, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions include/pybind11/detail/class.h
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,8 @@ extern "C" inline void pybind11_object_dealloc(PyObject *self) {
#endif
}

std::string error_string();

/** Create the type which can be used as a common base for all classes. This is
needed in order to satisfy Python's requirements for multiple inheritance.
Return value: New reference. */
Expand Down Expand Up @@ -490,7 +492,7 @@ inline PyObject *make_object_base_type(PyTypeObject *metaclass) {
type->tp_weaklistoffset = offsetof(instance, weakrefs);

if (PyType_Ready(type) < 0) {
pybind11_fail("PyType_Ready failed in make_object_base_type():" + error_string());
pybind11_fail("PyType_Ready failed in make_object_base_type(): " + error_string());
}

setattr((PyObject *) type, "__module__", str("pybind11_builtins"));
Expand Down Expand Up @@ -707,7 +709,7 @@ inline PyObject *make_new_python_type(const type_record &rec) {
}

if (PyType_Ready(type) < 0) {
pybind11_fail(std::string(rec.name) + ": PyType_Ready failed (" + error_string() + ")!");
pybind11_fail(std::string(rec.name) + ": PyType_Ready failed: " + error_string());
}

assert(!rec.dynamic_attr || PyType_HasFeature(type, Py_TPFLAGS_HAVE_GC));
Expand Down
67 changes: 0 additions & 67 deletions include/pybind11/detail/type_caster_base.h
Original file line number Diff line number Diff line change
Expand Up @@ -470,73 +470,6 @@ PYBIND11_NOINLINE bool isinstance_generic(handle obj, const std::type_info &tp)
return isinstance(obj, type);
}

PYBIND11_NOINLINE std::string error_string(const char *called) {
error_scope scope; // Fetch error state (will be restored when this function returns).
if (scope.type == nullptr) {
if (called == nullptr) {
called = "pybind11::detail::error_string()";
}
pybind11_fail("Internal error: " + std::string(called)
+ " called while Python error indicator not set.");
}

PyErr_NormalizeException(&scope.type, &scope.value, &scope.trace);
if (scope.trace != nullptr) {
PyException_SetTraceback(scope.value, scope.trace);
}

std::string errorString;
if (scope.type) {
errorString += handle(scope.type).attr("__name__").cast<std::string>();
errorString += ": ";
}
if (scope.value) {
errorString += (std::string) str(scope.value);
}

#if !defined(PYPY_VERSION)
if (scope.trace) {
auto *trace = (PyTracebackObject *) scope.trace;

/* Get the deepest trace possible */
while (trace->tb_next) {
trace = trace->tb_next;
}

PyFrameObject *frame = trace->tb_frame;
Py_XINCREF(frame);
errorString += "\n\nAt:\n";
while (frame) {
# if PY_VERSION_HEX >= 0x030900B1
PyCodeObject *f_code = PyFrame_GetCode(frame);
# else
PyCodeObject *f_code = frame->f_code;
Py_INCREF(f_code);
# endif
int lineno = PyFrame_GetLineNumber(frame);
errorString += " ";
errorString += handle(f_code->co_filename).cast<std::string>();
errorString += '(';
errorString += std::to_string(lineno);
errorString += "): ";
errorString += handle(f_code->co_name).cast<std::string>();
errorString += '\n';
Py_DECREF(f_code);
# if PY_VERSION_HEX >= 0x030900B1
auto *b_frame = PyFrame_GetBack(frame);
# else
auto *b_frame = frame->f_back;
Py_XINCREF(b_frame);
# endif
Py_DECREF(frame);
frame = b_frame;
}
}
#endif

return errorString;
}

PYBIND11_NOINLINE handle get_object_handle(const void *ptr, const detail::type_info *type) {
auto &instances = get_internals().registered_instances;
auto range = instances.equal_range(ptr);
Expand Down
20 changes: 12 additions & 8 deletions include/pybind11/pybind11.h
Original file line number Diff line number Diff line change
Expand Up @@ -2625,17 +2625,21 @@ void print(Args &&...args) {
detail::print(c.args(), c.kwargs());
}

error_already_set::~error_already_set() {
if (m_type) {
gil_scoped_acquire gil;
error_scope scope;
m_type.release().dec_ref();
m_value.release().dec_ref();
m_trace.release().dec_ref();
}
inline void
error_already_set::m_fetched_error_deleter(detail::error_fetch_and_normalize *raw_ptr) {
gil_scoped_acquire gil;
Skylion007 marked this conversation as resolved.
Show resolved Hide resolved
error_scope scope;
delete raw_ptr;
}

inline const char *error_already_set::what() const noexcept {
gil_scoped_acquire gil;
error_scope scope;
return m_fetched_error->error_string().c_str();
}

PYBIND11_NAMESPACE_BEGIN(detail)

inline function
get_type_override(const void *this_ptr, const type_info *this_type, const char *name) {
handle self = get_object_handle(this_ptr, this_type);
Expand Down
Loading