-
Notifications
You must be signed in to change notification settings - Fork 584
Improve crashlog #8373
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve crashlog #8373
Conversation
64ecccc
to
460bd8f
Compare
460bd8f
to
3440f97
Compare
Requesting some preliminary reviews before also testing this on more platforms and older systems. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Feel free to squash the fixes into the respective comments.
bbf2853
to
8e3fbb3
Compare
8e3fbb3
to
b213d85
Compare
Package pipelines for more testing: |
Testing on FreeBSD showed that the symbol name resolution boost::stacktrace returns broken values there. Default format (
Custom output iterating over the frames in the stack using
Resolving the symbols from the Boost stacktrace using
I see the following possible options:
|
I'll try to do the following: Add a new define |
Remember to inform the packagers. |
About what in particular? |
About |
I would just set this in our CMakeLists.txt, so no need to update any packaging. |
0c6a5ff
to
81bdef0
Compare
…n and document behavior The logic for selecting the traces to print stays the same, but there are fewer nested ifs now. This changes the format of the returned string a bit by adding a heading for both traces.
Maybe this will save the next person who has to look at this code some time. Please don't blame me for the implementation, I'm just trying to reconstruct what it does.
Older versions of MSVC fail to rethrow an unhandled C++ exception (using `throw;`) in the termination handler (`std::set_terminate`), however Icinga relies on this behavior in its crash handler (`Application::ExceptionHandler`).
This makes the format more similar to what the uncaught C++ and SEH exception handlers write. Previously there was no indication in the crash log that a SIGABRT happened.
Unfortunately, the symbol resolution of boost::stacktrace is broken on FreeBSD, therefore fall back to using backtrace_symbols() to print the stack trace saved by Boost. Additionally, -D_GNU_SOURCE is required on FreeBSD for the _Unwind_Backtrace function used by boost::stacktrace.
So far, the check that actually sets HAVE_LIBEXECINFO was executed after it was already used to add dependencies.
Needed by `boost::stacktrace` for `_Unwind_Backtrace()`.
6cd09bb
to
00caf3d
Compare
- Explicitly disable optimizations for MSVC - Make stack_test_func_a bigger
00caf3d
to
c5626cb
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Great work! 👍
@Al2Klimov I just noticed that I pushed to this PR since your last review. Do you want to take another look at this before merging this? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Test subject
W/o vs. w/ this PR
Test base
Test results
Linux
DEBUG_ABORT
Before
root@debian-gnu-linux-10-vm:~# DEBUG_ABORT=1 icinga2 daemon -x warning
Caught SIGABRT.
Current time: 2021-06-29 12:39:57 +0200
[2021-06-29 12:39:57 +0200] critical/Application: Icinga 2 has terminated unexpectedly. Additional information can be found in '/var/log/icinga2/crash/report.1624963197.837719'
root@debian-gnu-linux-10-vm:~# cat /var/log/icinga2/crash/report.1624963197.837719
Application version: v2.12.0-511-gb36cf3e08
System information:
Platform: Debian GNU/Linux
Platform version: 10 (buster)
Kernel: Linux
Kernel version: 4.19.0-10-amd64
Architecture: x86_64
Build information:
Compiler: GNU 8.3.0
Build host: 0d188d083bb1
OpenSSL version: OpenSSL 1.1.1d 10 Sep 2019
Application information:
General paths:
Config directory: /etc/icinga2
Data directory: /var/lib/icinga2
Log directory: /var/log/icinga2
Cache directory: /var/cache/icinga2
Spool directory: /var/spool/icinga2
Run directory: /run/icinga2
Old paths (deprecated):
Installation root: /usr
Sysconf directory: /etc
Run directory (base): /run
Local state directory: /var
Internal paths:
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid
Stacktrace:
(0) libc.so.6: gsignal (+0x10b) [0x7f5e99cf27bb]
(1) libc.so.6: abort (+0x121) [0x7f5e99cdd535]
(2) icinga2: <unknown function> (+0x2c3ad8) [0x5639288e8ad8]
(3) icinga2: <unknown function> (+0x614dd0) [0x563928c39dd0]
(4) icinga2: icinga::DaemonCommand::Run(boost::program_options::variables_map const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) const (+0x9ab) [0x563928c3aa3b]
(5) icinga2: <unknown function> (+0x823316) [0x563928e48316]
(6) icinga2: main (+0xcf) [0x56392896218f]
(7) libc.so.6: __libc_start_main (+0xeb) [0x7f5e99cdf09b]
(8) icinga2: _start (+0x2a) [0x56392896223a]
***
* This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
* please submit a bug report at https://github.com/Icinga/icinga2 and include this stack trace as well as any other
* information that might be useful in order to reproduce this problem.
***
Failed to launch GDB: No such file or directory
root@debian-gnu-linux-10-vm:~#
After
root@debian-gnu-linux-10-vm:~# DEBUG_ABORT=1 icinga2 daemon -x warning
Caught SIGABRT.
Current time: 2021-06-29 12:42:32 +0200
[2021-06-29 12:42:32 +0200] critical/Application: Icinga 2 has terminated unexpectedly. Additional information can be found in '/var/log/icinga2/crash/report.1624963352.489917'
root@debian-gnu-linux-10-vm:~# cat /var/log/icinga2/crash/report.1624963352.489917
Caught SIGABRT.
Current time: 2021-06-29 12:42:32 +0200
Application version: v2.12.0-530-g74218807b
System information:
Platform: Debian GNU/Linux
Platform version: 10 (buster)
Kernel: Linux
Kernel version: 4.19.0-10-amd64
Architecture: x86_64
Build information:
Compiler: GNU 8.3.0
Build host: 4dd22d1c669b
OpenSSL version: OpenSSL 1.1.1d 10 Sep 2019
Application information:
General paths:
Config directory: /etc/icinga2
Data directory: /var/lib/icinga2
Log directory: /var/log/icinga2
Cache directory: /var/cache/icinga2
Spool directory: /var/spool/icinga2
Run directory: /run/icinga2
Old paths (deprecated):
Installation root: /usr
Sysconf directory: /etc
Run directory (base): /run
Local state directory: /var
Internal paths:
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid
Stacktrace:
0# icinga::Application::SigAbrtHandler(int) in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
1# 0x00007FC44C034730 in /lib/x86_64-linux-gnu/libpthread.so.0
2# gsignal in /lib/x86_64-linux-gnu/libc.so.6
3# abort in /lib/x86_64-linux-gnu/libc.so.6
4# 0x0000556795553B42 in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
5# 0x00005567958A5380 in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
6# icinga::DaemonCommand::Run(boost::program_options::variables_map const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) const in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
7# 0x0000556795AB41F6 in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
8# main in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
9# __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6
10# _start in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
***
* This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
* please submit a bug report at https://github.com/Icinga/icinga2 and include this stack trace as well as any other
* information that might be useful in order to reproduce this problem.
***
Failed to launch GDB: No such file or directory
root@debian-gnu-linux-10-vm:~#
DEBUG_THROW_CXX
Before
root@debian-gnu-linux-10-vm:~# DEBUG_THROW_CXX=1 icinga2 daemon -x warning
[2021-06-29 12:41:10 +0200] critical/cli: Exception in main process: Error: test exception
(0) icinga2: <unknown function> (+0x614dd0) [0x5582f9472dd0]
(1) icinga2: icinga::DaemonCommand::Run(boost::program_options::variables_map const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) const (+0x9ab) [0x5582f9473a3b]
(2) icinga2: <unknown function> (+0x823316) [0x5582f9681316]
(3) icinga2: main (+0xcf) [0x5582f919b18f]
(4) libc.so.6: __libc_start_main (+0xeb) [0x7f06bfb1b09b]
(5) icinga2: _start (+0x2a) [0x5582f919b23a]
root@debian-gnu-linux-10-vm:~#
After
root@debian-gnu-linux-10-vm:~# DEBUG_THROW_CXX=1 icinga2 daemon -x warning
[2021-06-29 12:43:23 +0200] critical/cli: Exception in main process: Error: test exception
Stacktrace:
0# __cxa_throw in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
1# 0x0000556843B78B3D in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
2# 0x0000556843ECA380 in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
3# icinga::DaemonCommand::Run(boost::program_options::variables_map const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) const in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
4# 0x00005568440D91F6 in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
5# main in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
6# __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6
7# _start in /usr/lib/x86_64-linux-gnu/icinga2/sbin/icinga2
root@debian-gnu-linux-10-vm:~#
Windows
DEBUG_ABORT
Before
Only "Retry" gives a result:
[2021-06-29 12:16:50 +0000] critical/Application: Icinga 2 has terminated unexpectedly. Additional information can be found in 'C:\ProgramData\icinga2\var\log\icinga2/crash/report.1624969010.804000'
PS C:\Users\Administrator>
Application version: v2.12.0-684-gfd0c9c39b
System information:
Platform: Windows
Platform version: 8
Kernel: Windows
Kernel version: 6.2
Architecture: x86_64
Build information:
Compiler: MSVC 19.29.30038.1
Build host: DESKTOP-V1GF2GS
OpenSSL version: OpenSSL 1.1.1h 22 Sep 2020
Application information:
General paths:
Config directory: C:\ProgramData\icinga2\etc\icinga2
Data directory: C:\ProgramData\icinga2\var\lib\icinga2
Log directory: C:\ProgramData\icinga2\var\log\icinga2
Cache directory: C:\ProgramData\icinga2\var\cache\icinga2
Spool directory: C:\ProgramData\icinga2\var\spool\icinga2
Run directory: C:\ProgramData\icinga2\var\run\icinga2
Old paths (deprecated):
Installation root: C:\Program Files\ICINGA2\
Sysconf directory: C:\ProgramData\icinga2\etc
Run directory (base): C:\ProgramData\icinga2\var\run
Local state directory: C:\ProgramData\icinga2\var
Internal paths:
Package data directory: C:\Program Files\ICINGA2\\share\icinga2
State path: C:\ProgramData\icinga2\var\lib\icinga2/icinga2.state
Modified attributes path: C:\ProgramData\icinga2\var\lib\icinga2/modified-attributes.conf
Objects path: C:\ProgramData\icinga2\var\cache\icinga2/icinga2.debug
Vars path: C:\ProgramData\icinga2\var\cache\icinga2/icinga2.vars
PID path: C:\ProgramData\icinga2\var\run\icinga2/icinga2.pid
Caught unhandled SEH exception.
Current time: 2021-06-29 12:16:50 +0000
Stacktrace:
(0): throw_test_exception+182
(1): RunWorker+978
(2): icinga::DaemonCommand::Run+2191
(3): Main+13665
(4): main+338
(5): invoke_main+57
(6): __scrt_common_main_seh+302
(7): __scrt_common_main+14
(8): mainCRTStartup+14
(9): BaseThreadInitThunk+20
(10): RtlUserThreadStart+33
***
* This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
* please submit a bug report at https://github.com/Icinga/icinga2 and include this stack trace as well as any other
* information that might be useful in order to reproduce this problem.
***
After
Caught unhandled SEH exception.
Current time: 2021-06-29 12:34:33 +0000
Application version: v2.12.0-704-ge6def2acf
System information:
Platform: Windows
Platform version: 8
Kernel: Windows
Kernel version: 6.2
Architecture: x86_64
Build information:
Compiler: MSVC 19.29.30038.1
Build host: DESKTOP-V1GF2GS
OpenSSL version: OpenSSL 1.1.1h 22 Sep 2020
Application information:
General paths:
Config directory: C:\ProgramData\icinga2\etc\icinga2
Data directory: C:\ProgramData\icinga2\var\lib\icinga2
Log directory: C:\ProgramData\icinga2\var\log\icinga2
Cache directory: C:\ProgramData\icinga2\var\cache\icinga2
Spool directory: C:\ProgramData\icinga2\var\spool\icinga2
Run directory: C:\ProgramData\icinga2\var\run\icinga2
Old paths (deprecated):
Installation root: C:\Program Files\ICINGA2\
Sysconf directory: C:\ProgramData\icinga2\etc
Run directory (base): C:\ProgramData\icinga2\var\run
Local state directory: C:\ProgramData\icinga2\var
Internal paths:
Package data directory: C:\Program Files\ICINGA2\\share\icinga2
State path: C:\ProgramData\icinga2\var\lib\icinga2/icinga2.state
Modified attributes path: C:\ProgramData\icinga2\var\lib\icinga2/modified-attributes.conf
Objects path: C:\ProgramData\icinga2\var\cache\icinga2/icinga2.debug
Vars path: C:\ProgramData\icinga2\var\cache\icinga2/icinga2.vars
PID path: C:\ProgramData\icinga2\var\run\icinga2/icinga2.pid
SEH exception:
Code: 0x80000003
Address: 00007FFECE6C7475
Flags: 0
Stacktrace:
0# boost::stacktrace::basic_stacktrace<std::allocator<boost::stacktrace::frame> >::init at C:\local\boost_1_71_0-Win64\boost\stacktrace\stacktrace.hpp:75
1# boost::stacktrace::basic_stacktrace<std::allocator<boost::stacktrace::frame> >::basic_stacktrace<std::allocator<boost::stacktrace::frame> > at C:\local\boost_1_71_0-Win64\boost\stacktrace\stacktrace.hpp:127
2# icinga::Application::SEHUnhandledExceptionFilter at C:\Users\Administrator\icinga2\lib\base\application.cpp:958
3# UnhandledExceptionFilter in KERNELBASE
4# memset in ntdll
5# _C_specific_handler in ntdll
6# _chkstk in ntdll
7# RtlWalkFrameChain in ntdll
8# KiUserExceptionDispatcher in ntdll
9# _threadid in ucrtbased
10# _threadid in ucrtbased
11# abort in ucrtbased
12# throw_test_exception at C:\Users\Administrator\icinga2\lib\cli\daemoncommand.cpp:225
13# RunWorker at C:\Users\Administrator\icinga2\lib\cli\daemoncommand.cpp:324
14# icinga::DaemonCommand::Run at C:\Users\Administrator\icinga2\lib\cli\daemoncommand.cpp:736
15# Main at C:\Users\Administrator\icinga2\icinga-app\icinga.cpp:660
16# main at C:\Users\Administrator\icinga2\icinga-app\icinga.cpp:946
17# invoke_main at d:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:79
18# __scrt_common_main_seh at d:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
19# __scrt_common_main at d:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:331
20# mainCRTStartup at d:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp:17
21# BaseThreadInitThunk in KERNEL32
22# RtlUserThreadStart in ntdll
***
* This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
* please submit a bug report at https://github.com/Icinga/icinga2 and include this stack trace as well as any other
* information that might be useful in order to reproduce this problem.
***
DEBUG_THROW_CXX
Before
"Retry" just returns the shell.
After
All quiet on the western front.
DEBUG_THROW_SEH
Before
PS C:\Users\Administrator> $Env:DEBUG_THROW_SEH = 1
PS C:\Users\Administrator> .\icinga2\build\Bin\Release\Debug\icinga2.exe daemon
[2021-06-29 12:25:38 +0000] information/cli: Icinga application loader (version: v2.12.0-684-gfd0c9c39b; debug)
[2021-06-29 12:25:38 +0000] information/cli: Loading configuration file(s).
[2021-06-29 12:25:38 +0000] information/ConfigItem: Committing config item(s).
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 1 NotificationComponent.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 1 CheckerComponent.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 1 User.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 1 UserGroup.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 2 NotificationCommands.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 3 ServiceGroups.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 10 Notifications.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 9 Services.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 1 ScheduledDowntime.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 3 Zones.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 3 TimePeriods.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 1 Endpoint.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 1 WindowsEventLogLogger.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 244 CheckCommands.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 2 HostGroups.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 1 Host.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Instantiated 1 IcingaApplication.
[2021-06-29 12:25:41 +0000] information/ScriptGlobal: Dumping variables to file 'C:\ProgramData\icinga2\var\cache\icinga2/icinga2.vars'
[2021-06-29 12:25:41 +0000] information/ConfigItem: Triggering Start signal for config items
[2021-06-29 12:25:41 +0000] information/NotificationComponent: 'notification' started.
[2021-06-29 12:25:41 +0000] information/CheckerComponent: 'checker' started.
[2021-06-29 12:25:41 +0000] information/ConfigItem: Activated all objects.
[2021-06-29 12:25:41 +0000] critical/Application: Icinga 2 has terminated unexpectedly. Additional information can be found in 'C:\ProgramData\icinga2\var\log\icinga2/crash/report.1624969541.631000'
PS C:\Users\Administrator>
Application version: v2.12.0-684-gfd0c9c39b
System information:
Platform: Windows
Platform version: 8
Kernel: Windows
Kernel version: 6.2
Architecture: x86_64
Build information:
Compiler: MSVC 19.29.30038.1
Build host: DESKTOP-V1GF2GS
OpenSSL version: OpenSSL 1.1.1h 22 Sep 2020
Application information:
General paths:
Config directory: C:\ProgramData\icinga2\etc\icinga2
Data directory: C:\ProgramData\icinga2\var\lib\icinga2
Log directory: C:\ProgramData\icinga2\var\log\icinga2
Cache directory: C:\ProgramData\icinga2\var\cache\icinga2
Spool directory: C:\ProgramData\icinga2\var\spool\icinga2
Run directory: C:\ProgramData\icinga2\var\run\icinga2
Old paths (deprecated):
Installation root: C:\Program Files\ICINGA2\
Sysconf directory: C:\ProgramData\icinga2\etc
Run directory (base): C:\ProgramData\icinga2\var\run
Local state directory: C:\ProgramData\icinga2\var
Internal paths:
Package data directory: C:\Program Files\ICINGA2\\share\icinga2
State path: C:\ProgramData\icinga2\var\lib\icinga2/icinga2.state
Modified attributes path: C:\ProgramData\icinga2\var\lib\icinga2/modified-attributes.conf
Objects path: C:\ProgramData\icinga2\var\cache\icinga2/icinga2.debug
Vars path: C:\ProgramData\icinga2\var\cache\icinga2/icinga2.vars
PID path: C:\ProgramData\icinga2\var\run\icinga2/icinga2.pid
Caught unhandled SEH exception.
Current time: 2021-06-29 12:25:41 +0000
Stacktrace:
(0): icinga::DaemonCommand::Run+2191
(1): Main+13665
(2): main+338
(3): invoke_main+57
(4): __scrt_common_main_seh+302
(5): __scrt_common_main+14
(6): mainCRTStartup+14
(7): BaseThreadInitThunk+20
(8): RtlUserThreadStart+33
***
* This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
* please submit a bug report at https://github.com/Icinga/icinga2 and include this stack trace as well as any other
* information that might be useful in order to reproduce this problem.
***
After
Caught unhandled SEH exception.
Current time: 2021-06-29 12:37:57 +0000
Application version: v2.12.0-704-ge6def2acf
System information:
Platform: Windows
Platform version: 8
Kernel: Windows
Kernel version: 6.2
Architecture: x86_64
Build information:
Compiler: MSVC 19.29.30038.1
Build host: DESKTOP-V1GF2GS
OpenSSL version: OpenSSL 1.1.1h 22 Sep 2020
Application information:
General paths:
Config directory: C:\ProgramData\icinga2\etc\icinga2
Data directory: C:\ProgramData\icinga2\var\lib\icinga2
Log directory: C:\ProgramData\icinga2\var\log\icinga2
Cache directory: C:\ProgramData\icinga2\var\cache\icinga2
Spool directory: C:\ProgramData\icinga2\var\spool\icinga2
Run directory: C:\ProgramData\icinga2\var\run\icinga2
Old paths (deprecated):
Installation root: C:\Program Files\ICINGA2\
Sysconf directory: C:\ProgramData\icinga2\etc
Run directory (base): C:\ProgramData\icinga2\var\run
Local state directory: C:\ProgramData\icinga2\var
Internal paths:
Package data directory: C:\Program Files\ICINGA2\\share\icinga2
State path: C:\ProgramData\icinga2\var\lib\icinga2/icinga2.state
Modified attributes path: C:\ProgramData\icinga2\var\lib\icinga2/modified-attributes.conf
Objects path: C:\ProgramData\icinga2\var\cache\icinga2/icinga2.debug
Vars path: C:\ProgramData\icinga2\var\cache\icinga2/icinga2.vars
PID path: C:\ProgramData\icinga2\var\run\icinga2/icinga2.pid
SEH exception:
Code: 0x2a
Address: 00007FFEFC885299
Flags: 0
Stacktrace:
0# boost::stacktrace::basic_stacktrace<std::allocator<boost::stacktrace::frame> >::init at C:\local\boost_1_71_0-Win64\boost\stacktrace\stacktrace.hpp:75
1# boost::stacktrace::basic_stacktrace<std::allocator<boost::stacktrace::frame> >::basic_stacktrace<std::allocator<boost::stacktrace::frame> > at C:\local\boost_1_71_0-Win64\boost\stacktrace\stacktrace.hpp:127
2# icinga::Application::SEHUnhandledExceptionFilter at C:\Users\Administrator\icinga2\lib\base\application.cpp:958
3# UnhandledExceptionFilter in KERNELBASE
4# memset in ntdll
5# _C_specific_handler in ntdll
6# _chkstk in ntdll
7# RtlWalkFrameChain in ntdll
8# RtlRaiseException in ntdll
9# RaiseException in KERNELBASE
10# throw_test_exception at C:\Users\Administrator\icinga2\lib\cli\daemoncommand.cpp:233
11# RunWorker at C:\Users\Administrator\icinga2\lib\cli\daemoncommand.cpp:324
12# icinga::DaemonCommand::Run at C:\Users\Administrator\icinga2\lib\cli\daemoncommand.cpp:736
13# Main at C:\Users\Administrator\icinga2\icinga-app\icinga.cpp:660
14# main at C:\Users\Administrator\icinga2\icinga-app\icinga.cpp:946
15# invoke_main at d:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:79
16# __scrt_common_main_seh at d:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
17# __scrt_common_main at d:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:331
18# mainCRTStartup at d:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp:17
19# BaseThreadInitThunk in KERNEL32
20# RtlUserThreadStart in ntdll
***
* This would indicate a runtime problem or configuration error. If you believe this is a bug in Icinga 2
* please submit a bug report at https://github.com/Icinga/icinga2 and include this stack trace as well as any other
* information that might be useful in order to reproduce this problem.
***
Conclusion
- Linux: information which IMAO doesn’t matter gets lost
- Windows: information which matters gets added
The current crash handlers show some shortcomings, especially on Windows. For example, if an uncaught C++ exception occurs on Windows, currently the SEH handler is executed which prints no information about the actual exception. Also the stacktrace printed on Windows is of little use for release builds as it only prints
(unknown function)
without even an address. This is addressed by using boost instead of the custom stacktrace implementation.New Crashlog Examples
Windows, Debug Build, SEH Exception
Windows, Debug Build, C++ Exception
Windows, Snapshot Build, SEH Exception
Windows, Snapshot Build, C++ Exception
Linux, Debug Build, C++ Exception
Linux, Debug Build, abort()
Linux, Snapshot Build, C++ Exception
Linux, Snapshot Build, abort()
TODO
icinga::StackTrace
can also be replaced withboost::stacktrace