Illegal instruction on FreeBSD

[probonopd]

git clone https://github.com/LeFroid/Viper-Browser.git viper-browser
mkdir -p viper-browser/build && cd viper-browser/build
cmake ..
make -j4
./src/app/viper-browser

zsh: illegal hardware instruction  ./src/app/viper-browser

The crash seems to happen when viper-browser tries to do stuff with /tmp/qipc_sharedmemory_ViperBrowserIP....

truss ./src/app/viper-browser log attached:

truss.log.zip

System information:

[user@FreeBSD ~/viper-browser/build]$ uname -v
FreeBSD 12.2-RELEASE r366954 GENERIC 

[user@FreeBSD ~/viper-browser/build]$ pkg info qt5-webengine
qt5-webengine-5.15.2
Name           : qt5-webengine
Version        : 5.15.2
Installed on   : Tue Apr  6 08:25:36 2021 CEST

[LeFroid]

Looking into it

[probonopd]

Thanks 👍 Let me know if I can help by providing more testing/logs but you need to tell me exactly which commands to run as I still need to smarten up on gdb and friends.

[LeFroid]

So far I'm unable to reproduce the issue, using FreeBSD 12.2 AMD64 on a VirtualBox VM.
Are you able to use LLDB or GDB and send the stack trace? This page may help if you can spare the time, https://lldb.llvm.org/use/tutorial.html

The browser would have to be recompiled with these commands

cmake -DCMAKE_BUILD_TYPE=Debug .. && make -j4

Edit: The basic flow with lldb would look like this,

$ lldb ./src/app/viper-browser
  ...
 > run
   (wait for crash)
> thread backtrace all

[probonopd]

Interesting that you are not seeing the crash.
Could it be dependent on the CPU model?

git clone https://github.com/LeFroid/Viper-Browser.git viper-browser
mkdir -p viper-browser/build && cd viper-browser/build
git checkout 92c2c3b # Last one without libsecret dependency which draws in glib, gobject, gio, see #26
cmake -DCMAKE_BUILD_TYPE=Debug .. && make -j4
lldb ./src/app/viper-browser
run
thread backtrace all

Does not crash.

Also, just running ./src/app/viper-browser built in this way does not crash.

But as soon as I build with -DCMAKE_BUILD_TYPE=Release it does crash:

FreeBSD% ./src/app/viper-browser                                
zsh: illegal hardware instruction  ./src/app/viper-browser
FreeBSD% lldb ./src/app/viper-browser
(lldb) target create "./src/app/viper-browser"
Current executable set to '/tmp/viper-browser/build/src/app/viper-browser' (x86_64).
(lldb) run
Process 5218 launching
Process 5218 launched: '/tmp/viper-browser/build/src/app/viper-browser' (x86_64)
Process 5218 stopped
* thread #1, name = 'viper-browser', stop reason = signal SIGILL: privileged instruction
    frame #0: 0x000000000063e800 viper-browser`std::__1::pair<std::__1::__hash_iterator<std::__1::__hash_node<std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, void*>*>, bool> std::__1::__hash_table<std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::__unordered_map_hasher<AppInitKey, std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, AppInitKeyHash, true>, std::__1::__unordered_map_equal<AppInitKey, std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<AppInitKey>, true>, std::__1::allocator<std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > >::__emplace_unique_key_args<AppInitKey, std::__1::pair<AppInitKey const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > const&>(AppInitKey const&, std::__1::pair<AppInitKey const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > const&) + 32
viper-browser`std::__1::__hash_table<std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::__unordered_map_hasher<AppInitKey, std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, AppInitKeyHash, true>, std::__1::__unordered_map_equal<AppInitKey, std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<AppInitKey>, true>, std::__1::allocator<std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > >::__emplace_unique_key_args<AppInitKey, std::__1::pair<AppInitKey const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > const&>:
->  0x63e800 <+32>: popcntq %r13, %rcx
    0x63e805 <+37>: cmpq   $0x1, %rcx
    0x63e809 <+41>: jbe    0x63e836                  ; <+86>
    0x63e80b <+43>: movq   %r12, %r8
(lldb) thread backtrace all
* thread #1, name = 'viper-browser', stop reason = signal SIGILL: privileged instruction
  * frame #0: 0x000000000063e800 viper-browser`std::__1::pair<std::__1::__hash_iterator<std::__1::__hash_node<std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, void*>*>, bool> std::__1::__hash_table<std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::__unordered_map_hasher<AppInitKey, std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, AppInitKeyHash, true>, std::__1::__unordered_map_equal<AppInitKey, std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<AppInitKey>, true>, std::__1::allocator<std::__1::__hash_value_type<AppInitKey, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > >::__emplace_unique_key_args<AppInitKey, std::__1::pair<AppInitKey const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > const&>(AppInitKey const&, std::__1::pair<AppInitKey const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > const&) + 32
    frame #1: 0x000000000063c415 viper-browser`AppInitSettings::AppInitSettings() + 229
    frame #2: 0x00000000005cb75f viper-browser`main + 1279
    frame #3: 0x00000000005cade0 viper-browser`_start(ap=<unavailable>, cleanup=<unavailable>) at crt1.c:76:7
(lldb) 

Race condition? Something gets unallocated before it should, and adding Debug symbols changes the timing? Just speculating here.

[LeFroid]

That is interesting. The system was handling a custom hashing function differently in the Debug vs Release builds.
I've updated that portion of the code, and with a fresh release build, the browser is starting up successfully. Please let me know if you still see the illegal instruction getting thrown on your system.

[probonopd]

Hello @LeFroid thanks for taking the time, glad you were able to find the root cause.

Would love to test but cannot due to #26 (comment).

[ 49%] Linking CXX shared library libCredentialStoreSecret.so
ld: error: unable to find library -lsecret-1
ld: error: unable to find library -lgio-2.0
ld: error: unable to find library -lgobject-2.0
ld: error: unable to find library -lglib-2.0
ld: error: unable to find library -lintl
c++: error: linker command failed with exit code 1 (use -v to see invocation)
--- src/plugins/credential_store_secret/libCredentialStoreSecret.so ---
*** [src/plugins/credential_store_secret/libCredentialStoreSecret.so] Error code 1

make[2]: stopped in /tmp/viper-browser/build
1 error

make[2]: stopped in /tmp/viper-browser/build
--- src/plugins/credential_store_secret/CMakeFiles/CredentialStoreSecret.dir/all ---
*** [src/plugins/credential_store_secret/CMakeFiles/CredentialStoreSecret.dir/all] Error code 2

make[1]: stopped in /tmp/viper-browser/build
--- src/ui/CMakeFiles/viper-ui_autogen.dir/all ---
*** [src/ui/CMakeFiles/viper-ui_autogen.dir/all] Error code 2

make[1]: stopped in /tmp/viper-browser/build
2 errors

make[1]: stopped in /tmp/viper-browser/build
*** [all] Error code 2

make: stopped in /tmp/viper-browser/build
1 error

make: stopped in /tmp/viper-browser/build
zsh: no such file or directory: ./src/app/viper-browser
FreeBSD% 

Please make libCredentialStoreSecret optaional, thanks!

[LeFroid]

That plugin should only be compiled if you have all the requisite headers and libraries on your system. It is supposed to be conditionally pulled in by CMake.

I'll take a closer look at the CMake files to check for correctness, but in the meantime you can force the plugin to be disabled by passing -DDISABLE_LIBSECRET=1 to cmake (since d9322f9)

[probonopd]

Unfortunately this did not help yet:

git clone https://github.com/LeFroid/Viper-Browser.git viper-browser
mkdir -p viper-browser/build && cd viper-browser/build
cmake .. -DDISABLE_LIBSECRET=1 -DDISABLE_KWALLET=1
make -j4
./src/app/viper-browser
zsh: illegal hardware instruction  ./src/app/viper-browser

git log
commit d9322f9b4f786f59c19938fa7794fa664f81a95e (HEAD -> master, tag: continuous, origin/master, origin/HEAD)

[LeFroid]

I'm still unable to reproduce this on a clean debug or release build.
What version of clang or gcc was used to compile the browser? Are any custom kernel settings enabled in /etc/sysctl.conf or elsewhere? Any custom compiler flags? Or any other bits of information that you think would help me to reproduce this on my end?

[probonopd]

The CXX compiler identification is Clang 10.0.1.
No custom settings I am aware of.

Let's find out whether this is a compile-time issue or a runtime issue. Can you please run

mkdir -p viper-browser/build && cd viper-browser/build
cmake .. -DDISABLE_LIBSECRET=1 -DDISABLE_KWALLET=1
make -j4 DESTDIR=appdir
zip -r -y Viper.zip appdir

and upload it? Then I could test whether your build can run on my machine without a crash.

[LeFroid]

Ok, same compiler and most likely same kernel settings as I tested with.
Could be something unique about my VirtualBox guest environment, if this is a runtime issue.

Following the steps you listed above, here is the output: Viper.zip

[probonopd]

Thanks @LeFroid. This also gives me the segfault. So it is not about the compiler but somehow about my runtime environment. Will test more.