[R][C++] Fix crashing test in test-dataset.R on 32-bit Windows from ARROW-7979

[asfimport]

If we can obtain a gdb backtrace from the failed test in #6638 then we can sort out what's wrong.

Reporter: Wes McKinney / @wesm
Assignee: Neal Richardson / @nealrichardson

Related issues:

• [C++] Implement experimental buffer compression in IPC messages (relates to)
• [C++] Add -Wa,-mbig-obj when compiling with MinGW to avoid linking errors (relates to)
• [R] Fix segfault in Table to Array conversion (is related to)

PRs and other links:

• GitHub Pull Request #6799

_{Note: This issue was originally created as ARROW-8217. Please see the migration documentation for further details.}

[asfimport]

Neal Richardson / @nealrichardson:
I managed to run with gdb (included in Rtools) like this:

C:\Program Files\R\R-3.6.0\bin\i386>\Rtools\mingw_32\bin\gdb R.exe

Unfortunately, there's no backtrace on exit:

Start test: IPC/Arrow format data
  test-dataset.R#172:1 [success]
  test-dataset.R#173:1 [success]
[Inferior 1 (process 4920) exited with code 030000000005]
(gdb) bt
No stack.
(gdb)

Retrying with break exit, there's nothing useful, presumably because R needs to be rebuilt with debug symbols:

Start test: IPC/Arrow format data
  test-dataset.R#172:1 [success]
  test-dataset.R#173:1 [success]
Breakpoint 1, 0x767866f5 in msvcrt!exit () from C:\Windows\System32\msvcrt.dll
(gdb) bt
#0  0x767866f5 in msvcrt!exit () from C:\Windows\System32\msvcrt.dll
#1  0x00405787 in ?? ()
#2  0x004013e2 in ?? ()
#3  0x74ea6359 in KERNEL32!BaseThreadInitThunk () from C:\Windows\System32\kernel32.dll
#4  0x77247b74 in ntdll!RtlGetAppContainerNamedObjectPath () from C:\Windows\SYSTEM32\ntdll.dll
#5  0x77247b44 in ntdll!RtlGetAppContainerNamedObjectPath () from C:\Windows\SYSTEM32\ntdll.dll
#6  0x00000000 in ?? ()
(gdb)

On the one hand, no backtrace could mean that it's not a segfault, just some other hard exit. On the other, googling the exit code comes up with lots of hits suggesting that it is a segfault.

If someone wants to tell me other ideas of places to set breakpoints or whatever to try to see what's failing, I can try. I can also try making a DEBUG build of Arrow C++ (this is a release build). I'd rather not have to build R from source to get more debug symbols.

[asfimport]

Wes McKinney / @wesm:
A debug build of Arrow should tell you more, it's worth a shot

[asfimport]

Wes McKinney / @wesm:
If the R nightly setup can build a DEBUG package I can try to install that and debug myself, too

[asfimport]

Neal Richardson / @nealrichardson:
I can't get a debug build. I tried twice and the build hangs: https://github.com/ursa-labs/arrow-r-nightly/runs/535320005

I see two options:

1. Try to write the test that fails in R in C++ and see if that fails somewhere else. It's possible that R itself is not the problem, we just don't have a test that exercises this code path anywhere else.
2. Let this one lie until someone reports it. The exposure of this particular scenario is low right now: you'd have to be an R user, using 32-bit R on Windows, querying a dataset made of IPC/Feather files.

[asfimport]

Wes McKinney / @wesm:
I think a release build with debug symbols would be sufficient, can you try building with -DARROW_CXXFLAGS="-g". If that doesn't work then we can do one of the options you listed

[asfimport]

Neal Richardson / @nealrichardson:
I did a build with -DARROW_CXXFLAGS="-g", no difference. That was on the C++ build only though, any reason to think it would matter to compile the R bindings with that too?

Even after the hanging build was resolved on master, I still can't get a debug build: https://github.com/ursa-labs/arrow-r-nightly/runs/538449507?check_suite_focus=true#step:7:1126

[ 16%] Building CXX object src/arrow/CMakeFiles/arrow_static.dir/record_batch.cc.obj
C:/Rtools/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/as.exe: CMakeFiles/arrow_static.dir/array/diff.cc.obj: too many sections (43914)
D:\a\_temp\msys\msys64\tmp\ccqm015L.s: Assembler messages:
D:\a\_temp\msys\msys64\tmp\ccqm015L.s: Fatal error: can't write CMakeFiles/arrow_static.dir/array/diff.cc.obj: File too big
C:/Rtools/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/as.exe: CMakeFiles/arrow_static.dir/array/diff.cc.obj: too many sections (43914)
D:\a\_temp\msys\msys64\tmp\ccqm015L.s: Fatal error: can't close CMakeFiles/arrow_static.dir/array/diff.cc.obj: File too big

[asfimport]

Wes McKinney / @wesm:
It sort of suggests that that the segfault is originating in the R bindings instead of the C++ library, which would be weird but I suppose it's possible.

I think the debug build error can be resolved with ARROW-8246

[asfimport]

Wes McKinney / @wesm:
Since the debug build issue should be resolved, we might make one last attempt to get a backtrace before 0.17.0 goes out

[asfimport]

Neal Richardson / @nealrichardson:
Seems to have worked this time: https://github.com/ursa-labs/arrow-r-nightly/runs/550684026?check_suite_focus=true

So I'll try again with that tomorrow.

[asfimport]

Neal Richardson / @nealrichardson:
Finally got a local debug build installed (the R bindings failed on the link step on the 64-bit version but 32-bit worked, and that's all I needed).

Interestingly, the test executed cleanly, no crash...

[asfimport]

Wes McKinney / @wesm:
Issue resolved by pull request 6799
#6799

[asfimport]

Wes McKinney / @wesm:
See discussion in the PR. It seems that this was "fixed" by ARROW-7740. We didn't figure out exactly the reason, though