-
Notifications
You must be signed in to change notification settings - Fork 982
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
foverlaps memory issues in 32 bit windows #4580
Comments
Thanks for the report. |
This above issue is reproducible in the current development version of data.table. I tried an equivalent non-equi join as you suggested and I'm not seeing crashes or errors with the following code on either development or release data.table on i386 R.
|
This comment has been minimized.
This comment has been minimized.
…il I have a clear idea of what's going on to cause the crashes in windows i386. reported here: Rdatatable/data.table#4580
Thank you for providing extra info.
I tried few times, should that be enough? I have no idea how to proceed then, I don't have any other Windows machine to try that out. Could you provide flags that are being used during compilation? install.packages("data.table", type = "source", repos = "https://Rdatatable.gitlab.io/data.table")
#*** arch - i386
#"C:/rtools40/mingw32/bin/"gcc -I"C:/R-40~1.2/include" -DNDEBUG -fopenmp -O2 -Wall -std=gnu99 -mfpmath=sse -msse2 -mstackrealign -c assign.c -o assign.o and cat(readLines(system.file(package="data.table", "cc")), sep="\n")
#CC=gcc -std=gnu99
#CFLAGS=-g -O2 -fdebug-prefix-map=/build/r-base-aGvNeb/r-base-4.0.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g |
Sorry I pulled that from an existing test or something and forgot the seed, but the seed doesn't matter since I've been able to reproduce the error without that column even existing. Here's a slightly more minimal reprex:
Something is definitely weird though since there are sessions where I can't get it to happen at all but other sessions where it happens the first time I run foverlaps on these datasets. Actually, it seems like this error will not occur in the session in which the package is installed. try this:
I've replicated the above on both machines--the error doesn't occur in the session in which data.table was installed. Also it doesn't matter whether it was installed from source or binary. |
@myoung3 Thanks, now I am able to reproduce issue. |
@jangorecki Great, glad to hear my boxes aren't haunted. I've found that it's even a bit weirder than I described above. I can reproduce the issue in session in which data.table was installed, but it doesn't happen immediately.
which returns a "cannot allocate vector of size <> Gb" Whereas in a new session after installation, for me at least, the following causes an instant crash (not an error) on just a single foverlaps call:
|
I'll add that I tried replacing the |
@jangorecki yep that summary seems correct |
I'm getting some weird crashes exclusively on the i386 version of R in Windows both on win-builder and locally. All the crashes seem to have memory-related error messages, but the errors/crashes occur inconsistently (not every time I run the code) and the error seems to alternate randomly between "cannot allocate vector of length", "cannot allocate a vector of size", and an actual crash ("R encountered a fatal error" in Rstudio)
Here's a minimal reproducible example:
As you can see the datasets are small so despite i386's memory limitations this shouldn't be causing problems. The crash even happens with the
gc()
in the loop.Pretty sure this happens whether nomatch=NULL or the default.
The text was updated successfully, but these errors were encountered: