You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I understand that using OpenMP multi-threading in a fork of R session may end up in a deadlock as noted in this code. And threads are down limit to 1 in a fork. For safety concerns, the limit continues after fork.
This behavior makes RStudio Server users harder to leverage the multi-threading capabilities data.table has to offer since rstudio/rstudio#1827 notes that a refresh of RStudio server web page could trigger the behavior and since then data.table is limited to be single-threaded.
The recent release of fst package adopts the reset-after-fork behavior unless an option is toggled, as discussed at fstpackage/fst#110, fstpackage/fst#112. I'm not sure if it makes sense that data.table adopts this behavior in future if this is thoroughly tested?
The text was updated successfully, but these errors were encountered:
To be on the safe side of things, the default behavior for data.table could also be set to switch back to single-threaded mode after a fork, but to allow overriding that behavior with an option.
Once that proves stable, that logic could be reversed.
Perhaps an additional argument reset_after_fork could also be added to setDTthreads() to set this behavior after the package has been loaded (like in fst::threads_fst()), so:
I understand that using OpenMP multi-threading in a fork of R session may end up in a deadlock as noted in this code. And threads are down limit to 1 in a fork. For safety concerns, the limit continues after fork.
This behavior makes RStudio Server users harder to leverage the multi-threading capabilities data.table has to offer since rstudio/rstudio#1827 notes that a refresh of RStudio server web page could trigger the behavior and since then data.table is limited to be single-threaded.
The recent release of fst package adopts the reset-after-fork behavior unless an option is toggled, as discussed at fstpackage/fst#110, fstpackage/fst#112. I'm not sure if it makes sense that data.table adopts this behavior in future if this is thoroughly tested?
The text was updated successfully, but these errors were encountered: