R options: options() are not passed down to futures - should they? (document either way)

[HenrikBengtsson]

I'm adding this as a reminder to myself to document this (next release), but also to consider more automated solutions (a future release):

R's global options() are not passed down to futures. This has to be done manually right now, e.g.

fopts <- options()[c("digits", "repos")]
f <- future({
  options(fopts)
  [...]
})

This works, but it would be nice to have an automated version of this as well (cf. globals and packages) to better capture the state of the calling R process. Doing this might be a bit tricky since some options are read only / shouldn't be changed and others are specific the local machine, which may not work on a remote machine. Maybe it's possible to use static-code inspection to identify what options are used (similar to how globals are identified).

[HenrikBengtsson]

Some ideas:

Ignoring any "automatic" passing down of options, this could be done by adding an argument options analogue to globals, e.g.

f <- future(..., options = c("digits", "repos"))

That construct provides the developer control over options. For the end user to control options, there would need to be a similar argument that can be specified when setting the plan(), e.g.

plan(batchtools_sge, workers = 100L, options = c("digits", "repos"))

Options specified via plan() will always be set/passed down and additional ones specified via future() would be done in addition to what's in plan().

[rubenarslan]

So, future.global.maxSize is also not passed down. That leaves me in a pickle.
I send jobs to a Torque cluster from my local. The jobs call brms, which has a future = TRUE argument, so that each of its chains is run according to plan. But these chains are bigger than 500Mb so this fails and I cannot change it, because I cannot alter options in the futures that brms creates.
I think all future options should always be passed down.

[HenrikBengtsson]

Good point, I agree that future.globals.maxSize should be passed down - this is an oversight. As a workaround for now, you can override the default in the ~/.Rprofile on the machine in the chain that complains, e.g.

options(future.globals.maxSize = 1.0 * 1024^3)

[HenrikBengtsson]

Moving the discussion of future.* options to Issue #223 because it's a much more narrow problem than considering any/all of R's options.