[Feature request] function that tells what is the current strategy

[Jean-Romain]

Let define a simplified function in a package that looks like that:

pkg_fun <- function(fun, x) {
   f <- vector("list", length(x))
   for (i in seq_along(x)) {
      f[[i]] <- future::future({ fun(x) })
   }

   # some code to handle resolution and get values of futures...
}

Users can use it like that

library(pkg)
library(future)
plan(multisession)
pkg_fun(myfunction, mydata)

There is no way for the developer to know what is the plan used by the user. It is sequential? multicore on a single machine? multiple remote machine? I do agree that most of the time the developer do no need to know. But I'm personally facing multiple cases were I need to know to handle stuff internally or to throw informative warning or error. For examples:

1. myfunction may already be parallelized in C++ with OpenMP. Testing the strategy allows to enable or disable OpenMP and avoid nested parallelism
2. mydata can contain non serializable objects and the code subsequently fails with no informative error for parallel strategies (but works with plan(sequential)). Being able to catch those cases early would allow to handle them nicely.
3. myfunction can return non serializable objects. Same issue than 2.

So far I'm using a custom parser than analyses the ouptut of future::plan() but I think it would be a useful addition to add easy to parse functions for developpers in future. Something like that maybe

future::is_parallel_strategy()
future::get_strategy_workers()
future::is_remote_stragety()
future::is_nested_strategy()
# ...

Thanks

[HenrikBengtsson]

> current_plan <- plan()

> current_plan
sequential:
- args: function (..., envir = parent.frame())
- tweaked: FALSE
- call: NULL

> str(current_plan)
function (..., envir = parent.frame())  
 - attr(*, "class")= chr [1:5] "FutureStrategy" "sequential" "uniprocess" "future" ...

> inherits(current_plan, "sequential")
[1] TRUE
> 

[HenrikBengtsson]

Regarding your developer needs of being able to conditional evaluate via futures if the set future strategy supports it or not: Just as a heads-up, I'm working on adding support for specifying "resource" requirements per future. It's still another year or so, but the gist is that you'll be able to do things like:

f <- future({ some expression }, resources = ~ !multithreading)

to say the future should be evaluated without multithreading.

For non-serializable globals, there's also a plan to improve on that, cf. https://github.com/HenrikBengtsson/marshal. But, also not anywhere near being ready.

[HenrikBengtsson]

You might also be interested in nbrOfWorkers().

[Jean-Romain]

Thanks for your reply. Testing the class of the current plan is already the first step of my parser. But the entire code is more complex as it also tries to extract the number of workers from the plan. nbrOfWorkers() will likely simplify my code.

[jan-glx]

Per future resource requirements would indeed be awesome.
I currently hack around this with something like:

tryCatch({
    library(future)
    .att_bak <- attributes(environment(plan)[["stack"]][[1]])
    .future_mem_bak <<- formals(environment(plan)[["stack"]][[1]])$resources$memory
    formals(environment(plan)[["stack"]][[1]])$resources$memory <- as.character(fs::file_size("large_file_the_future_is_loading")]*3)
    attributes(environment(plan)[["stack"]][[1]]) <- .att_bak
}, error = \(e) warning(e))

x %<-% Sys.sleep(5)
x
tryCatch({ 
    .att_bak <- attributes(environment(plan)[["stack"]][[1]])
    formals(environment(plan)[["stack"]][[1]])$resources$memory <- .future_mem_bak
    attributes(environment(plan)[["stack"]][[1]]) <- .att_bak
}, error = \(e) warning(e))

(sry about slightly OT comment, did not find a more relevant issue to post this)

EDIT: backup and restore attributes of plan function ( did not realize they are dropped by ``formals<-`)