Add a function that can query the defaults of an underlying engine

[DavisVaughan]

In get_model_env() we document the original parameter names of the underlying engine

> parsnip::get_model_env()$decision_tree_args
# A tibble: 6 x 5
  engine parsnip         original               func             has_submodel
  <chr>  <chr>           <chr>                  <list>           <lgl>       
1 rpart  tree_depth      maxdepth               <named list [2]> FALSE       
2 rpart  min_n           minsplit               <named list [2]> FALSE       
3 rpart  cost_complexity cp                     <named list [2]> FALSE       
4 C5.0   min_n           minCases               <named list [2]> FALSE       
5 spark  tree_depth      max_depth              <named list [2]> FALSE       
6 spark  min_n           min_instances_per_node <named list [2]> FALSE  

but we don't say anything about the default values of those parameters.

If we record the function that those original parameters go along with, we could have a function that does a dynamic lookup like this:

lookup_default <- function(fn, arg) {
  arg_list <- formals(fn)
  
  has_arg <- arg %in% names(arg_list)
  if (!has_arg) {
    stop("Argument: `", arg, "` does not exist in the function.", call. = FALSE)
  }
  
  default_value <- formals(fn)[[arg]]
  default_value
}

lookup_default(rpart::rpart.control, "cp")
#> [1] 0.01

lookup_default(rpart::rpart.control, "cp_oops")
#> Error: Argument: `cp_oops` does not exist in the function.

It would be nice to expose this information to users in the form of a tibble like parsnip::get_model_env()$decision_tree_args that also has columns for the function that the original parameters come from, and the default values.

[apreshill]

Also, would be great to be able to pass these defaults to tune::fit_resamples(). So if this code:

library(tune)
dt_by_fold <- fit_resamples(downsampled, 
                            model = dt_learner, 
                            resamples = folds, 
                            metrics = metric_set(roc_auc),
                            control = control_resamples(save_pred = TRUE, 
                                                        save_defaults = TRUE))

Produced this kind of output:

> collect_metrics(dt_by_fold)
# A tibble: 10 x 7
   cost_complexity min_n .metric .estimator  mean     n std_err
             <dbl> <int> <chr>   <chr>      <dbl> <int>   <dbl>
 1            0.01   20  roc_auc binary     0.650    10  0.0157

[sckinta]

It seems that save_defaults is not a defined parameter in control_resamples function anymore.

control_resamples(
verbose = FALSE,
allow_par = TRUE,
extract = NULL,
save_pred = FALSE,
pkgs = NULL
)

[topepo]

@sckinta I don't think that there ever was a save_defaults option; I think that she was suggesting to add one.

[sckinta]

@topepo Gotach! Thank you for clarification!

[juliasilge]

We are going to close this issue as we think we've done what we can to surface the engine defaults in the model documentation. To check some out, see:

• random forest
• logistic regression
• boosted tree

There are enough special cases and hard-coded options that moving from this type of documentation to making the options available for interactive use, tuning, etc is not currently tractable.

If anyone wants to see the code that creates the engine default tables, check that out here, along with how it is used for a model! 👀

[github-actions]

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.