Skip to content

add extract_fit_time() for tuning results #880

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

simonpcouch
Copy link
Contributor

Parking these changes here with no intention to merge. This PR would introduce built-in support for extract_fit_time() without requiring the user to pass extract_fit_time() to control_*(), but it's likely not worth the added complexity.

library(tidymodels)

bt_res <-
  fit_resamples(
    workflow(
      recipe(tip ~ ., taxi) %>% step_dummy(all_nominal_predictors()),
      boost_tree("classification")
    ),
    vfold_cv(taxi)
  )

# Note: no changes to the contents of the tibble
bt_res
#> # Resampling results
#> # 10-fold cross-validation 
#> # A tibble: 10 × 4
#>    splits              id     .metrics         .notes          
#>    <list>              <chr>  <list>           <list>          
#>  1 <split [9000/1000]> Fold01 <tibble [3 × 4]> <tibble [0 × 4]>
#>  2 <split [9000/1000]> Fold02 <tibble [3 × 4]> <tibble [0 × 4]>
#>  3 <split [9000/1000]> Fold03 <tibble [3 × 4]> <tibble [0 × 4]>
#>  4 <split [9000/1000]> Fold04 <tibble [3 × 4]> <tibble [0 × 4]>
#>  5 <split [9000/1000]> Fold05 <tibble [3 × 4]> <tibble [0 × 4]>
#>  6 <split [9000/1000]> Fold06 <tibble [3 × 4]> <tibble [0 × 4]>
#>  7 <split [9000/1000]> Fold07 <tibble [3 × 4]> <tibble [0 × 4]>
#>  8 <split [9000/1000]> Fold08 <tibble [3 × 4]> <tibble [0 × 4]>
#>  9 <split [9000/1000]> Fold09 <tibble [3 × 4]> <tibble [0 × 4]>
#> 10 <split [9000/1000]> Fold10 <tibble [3 × 4]> <tibble [0 × 4]>

extract_fit_time(bt_res)
#> # A tibble: 30 × 5
#>    id     stage      process_id          time .config             
#>    <chr>  <chr>      <chr>              <dbl> <chr>               
#>  1 Fold01 preprocess prep.dummy_osY1q 0.00700 Preprocessor1_Model1
#>  2 Fold01 preprocess bake.dummy_osY1q 0.0190  Preprocessor1_Model1
#>  3 Fold01 model      boost_tree       0.089   Preprocessor1_Model1
#>  4 Fold02 preprocess prep.dummy_osY1q 0.00100 Preprocessor1_Model1
#>  5 Fold02 preprocess bake.dummy_osY1q 0.0170  Preprocessor1_Model1
#>  6 Fold02 model      boost_tree       0.087   Preprocessor1_Model1
#>  7 Fold03 preprocess prep.dummy_osY1q 0.00200 Preprocessor1_Model1
#>  8 Fold03 preprocess bake.dummy_osY1q 0.0170  Preprocessor1_Model1
#>  9 Fold03 model      boost_tree       0.0880  Preprocessor1_Model1
#> 10 Fold04 preprocess prep.dummy_osY1q 0.00200 Preprocessor1_Model1
#> # ℹ 20 more rows

extract_fit_time(bt_res, summarize = TRUE)
#> # A tibble: 20 × 4
#>    id     stage        time .config             
#>    <chr>  <chr>       <dbl> <chr>               
#>  1 Fold01 model      0.089  Preprocessor1_Model1
#>  2 Fold01 preprocess 0.0260 Preprocessor1_Model1
#>  3 Fold02 model      0.087  Preprocessor1_Model1
#>  4 Fold02 preprocess 0.0180 Preprocessor1_Model1
#>  5 Fold03 model      0.0880 Preprocessor1_Model1
#>  6 Fold03 preprocess 0.0190 Preprocessor1_Model1
#>  7 Fold04 model      0.0880 Preprocessor1_Model1
#>  8 Fold04 preprocess 0.0180 Preprocessor1_Model1
#>  9 Fold05 model      0.0900 Preprocessor1_Model1
#> 10 Fold05 preprocess 0.0190 Preprocessor1_Model1
#> # ℹ 10 more rows

Created on 2024-03-27 with reprex v2.1.0

[no ci]

@simonpcouch
Copy link
Contributor Author

Related to tidymodels/workflows#191 and friends.

Copy link

This pull request has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.

@github-actions github-actions bot locked and limited conversation to collaborators Apr 11, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant