uninformative "outcome should be a factor" error with missing predictors

[mariamaseng]

Hi! I want to use xgboost for classification but are getting a (simple?) error stating that my outcome variable is not a factor which it indeed is. Is there another better way of making the outcome variable be interpreted as a factor in the model?

All help is much appreciated.

train_test_split = rsample::initial_split(df_num, prop=train_set_prop, strata=kombucha_intake_num)
train = training(train_test_split)
test = testing(train_test_split)
train$kombucha_intake_num =as.factor(train$kombucha_intake_num)
test$kombucha_intake_num = as.factor(test$kombucha_intake_num)

xgb_mod <- 
  boost_tree(trees=50) %>%
  set_engine("xgboost") %>%
  set_mode("classification") %>% 
  fit(kombucha_intake_num ~ ., data=train)`

Error: Error in check_outcome():
! For a classification model, the outcome should be a factor.

[simonpcouch]

Thanks for the issue!

Can you please provide a minimal reprex (reproducible example)? A reprex will help me troubleshoot and fix your issue more quickly.🙂 Particularly, printing out class(train$kombucha_intake_num) would be especially helpful—I would expect that to read "factor".

[mariamaseng]

Yes, I will try making that. Until I figure how to use repex, here is this

[mariamaseng]

With reprex:

`class(train$kombucha_intake_num)`
`#> Error in eval(expr, envir, enclos): object 'train' not found
`
  xgb_mod <- 
  boost_tree(trees=50) %>%
  set_engine("xgboost") %>%
  set_mode("classification") %>% 
  fit(kombucha_intake_num ~ ., data=train)
`#> Error in boost_tree(trees = 50) %>% set_engine("xgboost") %>% set_mode("classification") %>% : could not find function `"%>%"`
`

I guess I need to have the df(?) within the repex({}), because I was able to class outside of it, but not inside...

[simonpcouch]

I would recommend converting to factor before your initial split, just to ensure the factor levels line up between your training and testing sets. I wouldn't expect that this is the cause of your issue, though—are you confident that your updated version of train is what you've passed to fit()?

Here's a quick reprex you can start off from:

library(tidymodels)
  
set.seed(1)

dat <- tibble(y = rep(c(0, 1), 50), x = rnorm(100))
 
dat
#> # A tibble: 100 × 2
#>        y      x
#>    <dbl>  <dbl>
#>  1     0 -0.626
#>  2     1  0.184
#>  3     0 -0.836
#>  4     1  1.60 
#>  5     0  0.330
#>  6     1 -0.820
#>  7     0  0.487
#>  8     1  0.738
#>  9     0  0.576
#> 10     1 -0.305
#> # … with 90 more rows

dat <- dat %>% mutate(y = as.factor(y))
dat_split <- initial_split(dat)
dat_train <- training(dat_split)

xgb_mod <- 
  boost_tree(trees=50) %>%
  set_engine("xgboost") %>%
  set_mode("classification")

fit(xgb_mod, y ~ x, data = dat_train)
#> parsnip model object
#> 
#> ##### xgb.Booster
#> raw: 62.8 Kb 
#> call:
#>   xgboost::xgb.train(params = list(eta = 0.3, max_depth = 6, gamma = 0, 
#>     colsample_bytree = 1, colsample_bynode = 1, min_child_weight = 1, 
#>     subsample = 1), data = x$data, nrounds = 50, watchlist = x$watchlist, 
#>     verbose = 0, nthread = 1, objective = "binary:logistic")
#> params (as set within xgb.train):
#>   eta = "0.3", max_depth = "6", gamma = "0", colsample_bytree = "1", colsample_bynode = "1", min_child_weight = "1", subsample = "1", nthread = "1", objective = "binary:logistic", validate_parameters = "TRUE"
#> xgb.attributes:
#>   niter
#> callbacks:
#>   cb.evaluation.log()
#> # of features: 1 
#> niter: 50
#> nfeatures : 1 
#> evaluation_log:
#>     iter training_logloss
#>        1        0.6277038
#>        2        0.5890678
#> ---                      
#>       49        0.3394112
#>       50        0.3375767

^{Created on 2023-02-22 with reprex v2.0.2}

[mariamaseng]

Thanks a lot.
I have converted to factor before split as well (did both as it doesnt work), and yes, updated trainset. I only have two levels in the factor. Your example works well.
I feel stuck and not sure how to share more info with you, but thanks a lot. Understand that it is not possible to help much...

[simonpcouch]

Ah, got it!

I was able to replicate this error by supplying training data with no complete cases, i.e. there is at least one predictor in every row with a missing value:

library(parsnip)

fit(
  boost_tree("classification", "xgboost"), 
  y ~ x, 
  data = data.frame(y = factor(rep(c(0, 1), 10)), x = NA)
)
#> Error in `check_outcome()`:
#> ! For a classification model, the outcome should be a factor.

#> Backtrace:
#>     ▆
#>  1. ├─generics::fit(...)
#>  2. └─parsnip::fit.model_spec(...)
#>  3.   └─parsnip:::form_xy(...)
#>  4.     └─parsnip:::check_outcome(env$y, object)
#>  5.       └─rlang::abort("For a classification model, the outcome should be a factor.")

^{Created on 2023-02-22 with reprex v2.0.2}

@mariamaseng, the error that we supply is unhelpful and ought to be improved.

If these conditions are true for your case (i.e. nrow(complete.cases(train)) == 0), you may want to look into recipe steps for imputation or removing highly missing columns.

[mariamaseng]

Yes, you made it! That was it exactly. Thank you so much. You are awsome!!

[simonpcouch]

Great to hear it! I'll re-open this issue to make sure it stays on our radar until it's fixed.

[EmilHvitfeldt]

and it would be awesome if we could have messages that list what the wrong input was. Like the outcome should be a factor, but was a "numeric"

[simonpcouch]

By the time this data reaches check_outcome() in the above reprex, class(y) returns c("matrix", "array"). This happens because will_make_matrix() misses the case of a 0-length factor:

parsnip/R/convert_data.R

Lines 326 to 336 in 2588181

	will_make_matrix <- function(y) {
	if (is.matrix(y) | is.vector(y))
	return(FALSE)
	cls <- unique(unlist(lapply(y, class)))
	if (length(cls) > 1)
	return(FALSE)
	can_convert <-
	vapply(y, function(x)
	is.atomic(x) & !is.factor(x), logical(1))
	all(can_convert)
	}

and thus converts y to a 0 x 1 character matrix. A factor is not a vector, and the can_convert() apply() statement returns logical(0) even though the function it applies applied only to y itself would be TRUE.

[simonpcouch]

@EmilHvitfeldt I agree! Let's make that a separate issue.