allow mutate() to allow combination of labelled and double classes

[bhaney22]

I would like the dplyr::mutate() command to allow an input variable with combination of "labelled", "haven_labelled", and "double" classes.

---

My workflow:

1. Use haven::read_sav() to read in an SPSS file, from the General Social Survey for example
2. Use the expss::mis_val() to recode var2 to set some categories as NA and remove labels associated with those categories
3. Use dplyr::mutate(newvar2 = case_when(var2 == 1 ~ 1, var2 == 2 ~ 0)) to create a new variable

EDIT: For clarity so that later comments will make sense, I added the "vctr_vctrs" class to the list of classes in the following statement so that it was complete.

Problem: Step 2. above causes var2 to have this combination of classes: "labelled" "haven_labelled" "vctrs_vctr" "double". This causes dplyr::mutate() to throw the following error when I try to do Step 3.

Error: Problem with mutate() input newvar2.
x Can't combine ..1 <labelled> and ..2 .
ℹ Input newvar2 is case_when(var2 == 1 ~ 1, var2 == 2 ~ 0).

I would like dplyr::mutate() to allow variables with this combination of classes ("labelled" "haven_labelled" "double") as input variables.

NOTE: dplyr::mutate() currently allows an input variable to have a combination of: "haven_labelled" "double"

Below is a minimum working example.

library(haven)
library(expss)
library(dplyr)

var1 <- haven::labelled(c(0, 1, 9, 0, 0),c(`No` = 0, `Yes` = 1, `DK` = 9))
var2 <- haven::labelled(c(0, 1, 2, 3, 1), c(`DK` = 0, `first` = 1, `second` = 2, `third` = 3))
mytest <- tibble(var1,var2)

# Classes before Step 2.
str(attributes(mytest$var1))
str(attributes(mytest$var2))

mytest$var1   <- expss::mis_val(mytest$var1, c(9))
mytest$var2   <- expss::mis_val(mytest$var2, c(0), with_labels = TRUE)

# Classes after Step 2.
str(attributes(mytest$var1))
str(attributes(mytest$var2))

# dplyr::mutate() works with input variable with combination of "haven_labelled" and "double" classes
mytest <- mytest %>%
  mutate(newvar1 = case_when(var1 == 1 ~ 1, var1 ==0 ~ 2)) 

# dplyr::mutate() does not work with combination of "labelled"  "haven_labelled" and "double" classes 

mytest <- mytest %>%
  mutate(newvar2 = case_when(var2 == 1 ~ 1, var2 == 2  ~ 0)) 

# Running the above command should produce the error commented below that I would like fixed.
#   Error: Problem with `mutate()` input `newvar2`.
#   x Can't combine `..1` <labelled<double>> and `..2` <double>.
#   ℹ Input `newvar2` is `case_when(var2 == 1 ~ 1, var2 == 2 ~ 0)`.

[bhaney22]

Since this is an issue with a combination of packages, I also initiated the following request with expss:

gdemin/expss#68 (comment)

[hadley]

Could you please rework your reproducible example to use the reprex package ? That makes it easier to see both the input and the output, formatted in such a way that I can easily re-run in a local session.

[bhaney22]

library(haven)
library(expss)
#> 
#> Attaching package: 'expss'
#> The following objects are masked from 'package:haven':
#> 
#>     is.labelled, read_spss
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:expss':
#> 
#>     between, compute, contains, first, last, na_if, recode, vars
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union

var1 <- haven::labelled(c(0, 1, 9, 0, 0),c(`No` = 0, `Yes` = 1, `DK` = 9))
var2 <- haven::labelled(c(0, 1, 2, 3, 1), c(`DK` = 0, `first` = 1, `second` = 2, `third` = 3))
mytest <- tibble(var1,var2)

# Classes before Step 2.
str(attributes(mytest$var1))
#> List of 2
#>  $ labels: Named num [1:3] 0 1 9
#>   ..- attr(*, "names")= chr [1:3] "No" "Yes" "DK"
#>  $ class : chr [1:3] "haven_labelled" "vctrs_vctr" "double"
str(attributes(mytest$var2))
#> List of 2
#>  $ labels: Named num [1:4] 0 1 2 3
#>   ..- attr(*, "names")= chr [1:4] "DK" "first" "second" "third"
#>  $ class : chr [1:3] "haven_labelled" "vctrs_vctr" "double"

mytest$var1   <- expss::mis_val(mytest$var1, c(9))
mytest$var2   <- expss::mis_val(mytest$var2, c(0), with_labels = TRUE)

# Classes after Step 2.
str(attributes(mytest$var1))
#> List of 2
#>  $ labels: Named num [1:3] 0 1 9
#>   ..- attr(*, "names")= chr [1:3] "No" "Yes" "DK"
#>  $ class : chr [1:3] "haven_labelled" "vctrs_vctr" "double"
str(attributes(mytest$var2))
#> List of 2
#>  $ labels: Named num [1:3] 1 2 3
#>   ..- attr(*, "names")= chr [1:3] "first" "second" "third"
#>  $ class : chr [1:4] "labelled" "haven_labelled" "vctrs_vctr" "double"

# dplyr::mutate() works with input variable with combination of "haven_labelled" and "double" classes
mytest <- mytest %>%
  mutate(newvar1 = case_when(var1 == 1 ~ 1, var1 ==0 ~ 2)) 

# dplyr::mutate() does not work with combination of "labelled"  "haven_labelled" and "double" classes 

 mytest <- mytest %>%
   mutate(newvar2 = case_when(var2 == 1 ~ 1, var2 == 2  ~ 0))
#> Error: Problem with `mutate()` input `newvar2`.
#> x Can't combine `..1` <labelled<double>> and `..2` <double>.
#> ℹ Input `newvar2` is `case_when(var2 == 1 ~ 1, var2 == 2 ~ 0)`.

[bhaney22]

EDIT: Here is the reproducible example with the session info included.

library(haven)
library(expss)
#> 
#> Use 'expss_output_rnotebook()' to display tables inside R Notebooks.
#>  To return to the console output, use 'expss_output_default()'.
#> 
#> Attaching package: 'expss'
#> The following objects are masked from 'package:haven':
#> 
#>     is.labelled, read_spss
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:expss':
#> 
#>     between, compute, contains, first, last, na_if, recode, vars
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union

var1 <- haven::labelled(c(0, 1, 9, 0, 0),c(`No` = 0, `Yes` = 1, `DK` = 9))
var2 <- haven::labelled(c(0, 1, 2, 3, 1), c(`DK` = 0, `first` = 1, `second` = 2, `third` = 3))
mytest <- tibble(var1,var2)

# Classes before Step 2.
str(attributes(mytest$var1))
#> List of 2
#>  $ labels: Named num [1:3] 0 1 9
#>   ..- attr(*, "names")= chr [1:3] "No" "Yes" "DK"
#>  $ class : chr [1:3] "haven_labelled" "vctrs_vctr" "double"
str(attributes(mytest$var2))
#> List of 2
#>  $ labels: Named num [1:4] 0 1 2 3
#>   ..- attr(*, "names")= chr [1:4] "DK" "first" "second" "third"
#>  $ class : chr [1:3] "haven_labelled" "vctrs_vctr" "double"

mytest$var1   <- expss::mis_val(mytest$var1, c(9))
mytest$var2   <- expss::mis_val(mytest$var2, c(0), with_labels = TRUE)

# Classes after Step 2.
str(attributes(mytest$var1))
#> List of 2
#>  $ labels: Named num [1:3] 0 1 9
#>   ..- attr(*, "names")= chr [1:3] "No" "Yes" "DK"
#>  $ class : chr [1:3] "haven_labelled" "vctrs_vctr" "double"
str(attributes(mytest$var2))
#> List of 2
#>  $ labels: Named num [1:3] 1 2 3
#>   ..- attr(*, "names")= chr [1:3] "first" "second" "third"
#>  $ class : chr [1:4] "labelled" "haven_labelled" "vctrs_vctr" "double"

# dplyr::mutate() works with input variable with combination of "haven_labelled" and "double" classes
mytest <- mytest %>%
  mutate(newvar1 = case_when(var1 == 1 ~ 1, var1 ==0 ~ 2)) 

# dplyr::mutate() does not work with combination of "labelled"  "haven_labelled" and "double" classes 

 mytest <- mytest %>%
   mutate(newvar2 = case_when(var2 == 1 ~ 1, var2 == 2  ~ 0)) 
#> Error: Problem with `mutate()` input `newvar2`.
#> x Can't combine `..1` <labelled<double>> and `..2` <double>.
#> ℹ Input `newvar2` is `case_when(var2 == 1 ~ 1, var2 == 2 ~ 0)`.

as_tibble(mytest)
#> # A tibble: 5 x 3
#>        var1        var2 newvar1
#>   <dbl+lbl>   <dbl+lbl>   <dbl>
#> 1   0 [No]  NA                2
#> 2   1 [Yes]  1 [first]        1
#> 3  NA        2 [second]      NA
#> 4   0 [No]   3 [third]        2
#> 5   0 [No]   1 [first]        2

sessionInfo()
#> R version 3.6.3 (2020-02-29)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: Ubuntu 18.04.4 LTS
#> 
#> Matrix products: default
#> BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
#> LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#> [1] dplyr_1.0.0  expss_0.10.5 haven_2.3.1 
#> 
#> loaded via a namespace (and not attached):
#>  [1] Rcpp_1.0.5         rstudioapi_0.11    knitr_1.29         magrittr_1.5      
#>  [5] hms_0.5.3          tidyselect_1.1.0   R6_2.4.1           rlang_0.4.7       
#>  [9] fansi_0.4.1        stringr_1.4.0      highr_0.8          tools_3.6.3       
#> [13] htmlTable_2.0.1    checkmate_2.0.0    data.table_1.12.8  xfun_0.15         
#> [17] utf8_1.1.4         cli_2.0.2          matrixStats_0.56.0 htmltools_0.5.0   
#> [21] ellipsis_0.3.1     assertthat_0.2.1   yaml_2.2.1         digest_0.6.25     
#> [25] tibble_3.0.3       lifecycle_0.2.0    crayon_1.3.4       purrr_0.3.4       
#> [29] htmlwidgets_1.5.1  vctrs_0.3.2        glue_1.4.1         evaluate_0.14     
#> [33] rmarkdown_2.3      stringi_1.4.6      compiler_3.6.3     pillar_1.4.6      
#> [37] generics_0.0.2     backports_1.1.8    forcats_0.5.0      foreign_0.8-76    
#> [41] pkgconfig_2.0.3

[hadley]

I'd say the root cause is the expss package — it shouldn't be adding the labelled class on to haven_labelled (or if it's doing it deliberately, it is then its responsibility to handle the combination of haven_labelled and labelled vectors)

[bhaney22]

Thank you for your quick response!

Actually, however, I respectfully disagree. I think this issue is related to haven. The error that results from the dplyr command is: <error/vctrs_error_incompatible_type> that results from haven's use of "vctr_vctrs" class.

Why do I think this?
Upon further research, I found a nearly identical open issue on haven:
tidyverse/haven#534. If you run my updated reprex example below, and then run the rlang::last_trace() command, you will get nearly the same results as in this haven open issue comment: tidyverse/haven#534 (comment).

In that haven open issue, the "vctr_vctrs" class when combined with haven's "labelled _spss" class results in the <error/vctrs_error_incompatible_type> in the vctrs package in base R for the sd() command, but not with the mean() command. So using the "vctr_vctrs" class seems to lead to unexpected errors.

In the code below, I remove the "vctr_vctrs" class and dplyr::mutate no longer causes an error. The input
variable still has both classes: "labelled" and "haven_labelled", and the error did not re-occur.

This suggests that the issue has to do with haven's use of the "vctr_vctrs" class, and not the labelled class as I had thought.

library(haven)
library(expss)
#> 
#> Use 'expss_output_viewer()' to display tables in the RStudio Viewer.
#>  To return to the console output, use 'expss_output_default()'.
#> 
#> Attaching package: 'expss'
#> The following objects are masked from 'package:haven':
#> 
#>     is.labelled, read_spss
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:expss':
#> 
#>     between, compute, contains, first, last, na_if, recode, vars
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union

var1 <- haven::labelled(c(0, 1, 9, 0, 0),c('No' = 0, 'Yes' = 1, 'DK' = 9))
var2 <- haven::labelled(c(0, 1, 2, 3, 1), c('DK' = 0, 'first' = 1, 'second' = 2, 'third' = 3))
mytest <- tibble(var1,var2)

str(attributes(mytest$var1))
#> List of 2
#>  $ labels: Named num [1:3] 0 1 9
#>   ..- attr(*, "names")= chr [1:3] "No" "Yes" "DK"
#>  $ class : chr [1:3] "haven_labelled" "vctrs_vctr" "double"
str(attributes(mytest$var2))
#> List of 2
#>  $ labels: Named num [1:4] 0 1 2 3
#>   ..- attr(*, "names")= chr [1:4] "DK" "first" "second" "third"
#>  $ class : chr [1:3] "haven_labelled" "vctrs_vctr" "double"

mytest$var1   <- expss::mis_val(mytest$var1, c(9))
mytest$var2   <- expss::mis_val(mytest$var2, c(0), with_labels = TRUE)

str(attributes(mytest$var1))
#> List of 2
#>  $ labels: Named num [1:3] 0 1 9
#>   ..- attr(*, "names")= chr [1:3] "No" "Yes" "DK"
#>  $ class : chr [1:3] "haven_labelled" "vctrs_vctr" "double"
str(attributes(mytest$var2))
#> List of 2
#>  $ labels: Named num [1:3] 1 2 3
#>   ..- attr(*, "names")= chr [1:3] "first" "second" "third"
#>  $ class : chr [1:4] "labelled" "haven_labelled" "vctrs_vctr" "double"

mytest <- mytest %>%
  mutate(newvar1 = case_when(var1 == 1 ~ 1, var1 ==0 ~ 2)) 

mytest <- mytest %>%
   mutate(newvar2 = case_when(var2 == 1 ~ 1, var2 == 2  ~ 0)) 
#> Error: Problem with `mutate()` input `newvar2`.
#> x Can't combine `..1` <labelled<double>> and `..2` <double>.
#> ℹ Input `newvar2` is `case_when(var2 == 1 ~ 1, var2 == 2 ~ 0)`.

rlang::last_error()
#> Error: Can't show last error because no error was recorded yet

rlang::last_trace()
#> Error: Can't show last error because no error was recorded yet

class(mytest$var2) = setdiff(class(mytest$var2), "vctrs_vctr")

str(attributes(mytest$var1))
#> List of 2
#>  $ labels: Named num [1:3] 0 1 9
#>   ..- attr(*, "names")= chr [1:3] "No" "Yes" "DK"
#>  $ class : chr [1:3] "haven_labelled" "vctrs_vctr" "double"
str(attributes(mytest$var2))
#> List of 2
#>  $ labels: Named num [1:3] 1 2 3
#>   ..- attr(*, "names")= chr [1:3] "first" "second" "third"
#>  $ class : chr [1:3] "labelled" "haven_labelled" "double"

mytest <- mytest %>%
  mutate(newvar2 = case_when(var2 == 1 ~ 1, var2 == 2  ~ 0))

# sessionInfo()

[hadley]

That’s a different problem. You can’t just pull off various pieces of the class vector and expect things to work. I’m the author of the haven_labelled class and a significant contributor to vctrs_vctr class and I’m at least reasonably confident the problem lies with expss.

[bhaney22]

I hear you, and I know that. Believe me, I do not take communicating what I believe is a problem with a tidyverse object to its creator lightly. And, I realize I am an old-school social scientist that is only two years into the R and tidyverse project and don't have much github cred, yet.

But, would you explain why you do not believe this issue has the same root problem as the current open issue with haven::labelled_spss: tidyverse/haven#534 (comment) ?

[hadley]

Yes, it's the same symptoms but it's a different root cause. The error message is a generic error message that occurs when we don't know how to combine two types of vector (we have recently adopted a principle that when we're not sure how to combine two different vectors we should throw an error rather than blinding smashing them together in the hope that it works). Here's the same basic error in a different context:

vctrs::vec_c(factor("x"), Sys.Date())
#> Error: Can't combine `..1` <factor<5a425>> and `..2` <date>.

^{Created on 2020-07-20 by the reprex package (v0.3.0)}

You can learn more about the vctrs package and how it interacts with dplyr at https://www.tidyverse.org/blog/2020/04/dplyr-1-0-0-and-vctrs/

[ronnyTodgers]

When did you make this change as it has thoroughly broken a number of our workflows, I'd like to hold the haven package at the point before you made the change.

Our workflows do not use expss. We simply label vars with haven and output zsavs - that side of things still works fine but with the new principle of throwing errors rather than "smashing them together" as you put it - doing lots of things to the labelled object just fail - I mean you really can combine a labelled double class and a non labelled double class right? it seems a little "error-happy" to throw some of these errors?

Now I see what's happening I can probably work around it - essentially stripping labels off and relabelling after - but it does feel a bit unnecessary and is impossible when using external packages to generate codebooks from the labelled data.

[hadley]

@ronnyTodgers please file an issue with reprex at https://github.com/tidyverse/haven