repair_names() feature requests

[jennybc]

I've just started running readxl's output through repair_names() so it stops producing tibbles with empty, NA, or duplicated column names 🎉.

But I noticed that tibble::repair_names() and readr are not consistent.

x <- list(1:3,
          var2 = letters[1:3],
          c(1, 2, 3) + 0.1,
          var2 = letters[26:24])
names(x)[3] <- NA

tibble::repair_names(tibble::as_data_frame(x, validate = FALSE))
#> # A tibble: 3 × 4
#>      V1  var2    V2 var21
#>   <int> <chr> <dbl> <chr>
#> 1     1     a   1.1     z
#> 2     2     b   2.1     y
#> 3     3     c   3.1     x

readr::read_csv(",var2,,var2\n1,'a',1.1,'z'\n2,'b',2.1,'y'\n3,'c',3.1,'x'")
#> Warning: Missing column names filled in: 'X1' [1], 'X3' [3]
#> Warning: Duplicated column names deduplicated: 'var2' => 'var2_1' [4]
#> # A tibble: 3 × 4
#>      X1  var2    X3 var2_1
#>   <int> <chr> <dbl>  <chr>
#> 1     1   'a'   1.1    'z'
#> 2     2   'b'   2.1    'y'
#> 3     3   'c'   3.1    'x'

Feature requests, some inspired by readr:

• Report what and where re: modifications
• Number wrt absolute column position, as opposed to relative within the missing names
• Separate the de-deduplicating suffix with something regex-friendly
• Use a double underscore as prefix for new names and separator in de-duplication, e.g. __X1 and var2__1.

The first one is for a better interactive experience. Otherwise, aimed at programmatic work with tibbles that may have been subjected to name repair.

[krlmlr]

We could add arguments absolute = FALSE and verbose = interactive(), in addition to the existing prefix and sep arguments. This doesn't change the default behavior, but allows to replicate what readr does. Would that help?

[jennybc]

Yes, it certainly helps.

It feels like the tidyverse should have a preferred remedy for nonexistent and duplicated column names. And tibble is currently the logical place to implement that. So I'd still like to reach a consensus across packages.

What do you think of exporting make_unique()? Sometimes you have the names before you have the object.

[hadley]

I'd like to make repair_names() do what readr currently does so readr can just call it.

I think it's probably a good idea to have two variants: one that works on a character vector of names, and the other that works on a named vector.

[jennybc]

Perhaps my suggestion about leading underscores is not so bright. It creates non-syntactic names. I still think the convention should be something easily detectable via regex, i.e. unlikely to be present in original column names.

I expected sep to only be used during de-duplication.

x <- list(1:3,
          var2 = letters[1:3],
          c(1, 2, 3) + 0.1,
          var2 = letters[26:24])
names(x)[3] <- NA

tibble::repair_names(tibble::as_data_frame(x, validate = FALSE),
                     prefix = "__X", sep = "__")
#> # A tibble: 3 × 4
#>   `__X__1`  var2 `__X__2` var2__1
#>      <int> <chr>    <dbl>   <chr>
#> 1        1     a      1.1       z
#> 2        2     b      2.1       y
#> 3        3     c      3.1       x

[jimhester]

A leading . would work for this and is not very likely to happen by chance, you can even use two .. and still be syntactically valid.

[jennybc]

@jimhester Do you propose . or .. as the separator to be used for deduplication as well? Seems like a good idea.

[jennybc]

Moving towards a consensus on what should happen in current example:

..X1, var2, ..X3, var2..1

Or we could decide whenever we incorporate a number, it refers to absolute column position

..X1, var2, ..X3, var2..4

Even more consistent? Drop the V or X.

..1, var2, ..3, var2..4

Now every name that needs repair simply gets ..j appended.

[jimhester]

FWIW I think the doubled identifiers are quite ugly, but I suppose that is part of the point...

[jennybc]

Yeah, I think it is a feature if the repaired names are unsightly.

[hadley]

I don't like the use of dots because it goes against the usual convention to use _. But I don't see a good alternative here, given that you can't start a syntactic name with _.

[krlmlr]

The way dplyr::rename() currently works, the names don't need to be syntactic. On the contrary, making them non-syntactic might encourage the user to use proper names as soon as possible, or to drop with select(-starts_with("__")).

I particularly like @jennybc's idea of simply appending the absolute position, without using V or X. I'm mildly concerned about backward compatibility, we might want to use pkgconfig to simplify transition from the old behavior.

[hadley]

I think we should:

• Switch to from relative to absolute positions.
• Use .. as common prefix. I think it's the best compromise of ugliness and utility.
• Show what transformation is happening (a la readr). We can add quiet = FALSE to suppress
• Add a syntactic argument which, when TRUE, ensures that all names are syntactic

I'm not sure what we should do with respect to backward compatibility. I suspect few packages depend on this behaviour; it's going to be more user code. I think as long as there's a clear way to get to the old behaviour it shouldn't cause much hassle (especially since we're now describing what's happening)

[jennybc]

@krlmlr What's the outlook on this re: next tibble release?