selectize column filters when values can have >1 option

[agilly]

This issue looks at a point that has been raised several times, e.g. #331 or #1154, but I think it could make for a nice improvement.

Consider this table:

# Sample data
data <- data.frame(
  Food = c("Peanut Butter", "Almond Milk", "Soy Sauce", "Wheat Bread", "Egg Salad", "Everything Bagel", "sesame oil" ),
  Allergens = c("peanuts", "almonds", "soy", "wheat", "eggs", "wheat,sesame", "sesame")
)

In particular the allergens column

Allergens
------
peanuts
almonds
soy
wheat
eggs
wheat,sesame
sesame

We've repeatedly had the use case that people want the nice selectize input that is awarded for factor columns in the column filters in the case where records can contain >1 option, such as the row before last. Here are the options I could find:

Option 1: regexes

The solution proposed in #1154 is to use a regex:

DT::datatable(test_df, filter = "top", options = list(search = list(regex = TRUE)))

Which allows users to type sesame|wheat and this would select rows 4,6,7 above. The problem here is that people do need to think about a regex and type them in full, without the convenience of a selectize. This also matters when clearing the filters, which is easier to do manually for a selectize than a typed regex. I consider this as more of a workaround than a solution.

Option 2: show all combinations

One way to get back the nice selectize behavior is to convert the column to a factor, which will display all options. This means you may have options wheat,sesame, wheat and sesame. This is kind of fine for the example above where the options are not "deeply combined", but in some use cases you can have combinations used pretty extensively, which makes users click through a large number of (potentially long) options to catch all the cases they want. This is not really a solution IMO.

Option 3: External control

library(shiny)
library(DT)

# Split allergens for selectizeInput options
allergen_options <- unique(unlist(strsplit(data$Allergens, ",")))

# Define UI
ui <- fluidPage(
  titlePanel("Food Allergen Filter"),
  selectizeInput("allergen_select", "Select Allergens:", choices = allergen_options, multiple = TRUE),
  radioButtons("and_or", "AND / OR", choices = list("AND" = "AND", "OR" = "OR"), inline = TRUE),
  DTOutput("food_table")
)

# Define server logic
server <- function(input, output, session) {
  output$food_table <- renderDT({
    datatable(
      data,
      filter = "top",
      options = list(search = list(regex = TRUE, caseInsensitive = TRUE)),
      rownames = FALSE
    )
  }, server = FALSE)
  
  proxy <- dataTableProxy("food_table")
  
  observe({
    req(input$allergen_select)
    if (input$and_or == "AND") {
      # Create a regex pattern that requires all selected allergens to be present
      search_pattern <- paste0("(?=.*\\b", paste(input$allergen_select, collapse = "\\b)(?=.*\\b"), "\\b)")
    } else {
      # Create a regex pattern that matches any of the selected allergens
      search_pattern <- paste(input$allergen_select, collapse = "|")
    }
    # Update the search filter for the Allergens column
    proxy %>% updateSearch(keywords = list(columns = c("", search_pattern)))
  })
}

# Run the application
shinyApp(ui = ui, server = server, options = list(port = 8877, launch.browser = FALSE))

This externalizes the creation of a regex to other controls. The issue here is that it takes real estate in the app. While it does away with the need to write regex filters yourself, it does still require you to clear them manually.

Ideas

I think it would be better if something like this was natively supported by DT. Here's some thoughts about the how:

1. Allow DT to support lists of factors. It's possible in e.g. data.table to have column type list, where each cell would contain a list of values. It could be supported natively, with elements displayed comma-separated (or a separator specified in the settings). There are issues around verification: list elements can theoretically have arbitrary types. The plus is that this would be the most intuitive for the user.
2. use settings to specify behavior. Settings have been introduced as a result of Pass options to selectize to sort the filter levels #1083. Maybe there could be a way to specify filter settings like list(options=allergen_options, combine="or", target=2). This would then handle the filtering while displaying the nice selectize boxes and search in the column header.

Let me know what you think!

By filing an issue to this repo, I promise that

• I have fully read the issue guide at https://yihui.org/issue/.
• I have provided the necessary information about my issue.
  • If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
  • If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('DT'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('rstudio/DT').
  • If I have posted the same issue elsewhere, I have also mentioned it in this issue.
• I have learned the Github Markdown syntax, and formatted my issue correctly.

I understand that my issue may be closed if I don't fulfill my promises.

r$> xfun::session_info('DT')
R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.3 LTS

Locale:
  LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8        LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8    LC_PAPER=C.UTF-8      
  LC_NAME=C              LC_ADDRESS=C           LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

Package version:
  base64enc_0.1.3   bslib_0.7.0       cachem_1.1.0      cli_3.6.2         crosstalk_1.2.0   digest_0.6.35     DT_0.29           evaluate_0.24.0   fastmap_1.2.0     fontawesome_0.5.2
  fs_1.6.4          glue_1.7.0        graphics_4.3.1    grDevices_4.3.1   highr_0.10        htmltools_0.5.8.1 htmlwidgets_1.6.4 httpuv_1.6.15     jquerylib_0.1.4   jsonlite_1.9.0   
  knitr_1.46        later_1.3.2       lazyeval_0.2.2    lifecycle_1.0.4   magrittr_2.0.3    memoise_2.0.1     methods_4.3.1     mime_0.12         promises_1.3.2    R6_2.5.1         
  rappdirs_0.3.3    Rcpp_1.0.12       rlang_1.1.5       rmarkdown_2.29    sass_0.4.9        stats_4.3.1       tinytex_0.49      tools_4.3.1       utils_4.3.1       xfun_0.50        
  yaml_2.3.8