implementation strategy: robust `ggplot2` functionality for visualizing `vctrs`-objects

[fabian-s]

posting here in the hopes of getting developer advice:

TL;DR:

vctrs-objects implemented as list-type vectors fail ggplot2's type checking for (continuous) scales, because scales are determined in train_position before stats are applied, and our vctrs- objects are only converted to x-y-tuples by the specific Stat we implement for our vctrs-class.

we are uncertain what a feasible and sensible strategy is:

• go the whole sfroute and implement entire custom coordinate systems etc (seems like it might be overkill in our case)
• just implement new primary scale types specifically for our vctrs-objects (how?)
• something else entirely?

we would appreciate any guidance or pointers to existing implementations solving similar issues.

more details:

Our package tf defines a vctrs-class tf for vectors of functions --basically: lists of 2-column matrices containing $(t, f_i(t))$-tuples.
We were able to define a custom StatTf and some custom geoms based on it (e.g. geom-spaghetti) to enable quick plots like this:

> library(tidyfun)
> library(ggplot2)
> d <- data.frame(
+     x = tf_rgp(3), #3 random functions
+     f = factor(c(1,1,2))
+ )    
> ggplot(d) + geom_spaghetti(aes(y = x, col = f))

This works for many different kinds of geoms and facetting etc etc but breaks down currently as soon as we try to use for example scale-transformations, see also tidyfun/tidyfun#135:

> ggplot(d) + geom_spaghetti(aes(y = x, col = f)) + scale_y_reverse()
Error in `as.character()`:
! Can't convert `x` <tfd_reg> to <character>.
Run `rlang::last_trace()` to see where the error occurred.
> rlang::last_trace()
<error/vctrs_error_cast>
Error in `as.character()`:
! Can't convert `x` <tfd_reg> to <character>.
---
Backtrace:
     ▆
  1. ├─base (local) `<fn>`(x)
  2. ├─ggplot2:::print.ggplot(x)
  3. │ ├─ggplot2::ggplot_build(x)
  4. │ └─ggplot2:::ggplot_build.ggplot(x)
  5. │   └─layout$train_position(data, scale_x(), scale_y())
  6. │     └─ggplot2 (local) train_position(..., self = self)
  7. │       └─self$facet$train_scales(...)
  8. │         └─ggplot2 (local) train_scales(...)
  9. │           └─ggplot2:::scale_apply(layer_data, y_vars, "train", SCALE_Y, y_scales)
[...]

[teunbrand]

A couple of quick observations:

go the whole sfroute and implement entire custom coordinate systems etc (seems like it might be overkill in our case)

Agreed on the assessment of overkill.

just implement new primary scale types specifically for our vctrs-objects (how?)

This should be possible, but I wouldn't set this in motion until other options are exhausted.

Error in as.character():

This error is thrown by the error message formatting of a different error. The message that should have reached you is the 'Discrete values supplied to continuous scale'. It fails to print examples of values it doesn't understand, as it cannot be converted to a character.

as soon as we try to use for example scale-transformations

The error triggers with any continuous scale, even plain scale_y_continuous() with no arguments.

I might come back to this after I have given this some more thought.

[fabian-s]

Thanks, I'd really appreciate any pointers, and, yes, you're correct about the "Discrete values supplied to continuous scale"-error -- I wanted to keep the description above as short as possible, but the core issue seems to be that scales are determined in train_position before stats are applied, i.e., before the plotting data for each layer is actually computed:

> ggplot2:::ggplot_build.ggplot
function (plot) 
{
    [....]
    scale_x <- function() scales$get_scales("x")
    scale_y <- function() scales$get_scales("y")   
    layout$train_position(data, scale_x(), scale_y())  #!! this is where the error happens
    data <- layout$map_position(data)
    data <- .expose_data(data)
    data <- by_layer(function(l, d) l$compute_statistic(d, layout),  #!! this is where tf get converted to x-y-tuples by StatTf
        layers, data, "computing stat")
    [....]
}

the error comes from ggplot2:::scale_apply at line

pieces <- lapply(seq_along(scales), function(i) { scales[[i]][[method]](data[[var]][scale_index[[i]]]) })

which calls:

scales[[1]][["train"]]
<ggproto method>
<Wrapper function>
  function (...) 
train(..., self = self)

<Inner function (f)>
  function (self, x) 
{
  if (length(x) == 0) {
      return()
  }
  if (is.factor(x) || !typeof(x) %in% c("integer", "double")) {
      if (inherits(self$range, "ContinuousRange")) {
          cli::cli_abort(c("Discrete values supplied to continuous scale.", 
              i = "Example values: {.and {.val {head(x, 5)}}}"), 
              call = self$call)
      }
  }
  self$range$train(x)
}

[teunbrand]

After some thought, I've convinced myself that the robust solution to this problem would be to have an S3 component to scales::train_continuous/_discrete(). That way, you could just write a method on how ranges should be updated with your vctrs class. Unfortunately, this takes a release cycle of {scales}/{ggplot2} for it to be available to you.

A frail solution would be to implement your own Scale class that can properly handle the vctrs class. In the past I've experimented with this and you can see an example here: https://github.com/teunbrand/ggchromatic/blob/master/R/scale_chromatic.R. However, I don't recommend this.

Instead, I'm going to recommend that geom_spaghetti() (or rather the Stat) does not have y as a required aesthetic. Instead, you choose an alternate aesthetic name, let's say z, that doesn't automatically invoke a position scale. Then in the Stat part of your layer, you "unpack" your vctr class from the z aesthetic into the usual x and y aesthetics just like regular computed aesthetics.

[fabian-s]

@teunbrand thanks so much -- not calling the required aesthetic y and then simply handling scale-transforms inside the Stat seems to work for all the use cases I could think of. You can close this if you want. Really appreciate your quick response.

[fabian-s]

to follow up on this, in case other people come across similar problems in the future:

another useful strategy here seems to be to simply override the user-specified aes-mapping to y with a mapping to tf (or whatever) inside your own geom_xx-functions (see code below)

this way:

• you still get to use a non-exotic aes-name like x or y in the user interface for your geom_xx
• ggplot_build will not mistake your weird aes-mapping of exotic vctrs-things with a standard position mapping and try to apply inappropriate scales at inopportune times

To do this, simply add something like:

if ("y" %in% names(mapping)) {
    names(mapping) <- sub("^y$", "tf", names(mapping))
  }

before your geom_xx function calls ggplot2::layer

[teunbrand]

I don't fully endorse editing the mapping inside your geom_xx() functions. It will work differently when mappings are directly supplied by user in the layer, or when mappings are inherited from the global mapping in the main ggplot() call. If you hardcode inherit.aes = FALSE, then this shouldn't be too much of a problem.

[teunbrand]

In any case, we're following up on training custom classes in r-lib/scales#480.
Since the problem at hand is resolved for now, I think it is time to close this issue.

[teunbrand]

From r-lib/scales#480 (comment):

it still remains the case that scale-training for ggplot2-layers on vctrs_vctr objects that also inherit from list does not work, because the train method for (e.g.) ScaleContinuous checks the typeof the original object, instead of the type of its range:

ggplot2 should be minimalist in its expectations and have {scales} handle expectations