Skip to content

Add check to validate_forecast() that predictions must not decrease with quantile #412

@nikosbosse

Description

@nikosbosse

Update: Current plan is to:

  • write a helper function to check that
  • enforce it in bias_quantile() as it's needed there
  • check + warn in validate_forecast().

This is currently implemented in bias_quantile_single_vector()

order <- order(quantile)
predicted <- predicted[order]
if (!all(diff(predicted) >= 0)) {
    stop("Predictions must not be decreasing with increasing quantile level")
  }

In principle, we could add this to assert_input_quantile(). Since assert_input_quantile() is called by every metric, it might add a bit of overhead, but I'm unsure how much. If we're concerned about overhead, we could also add it to validate_forecast() once.
Or not check at all..

another code snipped I had lying around somewhere:

booster_data |>
  arrange(model, target, submission_date, target_end_date, scale, quantile_level) |>
  group_by(model, target, submission_date, target_end_date, scale) |>
  mutate(value = predicted - lag(predicted)) |>
  filter(value < 0 )

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    No status

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions