Possible Error in DER Loss, Matching Losses from TF Implementation

[EdwardBerman]

Dear all,
I came across the ML4PS paper regarding this work recently, and found it super useful to a study I am conducting! Great work.

I was wondering if someone could look into the definition of the DER loss. I am comparing with this TensorFlow implementation from the original paper here . Firstly, in line 453 in your models.py, I think there should be a 1/beta term. You have

           + (coeff * torch.abs(error / w_st) * (2.0 * nu + alpha))

but the original paper defines in equation 12 this part to be $\rho_q (y_i − \gamma_i) · \phi$ where $\phi_i = 2 \nu_i + \alpha_i + 1/\beta_i$

Additionally, they define the width of the distribution through sampling an exponential as such

    tfd = tfp.distributions
    theta = (1.0-2.0*quantile)/(quantile*(1.0-quantile))
    mean_ = beta/(alpha-1)

    w_i_dis = tfd.Exponential(rate=1/mean_)

However, it appears you guys define it via

 w_st = torch.sqrt(beta * (1 + nu) / (alpha * nu))

I'm a little new to this area, so it's very possible I'm missing something. I'm also happy to submit a PR and contribute to the development of this package.

All the best,
Eddie

[EdwardBerman]

I think I've figured out the source of my confusion at least to some extent, I was looking at the regression losses and not the gaussian losses in the original code . This may be the cause of some but not all of the confusion. Still, some follow up would be appreciated. I'll be in person for ML4PS presenting some work, so maybe I'll get a chance to learn more about this then 😄