FocalLoss without logits and as class

[ori-kishony]

🚀 The feature

Making the focal loss api similar to BCE api. In which the difference between BCELoss and BCELossWithLogits is clear and is made as a class as well as a function.

Motivation, pitch

When using a model that explicitly uses the sigmoid activation at the end it is useful to be able to simply use focal loss without injecting the loss before this activation. The use of focal loss in the similar way to BCELoss as a class would also be more elegant and concise.

Alternatives

No response

Additional context

No response

[datumbox]

@ori-kishony Thanks for the proposal. It's a good argument.

As you know, right now we offer a sigmoid_focal_loss method that does an explicit sigmoid call inside the method. One approach could be adding another focal_loss method that doesn't make this call. They should both share code to avoid duplication but not do the same logging calls.

An alternative approach is really mimic the BCE paradigm and rename methods but I'm not sure it's worth deprecating the previous method just to rename it.

Concerning on whether we should even do this, right now all TorchVision's models require the extra sigmoid call and wouldn't make use of the proposed feature. This doesn't mean that the feature isn't useful but we don't have an immediate use-case to cover. @fmassa do you have any thoughts on this?