[ENH] Tag for capability to making probabilistic predictions for classification

[TonyBagnall]

Describe the feature or idea you want to propose

as requested here
#1587
it would be nice to have a way of testing if classifiers (and maybe clusterers) can return meaningful probability estimates rather than default 0/1/

Describe your proposed solution

add a tag, maybe capability:predict_probas?

Describe alternatives you've considered, if relevant

No response

Additional context

No response

[SebastianSchmidl]

Should predict_proba actually return 0/1 if it is not supported?
To make sure that the user notices that the classifier does not compute probabilities, we should at least raise a warning - I would even raise an error.

---

I'm in favor of introducing a new tag for the tests, anyway. I would name it like the method capability:predict_proba

[hadifawaz1999]

Should predict_proba actually return 0/1 if it is not supported? To make sure that the user notices that the classifier does not compute probabilities, we should at least raise a warning - I would even raise an error.

I'm in favor of introducing a new tag for the tests, anyway. I would name it like the method capability:predict_proba

i think its cases such as 1-NN no @TonyBagnall ?

[TonyBagnall]

generally agreed this is a good idea, as long as its tested, but some confusion about how to handle polymorphic classifiers in pipelines

[MatthewMiddlehurst]

Also support a tag with some testing attached if feasible. I think predict_proba is way too integrated into framework throughout the scikit-learn-like ecosystem to make raising an error worth it. As its part of the base class framework is exists for all (relevant) estimators, which is probably what people are checking for. 0/1 are valid probabilities, they are just very bad. I think anyone evaluating their model will be able to pick up on that pretty quickly if its relevant.