Skip to content

Is it possible to integrate a metric of imblearn as a scorer? #786

Open
@moelliDo

Description

@moelliDo

I would like to use the geometric mean as a metric to optimize classification models trained with the fit() method.
I thought that the autosklearn.metrics.make_scorer() would allow to define the geometric mean as a scorer like:
classifier.fit(X_train, y_train, feat_type=feat_type, metric=autosklearn.metrics.make_scorer("gm", imblearn.metrics.geometric_mean_score))
as the imblearn-package is fully compatible with sklearn.

However, after the model has been fitted, the sprint statistics indicate, that the definition as I did it does not seem to be working:

auto-sklearn results:
Dataset name: 6b31930a65e59cca700a5844fbab91a0
Metric: gm
Best validation score: 0.000000
Number of target algorithm runs: 187
Number of successful target algorithm runs: 82
Number of crashed target algorithm runs: 74
Number of target algorithms that exceeded the time limit: 18
Number of target algorithms that exceeded the memory limit: 13

Is it somehow possible to define the geometric mean as a metric to optimize the model?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions