DropHighPSIFeatures support for missing values?

[PaulZhutovsky]

If I understand it correctly, currently DropHighPSIFeatures only support either dropping entries with missing values completely or "forcing" the user to impute those beforehand.
An (easy) alternative could be to treat missings as their own bin during the calculation of the PSI value. Was there a reason why this wasn't implemented this way? And what would be a preferred/suggested way to deal with features which contain missing values during the calculation of PSI?

[solegalli]

Hi @PaulZhutovsky

Thanks for reaching out.

It is possible to treat missing values as their own bin as you say, but we haven't implemented it yet. The main reason is to try and keep the transformer simple. But it could be a new addition.

For categorical variables, if you impute the category with a specific string, using the CategoricalEncoder for example right before applying the DropHighPSIFeatures, then the resulting effect is the one you explain: nan are treated as their own bin.

For numerical variables, I am not sure it is so straightforward, because the variables are numeric, and while still doable in practice, nan may be better handled in a way that makes sense based on the nature of their missingness before applying the PSI. However, if we do not know how the NAN were introduced, we could as well treat them as their own bin.

I leave this issue open for further discussion. Thanks for raising it.

@gverbock @glevv

[PaulZhutovsky]

Thanks for the reply!

Indeed in the case of categorical variables encoding "missing" as its own category can easily solve my question but I'm more interested in how to apply this for numerical variables.
Currently, DropHighPSIFeatures forces the binning to be done through strategy = "equal_frequency" or "equal_width" meaning that it is not possible to specify your own bins per variable or your own transformer class. That change alone could already help the user to have the flexibility to treat missings the way they want.
(and of course making it possible for the transformers to treat missings as separate bins would be even easier)

[glevv]

Greetings!

I think the intended use is "scikit-learn style" - first impute the NaN values, then use the transformer. I have a feeling that in the end it will be the same as "treat NaNs as separate bin", because it is just imputing them as maximum or minimum value.

[PaulZhutovsky]

@glevv I would say the difference is that in case of first imputing + then binning the NaN values will be (most likely) merged together with "real" data. While if you treat them as a separate bin this would not happen (by definition). In some cases the first approach might make sense (if imputing the NaN values is making sense by itself), in other cases it will not make sense. That's why I would like to have this option available.