SFFS takes longer with larger training samples

[arilwan]

I run an SFFS selector and it typically takes over 2-weeks to select "best" subset with an initial 80-features, 67k samples, and random forest classifier.

The same number of samples with 240-features took over a month (and still running Features: 192/240 ). May likely takes 2-month to select best subset.

Is it possible to use ensemble of base-classifiers to fit the SFFS selector (reducing the number of records per classifier) to speed up selection process?

Something like:

base_model = RandomForestClassifier()

n_estimators = 10
clf = BaggingClassifier(base_model, max_samples=1.0 / n_estimators, n_estimators=n_estimators)

sffs = SFS(clf, k_features='best', forward=True, floating=True, n_jobs=-1, socring='roc_auc_ovr_weighted, cv=5)
sffs.fit(xtrain, ytrain)

[arilwan]

I see.

This simply adds time complexity to selection process (although improves metric scoring -this is expected).

SFFS with one RF classifier:

[Parallel(n_jobs=-1)]: Done  80 out of  80 | elapsed:  9.7min finished

[2023-07-04 11:16:33] Features: 1/80 -- score: 0.8006262493234073

SFFS with ensemble of 10 random forest classifiers:

[Parallel(n_jobs=-1)]: Done  80 out of  80 | elapsed: 24.8min finished

[2023-07-04 12:04:06] Features: 1/80 -- score: 0.8395997756479818

[rasbt]

Wow, weeks ... that's a pretty long runtime. I wonder if you could use ExtraTreesClassifier to speed things up. Or maybe a different RF alternative, maybe cuML?