SFS and GridSearchCV

[andregiuseppecavalli]

Goodmorning prof. I am relatively new to ML and I'm trying to combine in a sklearn pipeline the sequential feature selection function of mlxtend and GridSearchCV. I followed the example code found in the mlxtend documentation, but I don't really understand how that works. In my mind, maybe I'm completely wrong, by combining SFS and gridsearchcv, the SFS alg. should select the best features for each combination of hyperparameters used in the grid search.

set of hyperparams --> SFS --> CV --> new set of hyperparams --> SFS --> CV .... And so on...

So at the end we should obtain a model trained with the best found features and the best hyperparameters.
So why I have to fit first the gridsearchcv and then the pipeline, if the pipeline is used as estimator in the grid search? Because if I fit the pipeline it will do again SFS using as estimator the best parameters that I passed, but SFS should have already been done during the gridsearch cv. In this way it seems more like:

GridSearchCV --> best params --> SFS with best params.

Can you explain me more in detail how he flow of operations works? I hope I was clear enough.

Thank you very much for your time and help,

Andrea Cavalli

[rasbt]

That's a good question. It's been a few years since I worked on the underlying code, but I remember there being a restriction with respect to the hyperparameters changes being propagated -- I think due to the cloning behavior in Grid Search. I think it's related to the discussion here: #813