[Question] How to find the corresponding cv_results_ model information by Model_id

[leeprinxin]

[Question] How to find the corresponding cv_results_ model information by Model_id
automl.show_models() can output Ensemble models, and I want to get the CV evaluation scores for each model, but cv_results_ does not have an index for model_id, so how do I combine the information from show_models() and cv_results_?

[aron-bram]

Hi,

Thank you for your question.

Unfortunately, for now I can't think of a solution to combine the results of show_models and cv_results_, but I will look into this and come back with an answer soon. I would first have to take a look into how the model_ids are generated and why they are not tracked by cv_results_.

[eddiebergman]

@aron-barm, just a heads up that what we export in cv_results_ tries to corresponds to what scikit-learn advertise it does:

• https://scikit-learn.org/stable/modules/grid_search.html#analyzing-results-with-the-cv-results-attribute

To insert a model_id, I guess we can break their convention a bit and insert an extra column, inserting it into the column params makes no sense to me and doesn't make things any easier.

Might also make sense to have the function just construct a pandas dataframe upfront and finally convert to a dictionary before returning, easier to validate what's going on but I'll leave that up to you!

[aron-bram]

Hi,

I wanted to solve this issue last week but unfortunately I could only get onto it today. I hope it wasn't too urgent for you.
I will submit a PR today with the proposed chages.
Once it's merged, the model_ids will be tracked by cv_results so you should be able to combine information from show_models and cv_results.