Move nested CV into a separate file #4
Conversation
There was a problem hiding this comment.
Question 1: I've exported the nested CV to a separate file, but I'm not entirely satisfied with the current structure. The process involves first manipulating the data in the notebook, saving it as a pickle, then running a separate script, and finally bringing the results back to this notebook. Do you think it would be better to have the data processing as a separate script?
Since this is a tutorial, I think it is important to walk through the pre-processing steps in the notebook itself. It is kind of inconvenient for us as the makers of the tutorial, but I don't think there is a good workaround in this situation.
Question 2: Additionally, I've been struggling to calculate memory usage, so I could not do it.
No worries, thanks for looking into it.
Question 3: About the timing, right now, I'm tracking the time for each model within the outer loop. So, if we have, let's say, 100 iterations, we end up with separate entries in the results dictionary for each model and dataset (including taxonomic levels). However, I'm wondering if it would be more appropriate to have a cumulative time for each model type instead. For example, all the Random Forests together. What do you think?
Let's keep them as is. We can always add them up to find the cumulative time for each model type, should we want to calculate that.
Minor change request: I just noticed an oversight of mine from earlier on - please remove BMI from the feature set we use to train the model (since BMI is calculated from weight and height, which are both in there).
Aside from that, looks good.
|
I removed "BMI" from the features, as suggested. 🚀 |
I've exported the nested CV to a separate file, but I'm not entirely satisfied with the current structure. The process involves first manipulating the data in the notebook, saving it as a pickle, then running a separate script, and finally bringing the results back to this notebook. Do you think it would be better to have the data processing as a separate script?
Additionally, I've been struggling to calculate memory usage, so I could not do it. About the timing, right now, I'm tracking the time for each model within the outer loop. So, if we have, let's say, 100 iterations, we end up with separate entries in the results dictionary for each model and dataset (including taxonomic levels). However, I'm wondering if it would be more appropriate to have a cumulative time for each model type instead. For example, all the Random Forests together. What do you think?