Open
Description
Describe the feature or idea you want to propose
From the code comments:
# once the best parameter option has been estimated on the
# training data, perform a final pass with this parameter option
# to get the individual predictions with cross_cal_predict (
# Note: optimisation potentially possible here if a GridSearchCV
# was used previously. TO-DO: determine how to extract
# predictions for the best param option from GridSearchCV)
else:
best_model = KNeighborsTimeSeriesClassifier(
n_neighbors=1,
distance=this_measure,
distance_params=grid.best_params_["distance_params"],
n_jobs=self._n_jobs,
)
preds = cross_val_predict(
best_model, full_train_to_use, y, cv=LeaveOneOut()
)
acc = accuracy_score(y, preds)
Describe your proposed solution
Explore if the grid search and/or random parameter search can be optimised to avoid doing an extra cross-validation when weighting.
Describe alternatives you've considered, if relevant
No response
Additional context
This can be done in a few ways possible, dont know what is correct:
- Redo CV if
self.proportion_train_in_param_finding
is less than 1 - Use the subsamples train set to generate accuracy estimate and always avoid an extra CV