You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Matching silently substitutes the training mean on failure, and that score is fed to Optuna #210
Matching's hyperparameter tuning catches every exception and substitutes the training mean, with no log, no warning and no counter. That score is then handed to Optuna, so a parameter set under which matching fails on every record is scored as if it were a mean-predictor — and can be selected as best. autoimpute can then report Matching as the winning method on a score matching never produced.
The code
# microimpute/models/matching.py:642-646exceptException:
# If chunk fails, use mean of training data as predictionmean_val=X_train_fold[var].mean()
y_pred=np.full(len(X_val_var), mean_val)
and again at matching.py:663-667. Neither handler calls self.logger. y_pred flows straight into compute_loss and thence into the Optuna objective.
A mean-predictor is not a neutral score — on a low-signal target it can beat a genuine matching fit on quantile loss, so the failure is not merely hidden but actively rewarded.
Related, in the predict path
matching.py:228-236 fills a failed chunk with NaN. That one does log (self.logger.warning("Chunk N failed: ... Filling with NaN values.")), but _process_matching_results returns a frame with silent NaN blocks and no summary of how many records were affected, so a caller has no way to know what fraction of the output is missing without checking themselves.
Suggested fix
In the tuning path, let the exception propagate so Optuna prunes the trial. A failed trial should be pruned, not scored.
If a fallback must stay, log at warning with the exception, count the affected records, and attach the count to the result.
In the predict path, attach n_failed_records to the returned result so the caller can act on it.
Found during a pre-JOSS-submission audit (#201), confirmed verbatim by an independent reviewer.
Activity
Prune failed Matching trials instead of scoring the training mean
Fix tests that pass for the wrong reason, and export ZeroInflatedImputer
Prune failed Matching trials instead of scoring the training mean
Fix tests that pass for the wrong reason, and export ZeroInflatedImpu…