Unify model cleanup for Engines

[JackTemaki]

This was not tested yet from my side, but I think there should be no significant issues.

[JackTemaki]

Still, I wonder if we even want to have the same logic for both, or if we use this chance to remove the heuristics.

[albertz]

Still, I wonder if we even want to have the same logic for both, or if we use this chance to remove the heuristics.

What heuristics do you mean? I only see a single heuristic there, which is that if min(score) == max(score) for some key, it would not take that key into account when checking for the N-best checkpoints, as it would be totally random anyway. Which is not really so problematic, I think.

Or do you mean the defaults, e.g. for what models to keep by default? So, what defaults do you suggest instead? Or no defaults at all and force the user to configure it?

[JackTemaki]

Still, I wonder if we even want to have the same logic for both, or if we use this chance to remove the heuristics.

What heuristics do you mean? I only see a single heuristic there, which is that if min(score) == max(score) for some key, it would not take that key into account when checking for the N-best checkpoints, as it would be totally random anyway. Which is not really so problematic, I think.
Or do you mean the defaults, e.g. for what models to keep by default? So, what defaults do you suggest instead? Or no defaults at all and force the user to configure it?

Yes that, but this might be not nice to separate for the backends now.

[albertz]

This was implemented independently now via #1360, so this here is obsolete.