Non-standard naming in L-BFGS Learners (LogisticRegression, PoissonRegression)

[rogancarr]

In LogisticRegression and PoissonRegression (which use the same L-BFGS base), we have a parameter IterationsToRemember that refers to the number of gradients to accumulate in the history. While this terminology makes sense, it's not what we usually encounter in the field.

In the literature, we see this referred to as the "history size" (see e.g. wikipedia).

In the various toolkits that expose an L-BFGS solver, they use:
Scikit Learn: Doesn't expose it.
Spark: NumberOfCorrections
TensorFlow: num_correction_pairs
PyTorch: history_size

I would vote for HistorySize, with the docs explaining what it is.

What do you all think? Any big feelings around this?

[rogancarr]

@shauheen @wschin @TomFinley @markusweimer What do you think?

[Ivanidzo4ka]

Just to protect myself from pitchforks, before it was MemorySize. :D

[rogancarr]

Oh, I don't think IterationsToRemember is necessarily bad, just not standard :)

[TomFinley]

Just to protect myself from pitchforks, before it was MemorySize. :D

I don't remember that. 😉

Jokes aside, yes totally sounds fine with me. Memory size is a bad name since that could be misinterpreted. The idea of "corrections" is a bit weird; I mean, I guess if all the toolkits call it that, it's just weird to name it something so different from the literature which I seem to recall just being history.

I would also be fine with leaving it as is. I can't say I consider "correction size" fundamentally easier to remember than "iterations to remember," considering how L-BFGS uses that parameter. If anything that makes it more obscure, but I can't say I care very much.

[rogancarr]

@TomFinley HistorySize or IterationsToRemember?

[shauheen]

HistorySize +1