Interest in adding online Entropy estimation?

[robertfeldt]

Thanks for this great package.

I plan to implement some online Entropy estimation algorithms. Was planning to do as separate package but then realized it might fit in this one. Would this be of interest or too niche?

One technical concern might be that the calculated statistics might not be immediately available since the estimates need a certain number of data samples (in the stream) before they can be meaningfully calculated. I guess there might already be ways of handling this in the package though?!

[joshday]

Thanks!

Your stuff could either go here or in a new package that depends on OnlineStatsBase (if you want to use the OnlineStats interface). I'm happy to have it live in OnlineStats though, under the conditions that 1) there aren't new dependencies, and 2) you have tests for your stuff. There's pros and cons of contributing vs. writing your own package so you just choose whatever is best for you.

One technical concern might be that the calculated statistics might not be immediately available since the estimates need a certain number of data samples (in the stream) before they can be meaningfully calculated

The OnlineStats interface helps with this:

• fit! is for updating the "sufficient statistics" and doesn't necessarily update the estimate.

• value calculates the estimate from the "sufficient statistics". Your value function could spit out error bounds/warning about low samples until you've seen a sufficient amount.