Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import StatsBase into Statistics #2

Draft
wants to merge 349 commits into
base: StatsBase2021
Choose a base branch
from
Draft

Conversation

nalimilan
Copy link
Member

@nalimilan nalimilan commented Sep 28, 2019

Equivalent of JuliaLang/julia#31395. See original discussions there. See #87 for current design discussion.

This PR is against the StatsBase branch, as the idea is to port all features we want from StatsBase to it before we clean up the rest (and possibly purge the history from features we don't want). Only then we'll be able to merge it with a clean history (including StatsBase's) into master. (I created the StatsBase branch using git merge master --allow-unrelated-histories -s ours after fetching the StatsBase history from its repo.)

Progress:

  • common.jl
  • weights.jl: skipped wsum
  • moments.jl: skipped moment
  • scalarstats.jl: skipped mean_and_var, mean_and_std, zscore/zscore!, merged nquantile with quantile
  • robust.jl
  • deviation.jl
  • cov.jl: skipped mean_and_cov; skip scattermat?
  • counts.jl
  • ranking.jl
  • toeplitzsolvers.jl
  • rankcorr.jl
  • signalcorr.jl
  • partialcor.jl
  • empirical.jl
  • hist.jl
  • pairwise.jl
  • reliability.jl
  • misc.jl
  • sampling.jl
  • statmodels.jl: move to StatsAPI
  • transformations.jl

lindahua and others added 30 commits March 29, 2014 09:36
still not clear what to do with floating point in StepRange
It turns out there are many cases that are drastically easier this way.
Tests not passing yet, but getting there.
And add a simple test for hist()
Also split out parseint loops in strings.jl to better separate out exception types
Also some minor reorganization
- Implement sum(fn, A, region)
- Implement Base.sumabs across dimensions
- Use pairwise summation/BLAS for sums across first dimension
Also fix a bounds violation when computing the variance of a vector
with a single element
DilumAluthge and others added 3 commits October 7, 2020 11:03
* relax type definition of middle

this adds support for computing the median of unitful types, see PainterQubits/Unitful.jl#202

* updates docs and add tests for middle on non-reals
BioTurboNick and others added 7 commits December 19, 2020 23:18
stdlib modules need to use a custom rule, otherwise the code with which Julia was built is used.
As in `cor` we get square root I think it is safe to assume that the result should be floating point.

An example of current surprising behavior:
```
julia> cor([im])
true
```
bkamins and others added 3 commits February 19, 2021 09:35
When `sort=false`, we only partially sort the input, so `NaN`/`missing`
is not guaranteed to be in the last position. Also avoid throwing errors for
non-`Number` types, for which `isnan` may not be defined.
There is no reliable way to know only from the array eltype whether entries
support `isnan` or not. Better leave to the compiler to optimize out the
`isa Number` check when possible.
@carstenbauer
Copy link

Great effort guys! I'd love to see the StatsBase functionality in the Statistics stdlib.

What about weighted sampling (https://juliastats.org/StatsBase.jl/stable/sampling/#Sampling-from-Population-1). Will this go into Statistics as well? (It also has semantic overlap with Random I guess.)

@nalimilan nalimilan force-pushed the nl/weightedstats branch 2 times, most recently from 1277ff9 to 71cde77 Compare September 25, 2021 15:38
@nalimilan nalimilan changed the title Import weighted stats and moments from StatsBase to Statistics Import StatsBase into Statistics Sep 25, 2021
@nalimilan nalimilan changed the base branch from StatsBase to StatsBase2021 September 25, 2021 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.