-
-
Notifications
You must be signed in to change notification settings - Fork 18.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SparseArray is an ExtensionArray #22325
Merged
Merged
Changes from 1 commit
Commits
Show all changes
236 commits
Select commit
Hold shift + click to select a range
ee187eb
wip
TomAugspurger 32c1372
from scratch
TomAugspurger b265659
Updates
TomAugspurger 8dfc898
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 9c57725
WIP
TomAugspurger 13952ab
wip
TomAugspurger 7a6e7fa
wip take
TomAugspurger 1016af1
wip take
TomAugspurger 072abec
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 0ad61cc
take
TomAugspurger 5b0b524
take working
TomAugspurger 224744a
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 620b5fb
remove registry
TomAugspurger 164c401
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 65f83d6
missing
TomAugspurger 0b3c682
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 69a5d13
wip ops
TomAugspurger f2b5862
More ops wip
TomAugspurger fa80fc5
segfault!
TomAugspurger 3f20890
wip
TomAugspurger 484adb0
start docs
TomAugspurger 1df1190
2 failing extension tests
TomAugspurger 4246ac4
wip fillna
TomAugspurger a849699
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger c4da319
registry dtype, asarray
TomAugspurger a2f158f
astype interface
TomAugspurger 26b671a
"passing" extension tests
TomAugspurger 375e160
no sparse block
TomAugspurger 0a37050
wip
TomAugspurger 3c2cb0f
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 27c6378
wip
TomAugspurger e52dae9
a bit on concat
TomAugspurger b6d8430
revert concat changes
TomAugspurger 640c4a5
passing again
TomAugspurger 6b61597
More concat
TomAugspurger 427234f
fillna...
TomAugspurger e055629
wip
TomAugspurger a79359c
wip
TomAugspurger de3aa71
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 21f4ee3
reductions, ufuncs
TomAugspurger c1e594a
failing on ufuncs
TomAugspurger dc7f93f
wipo
TomAugspurger eb09d21
concat is broken
TomAugspurger 7dcf4b2
formatting failing
TomAugspurger b39658a
more wip
TomAugspurger a8b76bd
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger e041313
Extension test fixups
TomAugspurger 595535e
some indexing, sparse string
TomAugspurger 7700299
passing indexing
TomAugspurger f1ff7da
passing pivot
TomAugspurger 33fa6f7
broken broken broken
TomAugspurger 40c035e
sanitize
TomAugspurger 1d49cc7
broken broken broken
TomAugspurger 6f4b6b6
wip
TomAugspurger 6f037b5
working through series
TomAugspurger 7da220e
working through series
TomAugspurger bfbe4ab
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger c5666b6
series passing
TomAugspurger ff6037c
more tests
TomAugspurger 5c362ef
wip
TomAugspurger 55cac36
wip
TomAugspurger c4e8784
More test
TomAugspurger a00f987
skip internals tests
TomAugspurger a6d7eac
linting
TomAugspurger 4b4f9bd
cleanup
TomAugspurger 82801be
cleanup
TomAugspurger 1a149dc
cleanup
TomAugspurger fde19d7
remove debug code
TomAugspurger a7ba8f6
API: dispatch to EA.astype
TomAugspurger 5064217
API: ExtensionDtype._is_numeric
TomAugspurger e31e8aa
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 79c8e9c
update type
TomAugspurger 26993fe
Merge remote-tracking branch 'upstream/master' into ea-astype-dispatch
TomAugspurger 6eeec11
py2 compat
TomAugspurger 50de326
fixed test
TomAugspurger 5ef1747
test fill value
TomAugspurger f31970c
Test nbytes
TomAugspurger f1b860f
explainers
TomAugspurger 5c44275
linting
TomAugspurger 33bc8f8
Allow concatenating with different sparse dtypes
TomAugspurger 9bf13ad
Linting
TomAugspurger de1fb5b
lint
TomAugspurger da580cd
Wip
TomAugspurger 88b73c3
Merge branch 'ea-astype-dispatch' into ea-sparse-2
TomAugspurger afde64d
Merge branch 'ea-is-numeric' into ea-sparse-2
TomAugspurger e603d3d
fixup 33bc8f836
TomAugspurger ec5eb9a
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger a72ee1a
Fixed DataFrame.__setitem__ for updating to sparse.
TomAugspurger f147635
try removing
TomAugspurger c35c7c2
Merge branch 'ea-astype-dispatch' into ea-sparse-2
TomAugspurger e159ef2
wip
TomAugspurger d48a8fa
Fixup
TomAugspurger 3bcf57e
astype works
TomAugspurger 31d401f
Squashed commit of the following:
TomAugspurger a4369c2
Squashed commit of the following:
TomAugspurger 608b499
Fixed Series[sparse].to_sparse
TomAugspurger 14e60c9
Shift works
TomAugspurger 550f163
parametrize shift test
TomAugspurger 821cc91
Removed bogus test
TomAugspurger e21ed21
Un-xfail more
TomAugspurger aeb8c8c
scalar take raises
TomAugspurger 34c90ed
Move fill_value to dtyep
TomAugspurger 2103959
Move fill_value to dtyep
TomAugspurger 26af959
Merge branch 'ea-sparse-dtype-fill-value' into ea-sparse-2
TomAugspurger e5920c2
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 084a967
cleanup
TomAugspurger bb17760
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger dde7852
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger f1b4e6b
Setting fill value (but that's bad)
TomAugspurger 6a31077
Explicit fill value
TomAugspurger 02aa7f7
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 3a7ee2d
Fixed merge conflicts
TomAugspurger d6fe191
subdtype -> subtype
TomAugspurger b1ea874
subdtype -> subtype
TomAugspurger 2213b83
Fixed pickle
TomAugspurger 94664c4
test dtype
TomAugspurger e54160c
astype update
TomAugspurger 04a2dbb
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger fb01d1a
more
TomAugspurger f78ae81
lint
TomAugspurger 11d5b40
py2 compat
TomAugspurger ba70753
dtype tests
TomAugspurger 82bab3c
explainer
TomAugspurger 2990124
Delete things
TomAugspurger a9d0f17
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 0c52c37
NumPy 1.9 compat
TomAugspurger 998f113
implement divmod
TomAugspurger 38b0356
Fix broken fill value setting
TomAugspurger 7206d94
compare with lists
TomAugspurger fe771b5
clean
TomAugspurger 12e424c
fixed index ctor fail
TomAugspurger 3bd567f
New xfail
TomAugspurger f816346
Handle sparse reindex
TomAugspurger 1a1dcf4
concat mixed
TomAugspurger e3d9173
take note
TomAugspurger 2715cdb
Remove test.
TomAugspurger 4e40599
concat NA and empty
TomAugspurger 0aa3934
dum
TomAugspurger a3becb6
Fix lost fill value
TomAugspurger 5660b9a
override
TomAugspurger dd3cba5
Handle fill in unique
TomAugspurger cc65b8a
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 06dce5f
Faster isna
TomAugspurger f7351d3
Support old numpy
TomAugspurger 2055494
clean
TomAugspurger f310322
Simplified setter
TomAugspurger 0008164
Inplace not supported.
TomAugspurger 027f6d8
compat
TomAugspurger c0d9875
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 44b218c
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 47fa73a
32-bit compat
TomAugspurger c2c489f
Lint
TomAugspurger 3729927
Test fixups
TomAugspurger 9ba49e1
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 543ac7c
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger f66ef6f
CI passing
TomAugspurger ba8fc9d
Right numpy version
TomAugspurger 9185e33
linting
TomAugspurger 11799ab
Try intp
TomAugspurger 73e7626
32-bit compat
TomAugspurger ebece16
Doc cleanup
TomAugspurger 7db6990
Simplify is_sparse
TomAugspurger be21f42
Updated factorize
TomAugspurger e857363
Use ABC
TomAugspurger d0ee038
simplify interleave_dtype
TomAugspurger 54f4417
docstring, simplify
TomAugspurger 2082d86
fixup supers
TomAugspurger f846606
Linting
TomAugspurger ce8e0ac
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 1f6590e
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger b758469
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger f6b0924
move and fix conflict
TomAugspurger 232518c
doc note
TomAugspurger e8b37da
ENH: is_homogenous
TomAugspurger 0197e0c
BUG: Preserve dtype on homogeneous EA xs
TomAugspurger 62326ae
asarray test
TomAugspurger f008c38
Fixed asarray
TomAugspurger 88c6126
Merge remote-tracking branch 'upstream/master' into ea-xs
TomAugspurger 5c8662e
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 78798cf
is_homogeneous -> is_homogeneous_type
TomAugspurger b051424
lint
TomAugspurger 78979b6
Squashed commit of the following:
TomAugspurger 2333db1
Merge followup
TomAugspurger b41d473
Followup from merge
TomAugspurger d6a2479
lint
TomAugspurger a23c27c
Merge remote-tracking branch 'origin/ea-xs' into ea-sparse-2
TomAugspurger 7372eb3
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger cab8c54
handle unary ops
TomAugspurger 52ae275
linting
TomAugspurger 9c9b49e
compat, lint
TomAugspurger f5d7492
SparseSeries unary ops
TomAugspurger b4b4cbc
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger bf98b9d
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger f3d2681
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 7d4d3ba
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 57c03c2
splib
TomAugspurger 0dbc33e
collections -> compat
TomAugspurger c217cf5
updates
TomAugspurger 2ea7a91
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 8f2f228
Set dtype
TomAugspurger c83bed7
reveret
TomAugspurger 53e494e
clarify fillna
TomAugspurger 627b9ce
Remove old invert
TomAugspurger df0293a
some cleanup
TomAugspurger a590418
remove redundant whatsnew
TomAugspurger 7821f19
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger ee26c52
Update hashing, eq
TomAugspurger 40390f1
wip-comments
TomAugspurger 15a164d
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 88432c8
hashing
TomAugspurger 3e7ec90
dtype and datetime64
TomAugspurger 7b0a179
Updates
TomAugspurger 20d8815
index
TomAugspurger 3e81c69
wip
TomAugspurger 1098a7a
quantile test
TomAugspurger 10d204a
merge conflict
TomAugspurger 69075d8
use is_homogenous_type
TomAugspurger 0764baa
use assert_frame_equal
TomAugspurger a4a47c5
merge exp construction
TomAugspurger a5b6c39
API: Allow ExtensionArray.isna to be an EA
TomAugspurger 70d8268
document and test map
TomAugspurger 7aed79f
table formatting
TomAugspurger 11e55aa
fixup! API: Allow ExtensionArray.isna to be an EA
TomAugspurger 11606af
Restore subclass test
TomAugspurger 2f73179
Revert changes to test
TomAugspurger 1b3058a
quote
TomAugspurger f4ec928
fixup! API: Allow ExtensionArray.isna to be an EA
TomAugspurger 8c67ca2
lint
TomAugspurger cc89ec7
COMPAT: NumPy 1.9 bool-like indexing
TomAugspurger 3f713d4
misc. comments
TomAugspurger 886fe03
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
TomAugspurger 75099af
asarray on bool key for numpy compat
TomAugspurger 731fc06
Raise for non-default values
TomAugspurger f91141d
groupby / reduce compat
TomAugspurger 37a4b57
lint
TomAugspurger 4aad8e1
fix docs
jorisvandenbossche File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Merge remote-tracking branch 'upstream/master' into ea-sparse-2
- Loading branch information
commit 886fe0387ed05912a1ff3bfa9c007d02c3f4343e
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
You are viewing a condensed version of this merge commit. You can view the full changes here.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
really? we allow this. I agree this would be ok, but is reasonably tested?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Define reasonable :)
I'm reasonably sure there are places in pandas where we assume we have an ndarray, but may get an ExtensionArray instead.
The common case of
.isna().any()is well tested though, I think.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In any case, if you create the masks in other ways than
.isna(), eg with a comparison likedf['sparse_column'] == 1, you get exactly the same issue. Which means we basically have to support indexing with boolean sparse masks anyway I think.