Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CLN] Dispatch (some) Frame ops to Series, avoiding _data.eval #22019

Merged
merged 37 commits into from
Oct 3, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
5195dad
avoid casting to object dtype in mixed-type frames
jbrockmendel Jul 22, 2018
1a66906
Dispatch to Series ops in _combine_match_columns
jbrockmendel Jul 22, 2018
a45abec
comment
jbrockmendel Jul 22, 2018
8594c48
docstring
jbrockmendel Jul 22, 2018
108550e
flake8 fixup
jbrockmendel Jul 22, 2018
07fb477
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Jul 24, 2018
946e54a
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Jul 24, 2018
323f45e
dont bother with try_cast_result
jbrockmendel Jul 25, 2018
b3cedf1
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Jul 25, 2018
a0708d1
revert non-central change
jbrockmendel Jul 25, 2018
5d3db89
simplify
jbrockmendel Jul 25, 2018
aa41de3
revert try_cast_results
jbrockmendel Jul 25, 2018
01c3720
revert non-central changes
jbrockmendel Jul 25, 2018
bcb6735
Fixup typo syntaxerror
jbrockmendel Jul 25, 2018
b3ef417
simplify assertion
jbrockmendel Jul 25, 2018
4b2e21a
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Jul 26, 2018
330a94a
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Jul 27, 2018
4c4f626
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Aug 10, 2018
757e2ae
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Sep 8, 2018
ff96c0d
use dispatch_to_series in combine_match_columns
jbrockmendel Sep 8, 2018
17f33b6
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Sep 12, 2018
c30f898
Pass unwrapped op where appropriate
jbrockmendel Sep 12, 2018
82a7928
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Sep 12, 2018
52b7102
catch correct error
jbrockmendel Sep 12, 2018
453ae8e
whatsnew note
jbrockmendel Sep 12, 2018
93887cf
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Sep 13, 2018
dd115c8
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Sep 18, 2018
265ec78
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Sep 18, 2018
db5ca89
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Sep 27, 2018
f574c24
comment
jbrockmendel Sep 27, 2018
e6821a2
whatsnew section
jbrockmendel Sep 27, 2018
da2e32c
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Sep 28, 2018
a6a7f58
remove unnecessary tester
jbrockmendel Sep 29, 2018
b21a8a2
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Sep 29, 2018
858165f
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Oct 2, 2018
31d4089
Merge branch 'master' of https://github.com/pandas-dev/pandas into mops2
jbrockmendel Oct 2, 2018
5832c2b
doc fixup
jbrockmendel Oct 2, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
revert try_cast_results
  • Loading branch information
jbrockmendel committed Jul 25, 2018
commit aa41de3d80f24d1f9ae6a45f2910e1572888de2f
1 change: 0 additions & 1 deletion pandas/core/internals/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# -*- coding: utf-8 -*-
from .blocks import ( # noqa:F401
try_cast_result, # frame
_block2d_to_blocknd, _factor_indexer, _block_shape, # io.pytables
_safe_reshape, # io.packers
make_block, # io.pytables, io.packers
Expand Down
78 changes: 28 additions & 50 deletions pandas/core/internals/blocks.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,55 +74,6 @@
from pandas.io.formats.printing import pprint_thing


def try_cast_result(left, result, dtype=None):
"""
Try to cast the result to the original dtype for `left`; we may have
roundtripped thru object in the mean-time.

Parameters
----------
left : array-like
result : array-like
dtype : np.dtype, pd.dtype, or None (default None)

Returns
-------
maybe_casted : same type as `result`
"""
if dtype is None:
dtype = left.dtype

if (is_integer_dtype(left) or is_bool_dtype(left) or
is_datetime64_dtype(left) or is_datetime64tz_dtype(left)):
pass
elif is_object_dtype(left) and lib.infer_dtype(left) == 'boolean':
# disguised is_bool_dtype
pass
elif is_float_dtype(left) and result.dtype == left.dtype:

# protect against a bool/object showing up here
if isinstance(dtype, compat.string_types) and dtype == 'infer':
return result
if not isinstance(dtype, type):
dtype = dtype.type
if issubclass(dtype, (np.bool_, np.object_)):
if issubclass(dtype, np.bool_):
if isna(result).all():
return result.astype(np.bool_)
else:
result = result.astype(np.object_)
result[result == 1] = True
result[result == 0] = False
return result
else:
return result.astype(np.object_)

return result

# may need to change the dtype here
return maybe_downcast_to_dtype(result, dtype)


class Block(PandasObject):
"""
Canonical n-dimensional unit of homogeneous dtype contained in a pandas
Expand Down Expand Up @@ -748,7 +699,34 @@ def _try_cast_result(self, result, dtype=None):
""" try to cast the result to our original type, we may have
roundtripped thru object in the mean-time
"""
return try_cast_result(self, result, dtype)
if dtype is None:
dtype = self.dtype

if self.is_integer or self.is_bool or self.is_datetime:
pass
elif self.is_float and result.dtype == self.dtype:

# protect against a bool/object showing up here
if isinstance(dtype, compat.string_types) and dtype == 'infer':
return result
if not isinstance(dtype, type):
dtype = dtype.type
if issubclass(dtype, (np.bool_, np.object_)):
if issubclass(dtype, np.bool_):
if isna(result).all():
return result.astype(np.bool_)
else:
result = result.astype(np.object_)
result[result == 1] = True
result[result == 0] = False
return result
else:
return result.astype(np.object_)

return result

# may need to change the dtype here
return maybe_downcast_to_dtype(result, dtype)

def _try_coerce_args(self, values, other):
""" provide coercion to our input arguments """
Expand Down