Merge remote-tracking branch 'upstream/master' into jbrockmendel-less24024b

Author: TomAugspurger

.travis.yml

@@ -36,14 +36,6 @@ matrix:
       env:
         - JOB="3.7" ENV_FILE="ci/deps/travis-37.yaml" PATTERN="not slow and not network"
 
-    - dist: trusty
-      env:
-        - JOB="2.7, locale, slow, old NumPy" ENV_FILE="ci/deps/travis-27-locale.yaml" LOCALE_OVERRIDE="zh_CN.UTF-8" PATTERN="slow"
-      addons:
-        apt:
-          packages:
-          - language-pack-zh-hans
-
     - dist: trusty
       env:
         - JOB="2.7" ENV_FILE="ci/deps/travis-27.yaml" PATTERN="not slow"
@@ -60,14 +52,6 @@ matrix:
       env:
         - JOB="3.6, coverage" ENV_FILE="ci/deps/travis-36.yaml" PATTERN="not slow and not network" PANDAS_TESTING_MODE="deprecate" COVERAGE=true
 
-    - dist: trusty
-      env:
-        - JOB="3.7, NumPy dev" ENV_FILE="ci/deps/travis-37-numpydev.yaml" PATTERN="not slow and not network" TEST_ARGS="-W error" PANDAS_TESTING_MODE="deprecate"
-      addons:
-        apt:
-          packages:
-          - xsel
-
     # In allow_failures
     - dist: trusty
       env:

asv_bench/benchmarks/period.py

@@ -1,5 +1,5 @@
-from pandas import (DataFrame, Series, Period, PeriodIndex, date_range,
-                    period_range)
+from pandas import (
+    DataFrame, Period, PeriodIndex, Series, date_range, period_range)
 
 
 class PeriodProperties(object):
@@ -94,7 +94,7 @@ def time_value_counts(self, typ):
 class Indexing(object):
 
     def setup(self):
-        self.index = PeriodIndex(start='1985', periods=1000, freq='D')
+        self.index = period_range(start='1985', periods=1000, freq='D')
         self.series = Series(range(1000), index=self.index)
         self.period = self.index[500]
 

ci/azure/posix.yml

@@ -20,21 +20,36 @@ jobs:
           CONDA_PY: "27"
           PATTERN: "not slow and not network"
 
-        py37_locale:
-          ENV_FILE: ci/deps/azure-37-locale.yaml
-          CONDA_PY: "37"
-          PATTERN: "not slow and not network"
+        py27_locale_slow_old_np:
+          ENV_FILE: ci/deps/azure-27-locale.yaml
+          CONDA_PY: "27"
+          PATTERN: "slow"
           LOCALE_OVERRIDE: "zh_CN.UTF-8"
+          EXTRA_APT: "language-pack-zh-hans"
 
         py36_locale_slow:
           ENV_FILE: ci/deps/azure-36-locale_slow.yaml
           CONDA_PY: "36"
           PATTERN: "not slow and not network"
           LOCALE_OVERRIDE: "it_IT.UTF-8"
 
+        py37_locale:
+          ENV_FILE: ci/deps/azure-37-locale.yaml
+          CONDA_PY: "37"
+          PATTERN: "not slow and not network"
+          LOCALE_OVERRIDE: "zh_CN.UTF-8"
+
+        py37_np_dev:
+          ENV_FILE: ci/deps/azure-37-numpydev.yaml
+          CONDA_PY: "37"
+          PATTERN: "not slow and not network"
+          TEST_ARGS: "-W error"
+          PANDAS_TESTING_MODE: "deprecate"
+          EXTRA_APT: "xsel"
+
   steps:
     - script: |
-        if [ "$(uname)" == "Linux" ]; then sudo apt-get install -y libc6-dev-i386; fi
+        if [ "$(uname)" == "Linux" ]; then sudo apt-get install -y libc6-dev-i386 $EXTRA_APT; fi
         echo "Installing Miniconda"
         ci/incremental/install_miniconda.sh
         export PATH=$HOME/miniconda3/bin:$PATH

ci/deps/travis-27-locale.yaml renamed to ci/deps/azure-27-locale.yaml

@@ -3997,6 +3997,7 @@ objects.
    api.extensions.register_index_accessor
    api.extensions.ExtensionDtype
    api.extensions.ExtensionArray
+   arrays.PandasArray
 
 .. This is to prevent warnings in the doc build. We don't want to encourage
 .. these methods.

ci/deps/travis-37-numpydev.yaml renamed to ci/deps/azure-37-numpydev.yaml

@@ -71,8 +71,10 @@ the **array** property
    s.array
    s.index.array
 
-Depending on the data type (see :ref:`basics.dtypes`), :attr:`~Series.array`
-be either a NumPy array or an :ref:`ExtensionArray <extending.extension-type>`.
+:attr:`~Series.array` will always be an :class:`~pandas.api.extensions.ExtensionArray`.
+The exact details of what an ``ExtensionArray`` is and why pandas uses them is a bit
+beyond the scope of this introduction. See :ref:`basics.dtypes` for more.
+
 If you know you need a NumPy array, use :meth:`~Series.to_numpy`
 or :meth:`numpy.asarray`.
 
@@ -81,10 +83,30 @@ or :meth:`numpy.asarray`.
    s.to_numpy()
    np.asarray(s)
 
-For Series and Indexes backed by NumPy arrays (like we have here), this will
-be the same as :attr:`~Series.array`. When the Series or Index is backed by
-a :class:`~pandas.api.extension.ExtensionArray`, :meth:`~Series.to_numpy`
-may involve copying data and coercing values.
+When the Series or Index is backed by
+an :class:`~pandas.api.extension.ExtensionArray`, :meth:`~Series.to_numpy`
+may involve copying data and coercing values. See :ref:`basics.dtypes` for more.
+
+:meth:`~Series.to_numpy` gives some control over the ``dtype`` of the
+resulting :class:`ndarray`. For example, consider datetimes with timezones.
+NumPy doesn't have a dtype to represent timezone-aware datetimes, so there
+are two possibly useful representations:
+
+1. An object-dtype :class:`ndarray` with :class:`Timestamp` objects, each
+   with the correct ``tz``
+2. A ``datetime64[ns]`` -dtype :class:`ndarray`, where the values have
+   been converted to UTC and the timezone discarded
+
+Timezones may be preserved with ``dtype=object``
+
+.. ipython:: python
+
+   ser = pd.Series(pd.date_range('2000', periods=2, tz="CET"))
+   ser.to_numpy(dtype=object)
+
+Or thrown away with ``dtype='datetime64[ns]'``
+
+   ser.to_numpy(dtype="datetime64[ns]")
 
 :meth:`~Series.to_numpy` gives some control over the ``dtype`` of the
 resulting :class:`ndarray`. For example, consider datetimes with timezones.
@@ -109,7 +131,7 @@ Or thrown away with ``dtype='datetime64[ns]'``
 
 Getting the "raw data" inside a :class:`DataFrame` is possibly a bit more
 complex. When your ``DataFrame`` only has a single data type for all the
-columns, :attr:`DataFrame.to_numpy` will return the underlying data:
+columns, :meth:`DataFrame.to_numpy` will return the underlying data:
 
 .. ipython:: python
 
@@ -136,8 +158,9 @@ drawbacks:
 
 1. When your Series contains an :ref:`extension type <extending.extension-type>`, it's
    unclear whether :attr:`Series.values` returns a NumPy array or the extension array.
-   :attr:`Series.array` will always return the actual array backing the Series,
-   while :meth:`Series.to_numpy` will always return a NumPy array.
+   :attr:`Series.array` will always return an ``ExtensionArray``, and will never
+   copy data. :meth:`Series.to_numpy` will always return a NumPy array,
+   potentially at the cost of copying / coercing values.
 2. When your DataFrame contains a mixture of data types, :attr:`DataFrame.values` may
    involve copying data and coercing values to a common dtype, a relatively expensive
    operation. :meth:`DataFrame.to_numpy`, being a method, makes it clearer that the

doc/source/api.rst

@@ -146,11 +146,15 @@ If you need the actual array backing a ``Series``, use :attr:`Series.array`.
 
    s.array
 
-Again, this is often a NumPy array, but may instead be a
-:class:`~pandas.api.extensions.ExtensionArray`. See :ref:`basics.dtypes` for more.
 Accessing the array can be useful when you need to do some operation without the
 index (to disable :ref:`automatic alignment <dsintro.alignment>`, for example).
 
+:attr:`Series.array` will always be an :class:`~pandas.api.extensions.ExtensionArray`.
+Briefly, an ExtensionArray is a thin wrapper around one or more *concrete* arrays like a
+:class:`numpy.ndarray`. Pandas knows how to take an ``ExtensionArray`` and
+store it in a ``Series`` or a column of a ``DataFrame``.
+See :ref:`basics.dtypes` for more.
+
 While Series is ndarray-like, if you need an *actual* ndarray, then use
 :meth:`Series.to_numpy`.
 

doc/source/basics.rst

@@ -654,7 +654,7 @@ Previous Behavior:
 
 .. code-block:: ipython
 
-   In [1]: pi = pd.PeriodIndex(start='2000-01-01', freq='D', periods=10)
+   In [1]: pi = pd.period_range(start='2000-01-01', freq='D', periods=10)
 
    In [2]: s = pd.Series(np.arange(10), index=pi)
 
@@ -674,7 +674,7 @@ New Behavior:
 
 .. ipython:: python
 
-   pi = pd.PeriodIndex(start='2000-01-01', freq='D', periods=10)
+   pi = pd.period_range(start='2000-01-01', freq='D', periods=10)
 
    s = pd.Series(np.arange(10), index=pi)
 

doc/source/dsintro.rst

@@ -65,8 +65,11 @@ If you need an actual NumPy array, use :meth:`Series.to_numpy` or :meth:`Index.t
    idx.to_numpy()
    pd.Series(idx).to_numpy()
 
-For Series and Indexes backed by normal NumPy arrays, this will be the same thing (and the same
-as ``.values``).
+For Series and Indexes backed by normal NumPy arrays, :attr:`Series.array` will return a
+new :class:`arrays.PandasArray`, which is a thin (no-copy) wrapper around a
+:class:`numpy.ndarray`. :class:`arrays.PandasArray` isn't especially useful on its own,
+but it does provide the same interface as any extension array defined in pandas or by
+a third-party library.
 
 .. ipython:: python
 
@@ -75,7 +78,7 @@ as ``.values``).
    ser.to_numpy()
 
 We haven't removed or deprecated :attr:`Series.values` or :attr:`DataFrame.values`, but we
-recommend and using ``.array`` or ``.to_numpy()`` instead.
+highly recommend and using ``.array`` or ``.to_numpy()`` instead.
 
 See :ref:`Dtypes <basics.dtypes>` and :ref:`Attributes and Underlying Data <basics.attrs>` for more.
 
@@ -1148,7 +1151,7 @@ Deprecations
 - Timezone converting a tz-aware ``datetime.datetime`` or :class:`Timestamp` with :class:`Timestamp` and the ``tz`` argument is now deprecated. Instead, use :meth:`Timestamp.tz_convert` (:issue:`23579`)
 - :func:`pandas.api.types.is_period` is deprecated in favor of `pandas.api.types.is_period_dtype` (:issue:`23917`)
 - :func:`pandas.api.types.is_datetimetz` is deprecated in favor of `pandas.api.types.is_datetime64tz` (:issue:`23917`)
-- Creating a :class:`TimedeltaIndex` or :class:`DatetimeIndex` by passing range arguments `start`, `end`, and `periods` is deprecated in favor of :func:`timedelta_range` and :func:`date_range` (:issue:`23919`)
+- Creating a :class:`TimedeltaIndex`, :class:`DatetimeIndex`, or :class:`PeriodIndex` by passing range arguments `start`, `end`, and `periods` is deprecated in favor of :func:`timedelta_range`, :func:`date_range`, or :func:`period_range` (:issue:`23919`)
 - Passing a string alias like ``'datetime64[ns, UTC]'`` as the ``unit`` parameter to :class:`DatetimeTZDtype` is deprecated. Use :class:`DatetimeTZDtype.construct_from_string` instead (:issue:`23990`).
 - In :meth:`Series.where` with Categorical data, providing an ``other`` that is not present in the categories is deprecated. Convert the categorical to a different dtype or add the ``other`` to the categories first (:issue:`24077`).
 - :meth:`Series.clip_lower`, :meth:`Series.clip_upper`, :meth:`DataFrame.clip_lower` and :meth:`DataFrame.clip_upper` are deprecated and will be removed in a future version. Use ``Series.clip(lower=threshold)``, ``Series.clip(upper=threshold)`` and the equivalent ``DataFrame`` methods (:issue:`24203`)
@@ -1310,6 +1313,7 @@ Datetimelike
 - Bug in :meth:`Series.combine_first` not properly aligning categoricals, so that missing values in ``self`` where not filled by valid values from ``other`` (:issue:`24147`)
 - Bug in :func:`DataFrame.combine` with datetimelike values raising a TypeError (:issue:`23079`)
 - Bug in :func:`date_range` with frequency of ``Day`` or higher where dates sufficiently far in the future could wrap around to the past instead of raising ``OutOfBoundsDatetime`` (:issue:`14187`)
+- Bug in :func:`period_range` ignoring the frequency of ``start`` and ``end`` when those are provided as :class:`Period` objects (:issue:`20535`).
 - Bug in :class:`PeriodIndex` with attribute ``freq.n`` greater than 1 where adding a :class:`DateOffset` object would return incorrect results (:issue:`23215`)
 - Bug in :class:`Series` that interpreted string indices as lists of characters when setting datetimelike values (:issue:`23451`)
 - Bug in :class:`Timestamp` constructor which would drop the frequency of an input :class:`Timestamp` (:issue:`22311`)
@@ -1406,6 +1410,7 @@ Conversion
 ^^^^^^^^^^
 
 - Bug in :meth:`DataFrame.combine_first` in which column types were unexpectedly converted to float (:issue:`20699`)
+- Bug in :meth:`DataFrame.clip` in which column types are not preserved and casted to float (:issue:`24162`)
 
 Strings
 ^^^^^^^