From e5c04dcd08c2b0d38752881577a1ba9501eba2cb Mon Sep 17 00:00:00 2001 From: dcherian Date: Thu, 7 Mar 2019 13:31:40 -0800 Subject: [PATCH] Add Weather and Climate specific page. --- doc/data-structures.rst | 8 -- doc/index.rst | 2 + doc/time-series.rst | 137 ---------------------------------- doc/weather-climate.rst | 159 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 161 insertions(+), 145 deletions(-) create mode 100644 doc/weather-climate.rst diff --git a/doc/data-structures.rst b/doc/data-structures.rst index 1ff415b8812..618ccccff3e 100644 --- a/doc/data-structures.rst +++ b/doc/data-structures.rst @@ -549,14 +549,6 @@ binary operations that act on xarray objects. In the future, we hope to write more helper functions so that you can easily make your functions act like xarray's built-in arithmetic. -CF-compliant coordinate variables -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -`MetPy`_ adds a ``metpy`` accessor that allows accessing coordinates with appropriate CF metadata using generic names ``x``, ``y``, ``vertical`` and ``time``. See `their documentation`_ for more information. - -.. _`MetPy`: https://unidata.github.io/MetPy/dev/index.html -.. _`their documentation`: https://unidata.github.io/MetPy/dev/tutorials/xarray_tutorial.html#coordinates - Indexes ~~~~~~~ diff --git a/doc/index.rst b/doc/index.rst index dbe911011cd..1d3bb110ddb 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -52,6 +52,7 @@ Documentation * :doc:`reshaping` * :doc:`combining` * :doc:`time-series` +* :doc:`weather-climate` * :doc:`pandas` * :doc:`io` * :doc:`dask` @@ -70,6 +71,7 @@ Documentation reshaping combining time-series + weather-climate pandas io dask diff --git a/doc/time-series.rst b/doc/time-series.rst index 3249dad2ec6..53efcd45ba2 100644 --- a/doc/time-series.rst +++ b/doc/time-series.rst @@ -212,140 +212,3 @@ Data that has indices outside of the given ``tolerance`` are set to ``NaN``. For more examples of using grouped operations on a time dimension, see :ref:`toy weather data`. - - -.. _CFTimeIndex: - -Non-standard calendars and dates outside the Timestamp-valid range ------------------------------------------------------------------- - -Through the standalone ``cftime`` library and a custom subclass of -:py:class:`pandas.Index`, xarray supports a subset of the indexing -functionality enabled through the standard :py:class:`pandas.DatetimeIndex` for -dates from non-standard calendars commonly used in climate science or dates -using a standard calendar, but outside the `Timestamp-valid range`_ -(approximately between years 1678 and 2262). - -.. note:: - - As of xarray version 0.11, by default, :py:class:`cftime.datetime` objects - will be used to represent times (either in indexes, as a - :py:class:`~xarray.CFTimeIndex`, or in data arrays with dtype object) if - any of the following are true: - - - The dates are from a non-standard calendar - - Any dates are outside the Timestamp-valid range. - - Otherwise pandas-compatible dates from a standard calendar will be - represented with the ``np.datetime64[ns]`` data type, enabling the use of a - :py:class:`pandas.DatetimeIndex` or arrays with dtype ``np.datetime64[ns]`` - and their full set of associated features. - -For example, you can create a DataArray indexed by a time -coordinate with dates from a no-leap calendar and a -:py:class:`~xarray.CFTimeIndex` will automatically be used: - -.. ipython:: python - - from itertools import product - from cftime import DatetimeNoLeap - dates = [DatetimeNoLeap(year, month, 1) for year, month in - product(range(1, 3), range(1, 13))] - da = xr.DataArray(np.arange(24), coords=[dates], dims=['time'], name='foo') - -xarray also includes a :py:func:`~xarray.cftime_range` function, which enables -creating a :py:class:`~xarray.CFTimeIndex` with regularly-spaced dates. For -instance, we can create the same dates and DataArray we created above using: - -.. ipython:: python - - dates = xr.cftime_range(start='0001', periods=24, freq='MS', calendar='noleap') - da = xr.DataArray(np.arange(24), coords=[dates], dims=['time'], name='foo') - -For data indexed by a :py:class:`~xarray.CFTimeIndex` xarray currently supports: - -- `Partial datetime string indexing`_ using strictly `ISO 8601-format`_ partial - datetime strings: - -.. ipython:: python - - da.sel(time='0001') - da.sel(time=slice('0001-05', '0002-02')) - -- Access of basic datetime components via the ``dt`` accessor (in this case - just "year", "month", "day", "hour", "minute", "second", "microsecond", - "season", "dayofyear", and "dayofweek"): - -.. ipython:: python - - da.time.dt.year - da.time.dt.month - da.time.dt.season - da.time.dt.dayofyear - da.time.dt.dayofweek - -- Group-by operations based on datetime accessor attributes (e.g. by month of - the year): - -.. ipython:: python - - da.groupby('time.month').sum() - -- Interpolation using :py:class:`cftime.datetime` objects: - -.. ipython:: python - - da.interp(time=[DatetimeNoLeap(1, 1, 15), DatetimeNoLeap(1, 2, 15)]) - -- Interpolation using datetime strings: - -.. ipython:: python - - da.interp(time=['0001-01-15', '0001-02-15']) - -- Differentiation: - -.. ipython:: python - - da.differentiate('time') - -- Serialization: - -.. ipython:: python - - da.to_netcdf('example-no-leap.nc') - xr.open_dataset('example-no-leap.nc') - -- And resampling along the time dimension for data indexed by a :py:class:`~xarray.CFTimeIndex`: - -.. ipython:: python - - da.resample(time='81T', closed='right', label='right', base=3).mean() - -.. note:: - - - For some use-cases it may still be useful to convert from - a :py:class:`~xarray.CFTimeIndex` to a :py:class:`pandas.DatetimeIndex`, - despite the difference in calendar types. The recommended way of doing this - is to use the built-in :py:meth:`~xarray.CFTimeIndex.to_datetimeindex` - method: - - .. ipython:: python - :okwarning: - - modern_times = xr.cftime_range('2000', periods=24, freq='MS', calendar='noleap') - da = xr.DataArray(range(24), [('time', modern_times)]) - da - datetimeindex = da.indexes['time'].to_datetimeindex() - da['time'] = datetimeindex - - However in this case one should use caution to only perform operations which - do not depend on differences between dates (e.g. differentiation, - interpolation, or upsampling with resample), as these could introduce subtle - and silent errors due to the difference in calendar types between the dates - encoded in your data and the dates stored in memory. - -.. _Timestamp-valid range: https://pandas.pydata.org/pandas-docs/stable/timeseries.html#timestamp-limitations -.. _ISO 8601-format: https://en.wikipedia.org/wiki/ISO_8601 -.. _partial datetime string indexing: https://pandas.pydata.org/pandas-docs/stable/timeseries.html#partial-string-indexing diff --git a/doc/weather-climate.rst b/doc/weather-climate.rst new file mode 100644 index 00000000000..676705820e6 --- /dev/null +++ b/doc/weather-climate.rst @@ -0,0 +1,159 @@ +.. _weather-climate: + +Weather and climate data +======================== + +.. ipython:: python + :suppress: + + import xarray as xr + +``xarray`` can leverage metadata that follows the `Climate and Forecast (CF) conventions`_ if present. Examples include automatic labelling of plots with descriptive names and units if proper metadata is present (see :ref:`plotting`) and support for non-standard calendars used in climate science through the ``cftime`` module (see :ref:`CFTimeIndex`). There are also a number of geosciences-focused projects that build on xarray (see :ref:`related-projects`). + +.. _Climate and Forecast (CF) conventions: http://cfconventions.org + +.. _metpy_accessor: + +CF-compliant coordinate variables +--------------------------------- + +`MetPy`_ adds a ``metpy`` accessor that allows accessing coordinates with appropriate CF metadata using generic names ``x``, ``y``, ``vertical`` and ``time``. See `their documentation`_ for more information. + +.. _`MetPy`: https://unidata.github.io/MetPy/dev/index.html +.. _`their documentation`: https://unidata.github.io/MetPy/dev/tutorials/xarray_tutorial.html#coordinates + +.. _CFTimeIndex: + +Non-standard calendars and dates outside the Timestamp-valid range +------------------------------------------------------------------ + +Through the standalone ``cftime`` library and a custom subclass of +:py:class:`pandas.Index`, xarray supports a subset of the indexing +functionality enabled through the standard :py:class:`pandas.DatetimeIndex` for +dates from non-standard calendars commonly used in climate science or dates +using a standard calendar, but outside the `Timestamp-valid range`_ +(approximately between years 1678 and 2262). + +.. note:: + + As of xarray version 0.11, by default, :py:class:`cftime.datetime` objects + will be used to represent times (either in indexes, as a + :py:class:`~xarray.CFTimeIndex`, or in data arrays with dtype object) if + any of the following are true: + + - The dates are from a non-standard calendar + - Any dates are outside the Timestamp-valid range. + + Otherwise pandas-compatible dates from a standard calendar will be + represented with the ``np.datetime64[ns]`` data type, enabling the use of a + :py:class:`pandas.DatetimeIndex` or arrays with dtype ``np.datetime64[ns]`` + and their full set of associated features. + +For example, you can create a DataArray indexed by a time +coordinate with dates from a no-leap calendar and a +:py:class:`~xarray.CFTimeIndex` will automatically be used: + +.. ipython:: python + + from itertools import product + from cftime import DatetimeNoLeap + dates = [DatetimeNoLeap(year, month, 1) for year, month in + product(range(1, 3), range(1, 13))] + da = xr.DataArray(np.arange(24), coords=[dates], dims=['time'], name='foo') + +xarray also includes a :py:func:`~xarray.cftime_range` function, which enables +creating a :py:class:`~xarray.CFTimeIndex` with regularly-spaced dates. For +instance, we can create the same dates and DataArray we created above using: + +.. ipython:: python + + dates = xr.cftime_range(start='0001', periods=24, freq='MS', calendar='noleap') + da = xr.DataArray(np.arange(24), coords=[dates], dims=['time'], name='foo') + +For data indexed by a :py:class:`~xarray.CFTimeIndex` xarray currently supports: + +- `Partial datetime string indexing`_ using strictly `ISO 8601-format`_ partial + datetime strings: + +.. ipython:: python + + da.sel(time='0001') + da.sel(time=slice('0001-05', '0002-02')) + +- Access of basic datetime components via the ``dt`` accessor (in this case + just "year", "month", "day", "hour", "minute", "second", "microsecond", + "season", "dayofyear", and "dayofweek"): + +.. ipython:: python + + da.time.dt.year + da.time.dt.month + da.time.dt.season + da.time.dt.dayofyear + da.time.dt.dayofweek + +- Group-by operations based on datetime accessor attributes (e.g. by month of + the year): + +.. ipython:: python + + da.groupby('time.month').sum() + +- Interpolation using :py:class:`cftime.datetime` objects: + +.. ipython:: python + + da.interp(time=[DatetimeNoLeap(1, 1, 15), DatetimeNoLeap(1, 2, 15)]) + +- Interpolation using datetime strings: + +.. ipython:: python + + da.interp(time=['0001-01-15', '0001-02-15']) + +- Differentiation: + +.. ipython:: python + + da.differentiate('time') + +- Serialization: + +.. ipython:: python + + da.to_netcdf('example-no-leap.nc') + xr.open_dataset('example-no-leap.nc') + +- And resampling along the time dimension for data indexed by a :py:class:`~xarray.CFTimeIndex`: + +.. ipython:: python + + da.resample(time='81T', closed='right', label='right', base=3).mean() + +.. note:: + + + For some use-cases it may still be useful to convert from + a :py:class:`~xarray.CFTimeIndex` to a :py:class:`pandas.DatetimeIndex`, + despite the difference in calendar types. The recommended way of doing this + is to use the built-in :py:meth:`~xarray.CFTimeIndex.to_datetimeindex` + method: + + .. ipython:: python + :okwarning: + + modern_times = xr.cftime_range('2000', periods=24, freq='MS', calendar='noleap') + da = xr.DataArray(range(24), [('time', modern_times)]) + da + datetimeindex = da.indexes['time'].to_datetimeindex() + da['time'] = datetimeindex + + However in this case one should use caution to only perform operations which + do not depend on differences between dates (e.g. differentiation, + interpolation, or upsampling with resample), as these could introduce subtle + and silent errors due to the difference in calendar types between the dates + encoded in your data and the dates stored in memory. + +.. _Timestamp-valid range: https://pandas.pydata.org/pandas-docs/stable/timeseries.html#timestamp-limitations +.. _ISO 8601-format: https://en.wikipedia.org/wiki/ISO_8601 +.. _partial datetime string indexing: https://pandas.pydata.org/pandas-docs/stable/timeseries.html#partial-string-indexing