- ...
- Updated roocs.ini default values for atlas datasets for further methods of how to infer the project name (#113).
- Updated roocs.ini default values for atlas datasets (#111).
- Updated xarray_utils module to support reading kerchunk files (#106).
- Updated
realizationdimension in common coords (#108). - Linting.
- Added Python 3.11 to tests.
- Updated requirements for cf_xarray.
- Added
realizationdimension to known coords (#103). - Update pre-commit.
- Added c3s-cmip-decadal project to default roocs.ini (#101).
- Fixed
get_coords_by_typeinxarray_utilsto handle non existing coords (#99).
- Added data_node_root in
roocs.inifor C3S-CORDEX and C3S-CMIP5 (#97).
- Updated default
roocs.inifor C3S-CORDEX (#93, #95). - Fix added for get_bbox on C3S-CORDEX (#94).
- When a project was provided to
roocs_utils.project_utils.DatasetMapper, getting the base directory would be skipped, causing an error. This has been resolved. roocs_utils.project_utils.DatasetMappercan now accept fixed_path_mappings that include ".gz" (gzip) files. This is allowed because Xarray can read gzipped netCDF files.
Intake catalog maker removed, now in it's own package: roocs/catalog-maker
Change to input parameter classes:: * Added:
roocs_utils.parameter.time_components_parameter.TimeComponentsParameter* Modified input types required for classes:* ``roocs_utils.parameter.time_parameter.TimeParameter`` * ``roocs_utils.parameter.level_parameter.LevelParameter``
- They both now require their inputs to be one of::
*
roocs_utils.parameter.param_utils.Interval- to specify a range/interval *roocs_utils.parameter.param_utils.Series- to specify a series of values
- They both now require their inputs to be one of::
*
roocs_utils.xarray_utils.xarray_utilsnow accepts keyword arguments to pass through to xarray'sopen_datasetoropen_mfdataset. If the argument provided is not an option foropen_dataset, thenopen_mfdatasetwill be used, even for one file.The roocs.ini config file can now accept fixed_path_modifiers to work together with the fixed_path_mappings section. For example, you can specify parameters in the modifiers that will be expanded into the mappings:
fixed_path_modifiers = variable:cld dtr frs pet pre tmn tmp tmx vap wet fixed_path_mappings = cru_ts.4.04.{variable}:cru_ts_4.04/data/{variable}/*.nc cru_ts.4.05.{variable}:cru_ts_4.05/data/{variable}/cru_ts4.05.1901.2*.{variable}.dat.nc.gzIn this example, the variable parameter will be expanded out to each of the options provided in the list.
The
roocs_utils.xarray_utils.xarray_utils.open_xr_dataset()function was improved so that the time units of the first data file are preserved in:ds.time.encoding["units"]. A multi-file dataset has now keeps the time "units" of the first file (if present). This is useful for converting to other formats (e.g. CSV).
- Python 3.6 no longer tested in GitHub actions.
- Remove abcunit-backend and psycopg2 dependencies from requirements.txt, these must now be manually installed in order to use the catalog maker.
- Inventory maker now removed and replaced by intake catalog maker which writes a csv file with the dataset entries and a yaml description file.
- In
etc/roocs.inithe optionuse_inventoryhas been replaced byuse_catalogand the inventory maker options have been replaced with equivalent catalog options. However, the option to include file paths or not no longer exists. - The catalog maker now uses a database backend and creates a csv file so there are 3 new dependencies for the catalog maker: pandas and abcunit-backend and psycopg2.
This means a database backend must be specified and the paths for the pickle files in etc/roocs.ini are no longer necessary. For more information see the README.
- oyaml removed as a dependency
- Added
AnyCalendarDateTimeandstr_to_AnyCalendarDateTimetoutils.time_utilsto aid in handling date strings that may not exist in all calendar types. - Inventory maker will check latitude and longitude of the dataset it is scanning are within acceptable bounds and raise an exception if they are not.
- clean up imports ... remove pandas dependency.
- cf_xarray>=0.3.1 now required due to differing level identification of coordinates between versions.
- oyaml>=0.9 - new dependency for inventory
- Interface to inventory maker changed. Detailed instructions for use added in README.
- Adjusted file name template. Underscore removed before
__derive__time_range - New dev dependency: GitPython==3.1.12
- Added
use_inventoryoption toroocs.iniconfig and allow data to be used without checking an inventory. DatasetMapperclass and wrapper functions added toroocs_utils.project_utilsandroocs_utils.xarray_utils.xarray_utilsto resolve all paths and dataset ids in the same way.FileMapperadded inroocs_utils.utils.file_utilsto resolve resolve multiple files with the same directory to their directory path.- Fixed path mapping support added in
DatasetMapper - Added
DimensionParameterto be used with the average operation.
- Removed submodule for test data. Test data is now cloned from git using GitPython and cached
CollectionParamteraccepts an instance ofFileMapperor a sequence ofFileMapperobjects- Adjusted file name template to include an
extraoption before the file extension. - Swapped from travis CI to GitHub actions
- Replaced use of
cfunitsbycf_xarrayandcftime(new dependency) inroocs_utils.xarray_utils.
Fixing pip install
- Importing and using roocs-utils when pip installing now works
Fixing formatting of doc strings and imports
- Use of
roocs_utils.parameter.parameterise.parameterise:
import should now be from roocs_utils.parameter import parameterise
and usage should be, for example parameters = parameterise(collection=ds, time=time, area=area, level=level)
- Added a notebook to show examples
- Updated formatting of doc strings
Updating the documentation and improving the changelog.
- Updated doc strings to improve documentation.
- Updated documentation.
Fixing mostly existing functionality to work more efficiently with the other packages in roocs.
environment.ymlhas been updated to bring it in line with requirements.txt.levelcoordinates would previously have been identified asNone. They are now identified aslevel.
parameterisefunction added inroocs_utils.parameterto use in all roocs packages.ROOCS_CONFIGenvironment variable can be used to override default config inetc/roocs.ini. To use a local config file setROOCS_CONFIGas the file path to this file. Several file paths can be provided separated by a:- Inventory functionality added - this can be used to create an inventory of datasets. See
READMEfor more info. project_utilsadded with the following functions to get the project name of a dataset and the base directory for that project.utils.commonandutils.time_utilsadded.is_levelimplemented inxarray_utilsto identify whether a coordinate is a level or not.
xarray_utils.xarray_utils.get_main_variableupdated to exclude common coordinates from the search for the main variable. This fixes a bug where coordinates such aslon_boundswould be returned as the main variable.
READMEupdate to explain inventory functionality.Blackandflake8formatting applied.- Fixed import warning with
collections.abc.
- First release.