Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
dc247e7
first draft
HGWright Oct 24, 2024
989c9ec
WIP
HGWright Oct 25, 2024
b756210
fix tests and example
HGWright Oct 25, 2024
78cb7dd
Merge branch 'main' into bounds_derived
HGWright Oct 25, 2024
22783f3
Updated environment lockfiles (#6197)
scitools-ci[bot] Oct 28, 2024
23c74fe
added in a vertical rule for surface fields (#5734)
ESadek-MO Oct 28, 2024
6efa889
Specify meta in dask.array.map_blocks (#5989)
bouweandela Oct 28, 2024
3d4df12
Update CF standard names table. (#6200)
ESadek-MO Oct 28, 2024
8687e0b
What's new updates for v3.11.0rc0 . (#6201)
ESadek-MO Oct 28, 2024
60cc56f
correct major minor in whatsnew (#6202)
ESadek-MO Oct 28, 2024
4ca96b8
Restore latest Whats New files.
ESadek-MO Oct 30, 2024
b0bdf28
[pre-commit.ci] pre-commit autoupdate (#6205)
pre-commit-ci[bot] Oct 30, 2024
6ec6360
Bump scitools/workflows from 2024.10.2 to 2024.10.3 (#6208)
dependabot[bot] Oct 31, 2024
4646752
WIP promoting the bounds to a cube
HGWright Feb 28, 2025
e743a67
Use the new re-entrant do-nothing script (#6062)
trexfeathers Nov 4, 2024
7104f26
Add a pytest-unittest translator to the conversion guide (#6213)
ESadek-MO Nov 4, 2024
4b5cf25
[pre-commit.ci] pre-commit autoupdate (#6217)
pre-commit-ci[bot] Nov 5, 2024
be9a04e
Bump scitools/workflows from 2024.10.3 to 2024.11.2 (#6220)
dependabot[bot] Nov 7, 2024
3d2f122
removed double reference of typehints and enhancements (#6219)
ESadek-MO Nov 7, 2024
4e4138c
V3.11.x.updates (#6224)
ESadek-MO Nov 11, 2024
0671e2b
Bump scitools/workflows from 2024.11.2 to 2024.11.3 (#6230)
dependabot[bot] Nov 14, 2024
8643b92
[pre-commit.ci] pre-commit autoupdate (#6226)
pre-commit-ci[bot] Nov 15, 2024
00a9191
[pre-commit.ci] pre-commit autoupdate (#6233)
pre-commit-ci[bot] Nov 18, 2024
0a95b5a
Bump codecov/codecov-action from 4 to 5 (#6235)
dependabot[bot] Nov 21, 2024
fde0fe7
Bump scitools/workflows from 2024.11.3 to 2024.11.5 (#6236)
dependabot[bot] Nov 21, 2024
91cba1e
[pre-commit.ci] pre-commit autoupdate (#6238)
pre-commit-ci[bot] Nov 25, 2024
af475b1
Reduce default number of benchmark rounds. (#6234)
trexfeathers Nov 28, 2024
80111b0
Bump scitools/workflows from 2024.11.5 to 2024.11.7 (#6241)
dependabot[bot] Nov 29, 2024
f959a95
[pre-commit.ci] pre-commit autoupdate (#6242)
pre-commit-ci[bot] Dec 3, 2024
fc6ce56
Fix broken link. (#6246)
pp-mo Dec 10, 2024
1308f6e
[pre-commit.ci] pre-commit autoupdate (#6245)
pre-commit-ci[bot] Dec 10, 2024
ab3cfac
Bump scitools/workflows from 2024.11.7 to 2024.12.0 (#6243)
dependabot[bot] Dec 10, 2024
697ba60
Remove freepik.com link due to linkcheck breakage (#6261)
trexfeathers Dec 17, 2024
a5d7424
[pre-commit.ci] pre-commit autoupdate (#6259)
pre-commit-ci[bot] Dec 17, 2024
50ffdbd
`date_microseconds` FUTURE flag (#6260)
trexfeathers Dec 17, 2024
1cb07a7
Pin dask <2024.9 (#6255)
stephenworsley Dec 18, 2024
68f66fe
Bump scitools/workflows from 2024.12.0 to 2024.12.3 (#6269)
dependabot[bot] Dec 19, 2024
80bcec7
mrePin dask <2024.9 (#6255)
stephenworsley Dec 18, 2024
99bc2d7
removed use of shared utils
ESadek-MO Dec 19, 2024
fed1e40
edited whatsnew
ESadek-MO Dec 20, 2024
d388bdc
What's new updates for v3.11.1 . (#6271)
ESadek-MO Dec 19, 2024
46d876e
Equalise cubes (#6257)
pp-mo Dec 20, 2024
fa29936
[pre-commit.ci] pre-commit autoupdate (#6274)
pre-commit-ci[bot] Dec 23, 2024
f58c787
Bump scitools/workflows from 2024.12.3 to 2024.12.4 (#6275)
dependabot[bot] Dec 26, 2024
47f455b
Bump scitools/workflows from 2024.12.4 to 2024.12.5 (#6276)
dependabot[bot] Jan 2, 2025
d035669
Adopt update common refresh-lockfiles action. (#6278)
pp-mo Jan 3, 2025
d09b683
Updated environment lockfiles (#6279)
scitools-ci[bot] Jan 3, 2025
599d7a9
Updated environment lockfiles (#6280)
scitools-ci[bot] Jan 4, 2025
48bda8d
[pre-commit.ci] pre-commit autoupdate (#6281)
pre-commit-ci[bot] Jan 6, 2025
9cd6998
Bump scitools/workflows from 2024.12.5 to 2025.01.1 (#6282)
dependabot[bot] Jan 9, 2025
8df3b42
Updated environment lockfiles (#6283)
scitools-ci[bot] Jan 11, 2025
4508dd5
[pre-commit.ci] pre-commit autoupdate (#6284)
pre-commit-ci[bot] Jan 13, 2025
11f6e55
Bump scitools/workflows from 2025.01.1 to 2025.01.3 (#6287)
dependabot[bot] Jan 16, 2025
e5acb50
Lazy median aggregator (#6167)
fnattino Jan 17, 2025
40ee9e7
Updated environment lockfiles (#6291)
scitools-ci[bot] Jan 18, 2025
ab2a347
Add support for saving string stash codes to PP (#6289)
rcomer Jan 20, 2025
fff727e
[pre-commit.ci] pre-commit autoupdate (#6294)
pre-commit-ci[bot] Jan 20, 2025
17c874a
Revert "Fix broken link. (#6246)" (#6297)
trexfeathers Jan 27, 2025
723f4e5
Lazy rectilinear interpolator (#6084)
fnattino Jan 27, 2025
6c0ce9c
Bump scitools/workflows from 2025.01.3 to 2025.01.4 (#6295)
dependabot[bot] Jan 28, 2025
99108cd
Bump scitools/workflows from 2025.01.4 to 2025.01.5 (#6300)
dependabot[bot] Jan 30, 2025
2839fca
Updated environment lockfiles (#6296)
scitools-ci[bot] Jan 30, 2025
9815900
Use cube chunks for weights in aggregations with smart weights (#6288)
schlunma Jan 30, 2025
a883b80
[pre-commit.ci] pre-commit autoupdate (#6298)
pre-commit-ci[bot] Feb 5, 2025
9f14d2f
Improve speed of loading small NetCDF files (#6229)
bouweandela Feb 6, 2025
27e532f
Updated environment lockfiles (#6279)
scitools-ci[bot] Feb 10, 2025
990291c
Bump scitools/workflows from 2025.01.5 to 2025.02.0 (#6306)
dependabot[bot] Feb 10, 2025
37ae6f7
[pre-commit.ci] pre-commit autoupdate (#6310)
pre-commit-ci[bot] Feb 12, 2025
6b9ecec
Bump scitools/workflows from 2025.02.0 to 2025.02.1 (#6313)
dependabot[bot] Feb 13, 2025
c8334f2
Cache Dask arrays created from `NetCDFDataProxy`s to speed up loading…
bouweandela Feb 14, 2025
799bdfd
[pre-commit.ci] pre-commit autoupdate (#6315)
pre-commit-ci[bot] Feb 18, 2025
4dd050b
replaced reference from build to python build (#6324)
ESadek-MO Feb 19, 2025
50d9ed6
WIP
HGWright Oct 25, 2024
9eb29de
rebase
HGWright Oct 25, 2024
656ee1c
WIP promoting the bounds to a cube
HGWright Feb 28, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci-manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ concurrency:
jobs:
manifest:
name: "check-manifest"
uses: scitools/workflows/.github/workflows/ci-manifest.yml@2024.10.2
uses: scitools/workflows/.github/workflows/ci-manifest.yml@2025.02.1
2 changes: 1 addition & 1 deletion .github/workflows/ci-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,6 @@ jobs:

- name: "upload coverage report"
if: ${{ matrix.coverage }}
uses: codecov/codecov-action@v4
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
2 changes: 1 addition & 1 deletion .github/workflows/refresh-lockfiles.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,5 @@ on:

jobs:
refresh_lockfiles:
uses: scitools/workflows/.github/workflows/refresh-lockfiles.yml@2024.10.2
uses: scitools/workflows/.github/workflows/refresh-lockfiles.yml@2025.02.1
secrets: inherit
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -79,3 +79,6 @@ docs/iris_image_test_output/

# Files generated during test runs.
lib/iris/tests/results/**/*.dot

# Uses of SciTools-incubator/nothing.
**/.nothing
12 changes: 6 additions & 6 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ repos:
- id: no-commit-to-branch

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: "v0.7.0"
rev: "v0.9.6"
hooks:
- id: ruff
types: [file, python]
Expand All @@ -38,32 +38,32 @@ repos:
types: [file, python]

- repo: https://github.com/codespell-project/codespell
rev: "v2.3.0"
rev: "v2.4.1"
hooks:
- id: codespell
types_or: [asciidoc, python, markdown, rst]
additional_dependencies: [tomli]

- repo: https://github.com/PyCQA/flake8
rev: 7.1.1
rev: 7.1.2
hooks:
- id: flake8
types: [file, python]

- repo: https://github.com/asottile/blacken-docs
rev: 1.19.0
rev: 1.19.1
hooks:
- id: blacken-docs
types: [file, rst]

- repo: https://github.com/aio-libs/sort-all
rev: v1.2.0
rev: v1.3.0
hooks:
- id: sort-all
types: [file, python]

- repo: https://github.com/pre-commit/mirrors-mypy
rev: 'v1.12.1'
rev: 'v1.15.0'
hooks:
- id: mypy
additional_dependencies:
Expand Down
52 changes: 52 additions & 0 deletions benchmarks/benchmarks/load/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,58 @@ def time_many_var_load(self) -> None:
_ = load(str(self.FILE_PATH))


class ManyCubes:
FILE_PATH = BENCHMARK_DATA / "many_cube_file.nc"

@staticmethod
def _create_file(save_path: str) -> None:
"""Run externally - everything must be self-contained."""
import numpy as np

from iris import save
from iris.coords import AuxCoord, DimCoord
from iris.cube import Cube, CubeList

data_len = 81920
bnds_len = 3
data = np.arange(data_len).astype(np.float32)
bnds_data = (
np.arange(data_len * bnds_len)
.astype(np.float32)
.reshape(data_len, bnds_len)
)
time = DimCoord(np.array([0]), standard_name="time")
lat = AuxCoord(
data, bounds=bnds_data, standard_name="latitude", units="degrees"
)
lon = AuxCoord(
data, bounds=bnds_data, standard_name="longitude", units="degrees"
)
cube = Cube(data.reshape(1, -1), units="unknown")
cube.add_dim_coord(time, 0)
cube.add_aux_coord(lat, 1)
cube.add_aux_coord(lon, 1)

n_cubes = 100
cubes = CubeList()
for i in range(n_cubes):
cube = cube.copy()
cube.long_name = f"var_{i}"
cubes.append(cube)
save(cubes, save_path)

def setup_cache(self) -> None:
if not REUSE_DATA or not self.FILE_PATH.is_file():
# See :mod:`benchmarks.generate_data` docstring for full explanation.
_ = run_function_elsewhere(
self._create_file,
str(self.FILE_PATH),
)

def time_many_cube_load(self) -> None:
_ = load(str(self.FILE_PATH))


class StructuredFF:
"""Test structured loading of a large-ish fieldsfile.

Expand Down
2 changes: 1 addition & 1 deletion benchmarks/bm_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
GH_REPORT_DIR = ROOT_DIR.joinpath(".github", "workflows", "benchmark_reports")

# Common ASV arguments for all run_types except `custom`.
ASV_HARNESS = "run {posargs} --attribute rounds=4 --interleave-rounds --show-stderr"
ASV_HARNESS = "run {posargs} --attribute rounds=3 --interleave-rounds --show-stderr"


def echo(echo_string: str):
Expand Down
2 changes: 1 addition & 1 deletion docs/src/community/iris_xarray.rst
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ output is not fully CF compliant (as-per `the cf checker <https://cfchecker.ncas
approach in Iris, and means that the use of the "coordinates" attribute in output is
often not CF compliant.
* dates are converted to datetime-like objects internally. There are special features
providing `support for non-standard calendars <https://docs.xarray.dev/en/stable/user-guide/weather-climate.html#non-standard-calendars-and-dates-outside-the-nanosecond-precision-range>`_,
providing `support for non-standard calendars <https://docs.xarray.dev/en/stable/user-guide/weather-climate.html#non-standard-calendars-and-dates-outside-the-precision-range>`_,
however date units may not always be saved correctly.
* CF-style coordinate bounds variables are not fully understood. The CF approach
where bounds variables do not usually define their units or standard_names can cause
Expand Down
2 changes: 1 addition & 1 deletion docs/src/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ def _dotv(version):
.. |python_version| replace:: {build_python_version}
.. |python_support| replace:: {python_support}
.. |iris_version| replace:: v{version}
.. |build_date| replace:: ({datetime.datetime.now().strftime('%d %b %Y')})
.. |build_date| replace:: ({datetime.datetime.now().strftime("%d %b %Y")})
"""

# Add any Sphinx extension module names here, as strings. They can be
Expand Down
76 changes: 76 additions & 0 deletions docs/src/developers_guide/contributing_pytest_conversions.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
.. include:: ../common_links.inc

.. _contributing_pytest_conversions:

*******************************************
Converting From ``unittest`` to ``pytest``
*******************************************

Conversion Checklist
--------------------
.. note::
Please bear in mind the following checklist is for general use; there may be
some cases which require extra context or thought before implementing these changes.

#. Before making any manual changes, run https://github.com/dannysepler/pytestify
on the file. This does a lot of the brunt work for you!
#. Check for references to :class:`iris.tests.IrisTest`. If a class inherits
from this, remove the inheritance. Inheritance is unnecessary for
pytest tests, so :class:`iris.tests.IrisTest` has been deprecated
and its convenience methods have been moved to the
:mod:`iris.tests._shared_utils` module.
#. Check for references to ``unittest``. Many of the functions within unittest
are also in pytest, so often you can just change where the function is imported
from.
#. Check for references to ``self.assert``. Pytest has a lighter-weight syntax for
assertions, e.g. ``assert x == 2`` instead of ``assertEqual(x, 2)``. In the
case of custom :class:`~iris.tests.IrisTest` assertions, the majority of these
have been replicated in
:mod:`iris.tests._shared_utils`, but with snake_case instead of camelCase.
Some :class:`iris.tests.IrisTest` assertions have not been converted into
:mod:`iris.tests._shared_utils`, as these were deemed easy to achieve via
simple ``assert ...`` statements.
#. Check for references to ``setUp()``. Replace this with ``_setup()`` instead.
Ensure that this is decorated with ``@pytest.fixture(autouse=True)``.

.. code-block:: python

@pytest.fixture(autouse=True)
def _setup(self):
...

#. Check for references to ``@tests``. These should be changed to ``@_shared_utils``.
#. Check for references to ``with mock.patch("...")``. These should be replaced with
``mocker.patch("...")``. ``mocker`` is a fixture, and can be passed into functions.
#. Check for ``np.testing.assert...``. This can usually be swapped for
``_shared_utils.assert...``.
#. Check for references to ``super()``. Most test classes used to inherit from
:class:`iris.tests.IrisTest`, so references to this should be removed.
#. Check for references to ``self.tmp_dir``. In pytest, ``tmp_path`` is used instead,
and can be passed into functions as a fixture.
#. Check for ``if __name__ == 'main'``. This is no longer needed with pytest.
#. Check for ``mock.patch("warnings.warn")``. This can be replaced with
``pytest.warns(match=message)``.
#. Check the file against https://github.com/astral-sh/ruff , using ``pip install ruff`` ->
``ruff check --select PT <file>``.

Common Translations
-------------------

.. list-table::
:widths: 50 50
:header-rows: 1

* - ``unittest`` method
- ``pytest`` equivalent
* - ``assertTrue(x)``
- ``assert x``
* - ``assertFalse(x)``
- ``assert not x``
* - ``assertRegex(x, y)``
- ``assert re.match(y, x)``
* - ``assertRaisesRegex(cls, msg_re)``
- ``with pytest.raises(cls, match=msg_re):``
* - ``mock.patch(...)``
- ``mocker.patch(...)``

2 changes: 1 addition & 1 deletion docs/src/developers_guide/contributing_running_tests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ experimental dependency not being present.
SKIPPED [1] lib/iris/tests/unit/util/test_demote_dim_coord_to_aux_coord.py:29: Test(s) require external data.

All Python decorators that skip tests will be defined in
``lib/iris/tests/__init__.py`` with a function name with a prefix of
``lib/iris/tests/_shared_utils.py`` with a function name with a prefix of
``skip_``.

You can also run a specific test module. The example below runs the tests for
Expand Down
147 changes: 0 additions & 147 deletions docs/src/developers_guide/contributing_testing.rst

This file was deleted.

Loading