Recipe test results for v2.13.0

[jlenh]

See updates at #4157 (comment)

---

@ESMValGroup/esmvaltool-developmentteam Here is an overview of the tests performed for the v2.13.0 release. The results are summarized in /home/b/b381312/projects/outputs_rc1/run1 (waiting for access to the VM).

Here is the conda environment: esmvaltool-core_v2.13.0rc1.txt

More details on the errors are added gradually!

Recipe running session 2025-08-22

136 out of 180 recipes ran successfully, 38 recipes failed. You can find below the errors summarized.

Recipes that failed because the diagnostic script failed (13 out of 180)

• recipe_arctic_ocean.yml
  • ERROR = in plt.savefig(), cartopy/io/shapereader.acquire_all_resources http.client.IncompleteRead
  • works fine
• ERROR = autoasses._plot_mo_metrics normalise AssertionError: ref and test must have same set of metrics
  • [RECIPE FAILURE]: recipe_autoassess_landsurface_soilmoisture.yml: AssertionError: ref and test must have same set of metrics #4130
  • Fix autoassess normalise error #4187
  • recipe_autoassess_landsurface_permafrost.yml
  • recipe_autoassess_landsurface_soilmoisture.yml
  • recipe_autoassess_landsurface_surfrad.yml
  • recipe_autoassess_stratosphere.yml
• recipe_carvalhais14nat.yml
  • ERROR = land_carbon_cycle.shared._apply_common_task#l29 KeyError: 'dat_mask0'
  • Fix recipe_carvalhais14nat.yml to avoid the usage of vars() inside a function #4170
• recipe_flato13ipcc_figure_914.yml
  • ERROR = in ipcc_ar5/ch09_fig09_14._fix_lons#L81 ValueError: Unable to avoid copy while creating an array as requested.
  • cf [RECIPE FAILURE]: ipccwg1ar5ch9/recipe_flato13ipcc_figure_914.yml: ValueError: Unable to avoid copy while creating an array as requested. #4034
  • Fix _fix_lons function in recipe_flato13ipcc_figure_914.yml due to numpy error [#4034] #4178
• recipe_hyint.yml
  • ERROR = cdo remapscon2... Operator >remapscon2< not found!
  • cdo remapscon2 eventually removed since CDO version 2.3.0 #4163
  • Solved w/ Update the name of the remapscon2 operator in R recipes #4164
• recipe_hyint_extreme_events.yml
  • ERROR = cdo remapscon2... Operator >remapscon2< not found!
  • cdo remapscon2 eventually removed since CDO version 2.3.0 #4163
  • Solved w/ Update the name of the remapscon2 operator in R recipes #4164
• recipe_ipccwg1ar6ch3_fig_3_42_b.yml
  • ERROR = perfmetrics/main.ncl, NCL: fatal:systemfunc: cannot create child process:[errno=12]
• recipe_seaice_drift.yml
  • ERROR = seaice_drift.seaice_drift._get_mask#L158, TypeError: Can't instantiate abstract class InsidePolygonFactory without an implementation for abstract method '_calculate_array'
  • Solved w/ Add missing method in seaice_drift.py #4161
• recipe_williams09climdyn_CREM.yml
  • ERROR = crem/ww09_esmvaltool.crem_calc#L606, AttributeError: module 'numpy' has no attribute 'NAN'.
  • Solved w/ Fix recipe_williams09climdyn_CREM.yml by replacing np.NAN with np.nan #4160
• recipe_zmnam.yml
  • ERROR = zmnam/zmnam_plot.zmnam_plot#L249, AttributeError: 'GeoContourSet' object has no attribute 'collections'
  • [v2.12.0] recipe_zmnam.yml diagnostic failure #3938
  • Fix small matplotlib errors in recipe_zmnam.yml #4177

Recipes that failed because of missing data (5 out of 180)

• recipe_aod_aeronet_assess.yml
  • formatting for changed version, see comment below
  • Solved cf Update recipe_check_obs.yml to match the latest AERONET data #4171
• recipe_check_obs.yml
  • see summarizing comment below
  • Remove end_year facet for NOAA-GML datasets in recipe_check_obs.yml #4167
• recipe_ocean_quadmap.yml
  • file structure from obs4mips does not correspond to files in OBS
  • [RECIPE FAILURE]: Wrong dataset description for WOA in recipe_ocean_quadmap #4175
  • Fix specification for WOA dataset in recipe_ocean_quadmap #4176
• recipe_perfmetrics_land_CMIP5.yml
  • Data was actually just not downloaded
  • Recipe working cf discussion in Recipe recipe_perfmetrics_land_CMIP5.yml fails because of wrong dataset name CERES-EBAF_Surface #4159
• recipe_ref_sea_ice_area_basic.yml
  • data needs re-downloading + formatting for changed version, see comment below

Recipes that failed because the run took too long (11 out of 180)

• recipe_climate_change_hotspot.yml
• recipe_eyring06jgr.yml
• recipe_eyring13jgr_12.yml (preprocessor creates output but then hangs for unknown reasons, diagnostic script is not called)
• recipe_flato13ipcc_figures_926_927.yml
• recipe_ipccwg1ar6ch3_fig_3_19.yml
• recipe_lauer22jclim_fig5_lifrac.yml (that's a known problem as extract_levels is very slow)
• recipe_miles_block.yml
• recipe_miles_eof.yml
• recipe_miles_regimes.yml
• recipe_wenzel14jgr.yml
• recipe_wenzel16jclim.yml

Recipes that failed because they used too much memory (2 out of 180)

• recipe_lauer22jclim_fig3-4_zonal.yml (that's a know problem as the bot does not have enough memory)
• recipe_ref_trend_regions.yml

Recipes that failed of other reasons or are still running (13 out of 180)

• ERROR = cftime._cftime._dateparse ValueError: zero not allowed as a reference year when has_year_zero=False
  • [RECIPE FAILURE]: ipccwg1ar5ch9/recipe_flato13ipcc_figure_96.yml: ValueError: zero not allowed as a reference year when has_year_zero=False #4032
  • Set correct calendar when reading file dates ESMValCore#2826
  • recipe_bock20jgr_fig_1-4.yml
  • recipe_bock20jgr_fig_6-7.yml
  • recipe_bock20jgr_fig_8-10.yml
  • recipe_flato13ipcc_figure_96.yml
  • recipe_flato13ipcc_figures_938_941_cmip3.yml
  • recipe_ipccwg1ar6ch3_fig_3_42_a.yml
  • recipe_ipccwg1ar6ch3_fig_3_43.yml
• recipe_climwip_brunner2019_med.yml
  • ERROR = ValueError: Unable to convert from 'Unit('1.15740740740741e-05 kg.s-4')' to 'Unit('W m-2')'.
  • [v2.11.0 release] Unable to convert units issue with recipe_climwip_brunner2019_med.yml #3694
• recipe_collins13ipcc.yml
  • ERROR = OSError: [Errno -101] NetCDF: HDF error
• recipe_ipccwg1ar6ch3_atmosphere.yml
  • ERROR = add_ancillary_variable ValueError: Failed to add land_area_fraction
  • Preprocessor add_supplementary_variable fails w/ datasets pr_Amon_EC-Earth3_historical_r2i1p1f1_gr/tas_Amon_EC-Earth3_historical_r2i1p1f1_gr and sftlf_fx_EC-Earth3_historical_r1i1p1f1_gr ESMValCore#2820
• recipe_ipccwg1ar6ch3_fig_3_9.yml
  • ERROR = add_ancillary_variable ValueError: Failed to add land_area_fraction
  • Preprocessor add_supplementary_variable fails w/ datasets pr_Amon_EC-Earth3_historical_r2i1p1f1_gr/tas_Amon_EC-Earth3_historical_r2i1p1f1_gr and sftlf_fx_EC-Earth3_historical_r1i1p1f1_gr ESMValCore#2820
• recipe_python.yml
  • ERROR = Failed to run preprocessor function 'concatenate', iris.exceptions.CoordinateNotFoundError: "Expected to find exactly 1 'time' coordinate, but found none.
  • works fine
• recipe_russell18jgr.yml
  • ERROR = add_ancillary_variable ValueError: Failed to add sea_area_fraction
  • similar to Preprocessor add_supplementary_variable fails w/ datasets pr_Amon_EC-Earth3_historical_r2i1p1f1_gr/tas_Amon_EC-Earth3_historical_r2i1p1f1_gr and sftlf_fx_EC-Earth3_historical_r1i1p1f1_gr ESMValCore#2820
  • Fix minor NCL errors for recipe_russel18jgr.yml #4183

Recipes that never ran (1 out of 180)

• recipy_capacity_factor.yml

[jlenh]

@ESMValGroup/obs-maintainers related issues (some need to be addressed directly on DKRZ):

recipe_check_obs.yml is failing because:

• some datasets/variables are missing from the obs data pool:
  • CMAP = pr
  • IAP = thetao/tos
  • JRA-55 = cli/clivi/clw/clwvi/clt/prw/rlus/rlut/rlutcs/rsus/rsuscs/rsut/rsutcs/ta/tas/wap (part of these are in the Tier1 folder but Tier2 is indicated in the recipe)
    • available cf. Recipe test results for v2.13.0 #4157 (comment)
  • NCEP-DOE-R2 = tauu/tauv/tos
  • Solved w/ Recipe test results for v2.13.0 #4157 (comment)
• some datasets have non-updated versions (needs downloading + formatting)
  • OSI-450-nh = day/OImon sic/areacello (only v2)
  • OSI-450-sh = day/OImon sic/areacello (only v2)
  • Soved w/ Recipe test results for v2.13.0 #4157 (comment)
  • AERONET.od440aer (version not updated)
    • Solved w/ Update recipe_check_obs.yml to match the latest AERONET data #4171
• some datasets have an issue handling the different version attributes inside the recipe
  • ESACCI-CLOUD = daily clwvi/cod
  • Solved w/ Version facet not being updated when using attributes from parent variable for ESACCI-CLOUD in recipe_check_obs #4158

Log available here
recipe_check_obs.19274716.txt

[valeriupredoi]

many thanks, Julien! For the Autoassess ones, I'll have to ask @alistairsellar to pop by and give you a hand, when he's got a minute, I'm literally just about to leave on annual leave until Sep 11, and hopefully the release will have happened by then 😁 🍻

[ehogan]

many thanks, Julien! For the Autoassess ones, I'll have to ask @alistairsellar to pop by and give you a hand, when he's got a minute, I'm literally just about to leave on annual leave until Sep 11, and hopefully the release will have happened by then 😁 🍻

@alistairsellar is also about to go on leave until Septmeber 🏖️ I will try my best to take a look 😊

[valeriupredoi]

many thanks, Julien! For the Autoassess ones, I'll have to ask @alistairsellar to pop by and give you a hand, when he's got a minute, I'm literally just about to leave on annual leave until Sep 11, and hopefully the release will have happened by then 😁 🍻

@alistairsellar is also about to go on leave until Septmeber 🏖️ I will try my best to take a look 😊

legend, Emma! Many thanks 🍻