Enable using instantaneous outputs for GDD generation #3445
Conversation
…DF3. Python has trouble writing large netCDF3 files. This makes it so it saves a netCDF4 file first, just to get *something*, then tries to convert to netCDF3 using nccopy. If that conversion fails for any reason, no error is raised, but a message is printed either way.
# Conflicts: # python/ctsm/crop_calendars/cropcal_utils.py # python/ctsm/crop_calendars/generate_gdds_functions.py
samsrabin
added
blocker
| if casebaseid.split("-")[-1] != "cropMonthOutput": | ||
| if casebaseid.split("-")[-1] != "cropMonthOutput" and not casebaseid.endswith( | ||
| "clm-cropMonthOutput--clm-h2a" | ||
| ): |
There was a problem hiding this comment.
Add comment explaining this.
| def convert_netcdf_to_classic(filepath): | ||
| """ | ||
| Try to convert netCDF to netCDF-3 Classic format using nccopy. | ||
| """ |
There was a problem hiding this comment.
convert_netcdf_to_classic() should be moved to netcdf_utils and unit-tested.
| temp_path = tmp.name | ||
|
|
||
| try: | ||
| subprocess.run(["nccopy", "-3", filepath, temp_path], check=True) |
There was a problem hiding this comment.
Use xarray's Dataset.to_netcdf() instead?
| patterns = [f"*h2i.{this_year-1}-01*.nc", f"*h2i.{this_year-1}-01*.nc.base"] | ||
|
|
||
| # TODO: Undo this, replacing with just h2i or h2a | ||
| patterns += [f"*h2a.{this_year-1}-01*.nc", f"*h2a.{this_year-1}-01*.nc.base"] |
There was a problem hiding this comment.
Look for h2i files first, falling back to h2a files if (a) h2i files not found or (b) variable not in h2i files.
| if convert_netcdf_to_classic(file_out): | ||
| logger.info("Conversion succeeded") | ||
| else: | ||
| logger.info("Conversion failed, perhaps because nccopy wasn't found in your shell") |
There was a problem hiding this comment.
- Check whether
nccopyis available in shell before the initialfile.to_netcdf()call. If it's not, just skip straight to old behavior of"NETCDF3_64BIT". - If conversion fails, try saving again via
file.to_netcdf(), this time straight to"NETCDF3_64BIT".
There was a problem hiding this comment.
Hey @samsrabin I'm just wondering the reasons behind the convert to classic being done here?
I don't know of any compelling reasons to prefer classic over other formats. And classic has some pretty strong restrictions. And it seems preferable to me to write it out in the preferred format from the get go rather than convert. But I would like to hear the thinking here.
There was a problem hiding this comment.
Yeah, I think I meant for this all to be netCDF-3 64-bit. Will fix.
samsrabin
modified the milestones:
cesm3.0.0 release branch tag,
ctsm6.0.0 (code freeze)
Description of changes
Enables using instantaneous instead of averaged outputs for GDD generation. Also includes some relevant performance improvements and testing.
Specific notes
Contributors other than yourself, if any: None
CTSM Issues Fixed:
Are answers expected to change (and if so in what way)? No
Any User Interface Changes (namelist or namelist defaults changes)? No
Does this create a need to change or add documentation? Did you do so? No
Testing performed, if any:
rxcropmaturityto confirm successful completion (and no diffs for tests that have baseline)clm_pymodssuites to confirm no diffs