Develop a tool for generating SLIM surface datasets (surdat files) #71
Conversation
I happened to start with the version sitting in my new_rawpftlai branch with git describe alpha-ctsm5.2.mksrf.14_ctsm5.1.dev111-5-gb1204ca46
SLIM's usermods_dirs/global_uniform uses this file: /glade/p/cesmdata/cseg/inputdata/lnd/slim/surdat/globalconst_alpha0.2_soilcv2e6_hc0.1_rs100.0_glc_hc0.01_f19_cdf5_20211105.nc and I looked at it for guidance in selecting idealized defaults.
make stest and make utest run 2 and 7 tests respectively and complete successfully
It's never supposed to be None, so the if statement could have masked a problem
Resolved conflicts: tools/README
Used as a template the jupyter notebook that Marysa recommended
|
I need to add metadata to the surdat variables (see suggestion in this post) |
|
I generated this file with the jupyter notebook: To review this work, others (@ekluzek, @marysa?, @islasimpson?) could look at and/or run this jupyter notebook draft by checking out my branch and following the instructions in @ekluzek we can discuss in more detail at Stand-up tomorrow. |
|
Taking a look at these notebooks now! |
|
Thank you @marysa! Oh, also I should mention that your comments are welcome, though I will not respond for a few weeks due to travel. |
|
No luck yet; I'm confused on a few fronts:
|
|
oooooh possibly scratch that, I see the .ipynb in your folder, just didn't have them in the copy I pulled from github. I'll copy yours and give it a go! |
|
@slevisconsulting and I talked about this. This can come in when it's ready. It's completion updates the minor version number of SLIM as it makes SLIM into a complete package. The PR I'm working on is on a different branch, and orthogonal to this, so the order doesn't matter. This will need to be updated and conflicts resolved to make it compatible with the latest cesm2_1_slim branch. That update will make it easier to see what this actually brings in. |
Modify the test mods so that fewer can be used. compsets set the scenario for most cases, and only use test mods if needed. Also add prealpha, prebeta and slim_sci test lists. Work on the README files a bit. Also don't remove all the history fields and add a list, just use the default list. Resolved conflicts: python/Makefile python/README.md python/pyproject.toml python/slim/.pylintrc python/slim/path_utils.py python/slim/utils.py I generally kept the mods from the slim0.2.002_release-cesm2.1.4 tag. However in python/slim/utils.py I added back a couple of functions and a few imports so that "make test" would pass. Test resutls: make test PASS make black PASS make lint gives this error: slim/modify_input_files/modify_surdat.py:169:19: E0602: Undefined variable 'os' (undefined-variable) I will resolve this when I finalize PR ESCOMP#69 which pertains to modify_surdat.py
|
In addition to the performed tests that I listed in the last commit notes, I also reran the python notebook from this PR and got bit-for-bit the same surdat file from it as before (last file I had generated in March). |
|
Running the test-suites. Cheyenne gnu OK except in some tests I get this NLCOMP difference. @ekluzek is it a problem? Cheyenne intel OK This test passed when I reran it: |
|
@slevis-lmwg I see those differences on occasion as well. I'm not sure why they happen. It's just changing the PIO type so should be identical. We do want to be using pnetcdf, and my baseline is the one without pnetcdf, so yours is correct anyway. Maybe there is a hidden environment variable that is different between the two of us? |
Fixes #36
As of opening this PR, my code reads a ctsm history file and outputs a file that contains:
glc_mask
lsmlon
lsmlat
Note that I started this branch from my surdat_modifier branch to avoid repeating a bunch of shared python work between that branch and this one.
I need to check with @marysa what calculations I need to include to generate a surdat file.