cam6_3_086: Introduce PUMAS DDT #632

cacraigucar · 2022-08-03T23:10:47Z

Brings in the PUMAS DDT (proc_rates) plus a few other minor PRs

closes #589
closes #703
closes #706
closes #707
closes #711

Partially address #708

cacraigucar · 2022-08-03T23:13:29Z

@andrewgettelman and @Katetc - This is the full implementation of the DDT based on the list of variables that @andrewgettelman provided in issue #589. Please review this and let me know if there are any issues or if additional fields should be added.

Also, the variable "prodsnow" is not used in cam_dev/micro_pumas_cam.F90 once it is returned. I just wanted to point this out

andrewgettelman

Why are some variables 'trop_cld_lev' and some just 'lev'? It doesn't seem to be logical? E.g. UMR, UMS = lev, but UMG = trop_cld_lev?
So now everything is in 'proc_rates'. I think this looks pretty readable to me and don't see an issue.

Thanks!

Andrew

cacraigucar · 2022-08-08T21:20:39Z

Why are some variables 'trop_cld_lev' and some just 'lev'? It doesn't seem to be logical? E.g. UMR, UMS = lev, but UMG = trop_cld_lev?

So now everything is in 'proc_rates'. I think this looks pretty readable to me and don't see an issue.

Thanks!

Andrew

The answer to #1 is that you found a bug! I have added trop_cld_lev to these variables and will push it soon.

#2 Yes, the new DDT is called proc_rates. The name can be anything, so if you have a better name, let me know.

src/physics/cam_dev/micro_pumas_cam.F90

sjsprecious · 2022-08-09T05:23:22Z

A naive question: can we pass all the columns and levels to PUMAS in the future so that we do not need to worry about 'ncol' vs. 'pcol' and 'trop_cld_lev' vs. 'lev'? By doing this, we will compute some additional columns and levels, but I guess we won't use them anywhere else and it should hurt nothing. Or will these additional columns/levels crash PUMAS later?

cacraigucar · 2022-08-09T17:25:52Z

A naive question: can we pass all the columns and levels to PUMAS in the future so that we do not need to worry about 'ncol' vs. 'pcol' and 'trop_cld_lev' vs. 'lev'? By doing this, we will compute some additional columns and levels, but I guess we won't use them anywhere else and it should hurt nothing. Or will these additional columns/levels crash PUMAS later?

@andrewgettelman and/or @Katetc - @sjsprecious question will need to be answered by one of you. I don't know the answer.

andrewgettelman · 2022-08-09T17:31:14Z

We decided for various reasons to have a smaller number of levels in the diagnostic output fields since they would just be extra memory to carry around. I still think it is a bit asymmetric, but for efficiency we decided to do this (less memory used).

pcols v. ncols has to do with potential uneven allocations by chunks I believe. I'm not sure why it goes all the way into the microphysics, I thought that was all interface level.

Katetc · 2022-08-09T17:31:19Z

@sjsprecious So columns that are >ncol but <pcol are typically not initialized or used anywhere in the model, and that could cause a crash. Levels above trop_cld_lev should, in theory, be ok but I don't think other parts of the model expect the physics to look at those, so they could have bad values. Also, the very high-top model, WACCMX, has many levels of atmosphere above the cloud top that should not have any microphysics working on, and we could slow down that model if we have to look at all of those levels in microphysics.

sjsprecious · 2022-08-09T18:09:58Z

Thanks @andrewgettelman and @Katetc for your clarifications. The reason I asked this question is that I have observed a GPU performance hit with the unpacked interface for PUMAS. This is due to the fact that GPU is accessing partial slices of an array here and GPU may have to generate an underlying temporary array first (@johnmauff could comment more on this issue). It won't be a problem if we could simply pass all the columns and levels to GPU, but the answers above make sense to me.

andrewgettelman · 2022-08-09T19:05:38Z

We need to assess whether the extra memory or the GPU efficiency is more important. How can we do that?

cacraigucar · 2022-08-09T19:08:17Z

I would suggest that this discussion of memory/GPU efficiency should become its own issue. A few months from now, this discussion may be hard to find as it is buried in the DDT PR.

sjsprecious · 2022-08-09T21:00:34Z

Thanks @andrewgettelman and @cacraigucar for your comments. I agree that my question should be a separate discussion and I apologize for bringing it up here. We can definitely discuss more in another GitHub issue.

One quick question about this PR: Is using proc_rates part of the efforts of SIMA or CCPP work? Or is it only to make the argument list shorter?

andrewgettelman · 2022-08-10T14:36:15Z

The derived type for process rates is just to make the argument list shorter. It eliminates some steps in adding diagnostics to PUMAS down the road.

…pumas_ddt

…umas_ddt

Merge pull request ESCOMP#700 from peverwhee/update_externals

Initializing variables to avoid startup interpolation error in cam_dev.

…pumas_ddt

Katetc

My only issue is that I don't understand what is up with bergso_grid. Other than that, it all looks great!!

src/physics/cam_dev/micro_pumas_cam.F90

nusbaume

Looks good to me! Just a couple final requests to add some test descriptions, and one question related to a cam-dev change. Thanks!

nusbaume · 2022-12-06T19:51:04Z

cime_config/testdefs/testlist_cam.xml

@@ -1516,7 +1516,7 @@
      <option name="wallclock">00:20:00</option>
    </options>
  </test>
-  <test compset="FCnudged" grid="ne0CONUSne30x8_ne0CONUSne30x8_mt12" name="SMS_D_Ln9_Vnuopc" testmods="cam/outfrq9s_refined_camchem">
+  <test compset="FCnudged" grid="ne0CONUSne30x8_ne0CONUSne30x8_mt12" name="SMS_D_Ln9_Vnuopc_P720x1" testmods="cam/outfrq9s">


Can we add a new <option name="comment" > entry for this test that describes in words what this is testing?

@nusbaume - I just changed the processor count and changed the testmods as @fvitt suggested in #708. I do not know what this is testing exactly as I did not introduce the test. I would suggest that test descriptions be added when the testing list is updated?

@fvitt can you provide a short one sentence description of what model configuration this is testing?

@cacraigucar Although I imagine we will certainly try to add in test descriptions when we do the test list update, adding descriptions in when we can beforehand will make that (big) job a little easier, and will save us from pinging Francis and other people a bunch of times when that development is ongoing.

Nudged CAM-Chem on a regionally refined grid

nusbaume · 2022-12-06T19:51:35Z

cime_config/testdefs/testlist_cam.xml

@@ -1622,7 +1622,7 @@
      <machine name="cheyenne" compiler="intel" category="camchem"/>
    </machines>
  </test>
-  <test compset="FCHIST" grid="ne0CONUSne30x8_ne0CONUSne30x8_mt12" name="SMS_D_Ln9_Vnuopc" testmods="cam/outfrq9s_refined_camchem">
+  <test compset="FCHIST" grid="ne0CONUSne30x8_ne0CONUSne30x8_mt12" name="SMS_D_Ln9_Vnuopc_P720x1" testmods="cam/outfrq9s">


Can we add a new <option name="comment" > entry for this test that describes in words what this is testing?

Free running CAM-Chem on a regionally refined grid

src/physics/cam/micro_pumas_cam.F90

src/physics/cam_dev/micro_pumas_cam.F90

…viewer's comments

cacraigucar added 9 commits May 11, 2022 16:47

rough prototype for DDT

d9d3327

Remove MG1 and prototype DDT with a couple of fields

82df5e6

First major chunk of variables in ddt

9872156

Second chunk of variables in ddt

8a8e52a

Completed initial list for DDT

6666fb6

Added comment to clarify hole filling pbuf fields

5e86e64

Minor cleanup

c2aac05

Remove commit of cam/micro_pumas_cam.F90 which should not have been made

25a9e2e

Update external to temporary PUMAS hash

1cdc45b

cacraigucar added enhancement New feature or request BFB bit for bit tag labels Aug 3, 2022

cacraigucar added this to the CESM2.3 milestone Aug 3, 2022

cacraigucar requested review from andrewgettelman and Katetc August 3, 2022 23:10

cacraigucar self-assigned this Aug 3, 2022

cacraigucar marked this pull request as draft August 3, 2022 23:13

andrewgettelman approved these changes Aug 8, 2022

View reviewed changes

andrewgettelman reviewed Aug 8, 2022

View reviewed changes

src/physics/cam_dev/micro_pumas_cam.F90 Outdated Show resolved Hide resolved

Fix dimension for addfld for UMR and UMS

1753457

cacraigucar added 5 commits November 30, 2022 15:04

Address reviewer's comments

f6c7ca8

Merge remote-tracking branch 'origin/cam_fix_init_cam_dev' into cam6_…

8f670f9

…pumas_ddt

Merge remote-tracking branch 'jim/config_changes' into cam6_pumas_ddt

e701690

Merge remote-tracking branch 'jim/orb_params_print_issue' into cam6_p…

b243853

…umas_ddt

Reduce number of PEs for CONUS regression tests

30dded9

cacraigucar requested a review from peverwhee December 1, 2022 18:03

fvitt removed their request for review December 1, 2022 18:11

cacraigucar and others added 6 commits December 1, 2022 11:45

add a couple of parameters

ceb57eb

Merge tag 'cam6_3_085' into cam6_pumas_ddt

0a03410

Merge pull request ESCOMP#700 from peverwhee/update_externals

Initializing variables to avoid startup interpolation error in cam_dev.

8407498

Merge pull request #15 from bstephens82/cam_fix_init_cam_dev

6358b7d

Initializing variables to avoid startup interpolation error in cam_dev.

Merge remote-tracking branch 'origin/cam_fix_init_cam_dev' into cam6_…

8431a93

…pumas_ddt

Update cime to fix create_newcase

960b294

Katetc requested changes Dec 6, 2022

View reviewed changes

nusbaume approved these changes Dec 6, 2022

View reviewed changes

nusbaume mentioned this pull request Dec 6, 2022

Output PUMAS variables on trop_cld_lev when possible #715

Open

cacraigucar requested a review from Katetc December 7, 2022 22:15

cacraigucar added 2 commits December 7, 2022 15:16

Update mosart external to work with updated cime and address final re…

259ac9a

…viewer's comments

Fix allocation problem

bed1694

Katetc approved these changes Dec 8, 2022

View reviewed changes

Add ChangeLog for cam6_3_086

ee94a75

cacraigucar merged commit 3d31917 into ESCOMP:cam_development Dec 9, 2022

cacraigucar deleted the cam6_pumas_ddt branch December 9, 2022 16:11

This was referenced Dec 9, 2022

Introduce process rate DDT for PUMAS #589 #589

Closed

create_newcase using updated cime does not start jobs on izumi #711

Closed

interpolation error on initialization in cam_dev #701

Closed

nvhpc compiler issue #705

Closed

cacraigucar changed the title ~~Introduce PUMAS DDT~~ cam6_3_086: Introduce PUMAS DDT Dec 9, 2022

cacraigucar mentioned this pull request Jan 2, 2024

Remove MG1 from cam_dev physics #611

Closed

cacraigucar mentioned this pull request Apr 22, 2024

Crash when running cam_dev with SILHS #844

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cam6_3_086: Introduce PUMAS DDT #632

cam6_3_086: Introduce PUMAS DDT #632

cacraigucar commented Aug 3, 2022 •

edited

Loading

cacraigucar commented Aug 3, 2022

andrewgettelman left a comment

cacraigucar commented Aug 8, 2022

sjsprecious commented Aug 9, 2022

cacraigucar commented Aug 9, 2022

andrewgettelman commented Aug 9, 2022

Katetc commented Aug 9, 2022

sjsprecious commented Aug 9, 2022

andrewgettelman commented Aug 9, 2022

cacraigucar commented Aug 9, 2022

sjsprecious commented Aug 9, 2022

andrewgettelman commented Aug 10, 2022

Katetc left a comment

nusbaume left a comment

nusbaume Dec 6, 2022

cacraigucar Dec 7, 2022

nusbaume Dec 7, 2022

fvitt Dec 7, 2022

nusbaume Dec 6, 2022

fvitt Dec 7, 2022

cam6_3_086: Introduce PUMAS DDT #632

cam6_3_086: Introduce PUMAS DDT #632

Conversation

cacraigucar commented Aug 3, 2022 • edited Loading

cacraigucar commented Aug 3, 2022

andrewgettelman left a comment

Choose a reason for hiding this comment

cacraigucar commented Aug 8, 2022

sjsprecious commented Aug 9, 2022

cacraigucar commented Aug 9, 2022

andrewgettelman commented Aug 9, 2022

Katetc commented Aug 9, 2022

sjsprecious commented Aug 9, 2022

andrewgettelman commented Aug 9, 2022

cacraigucar commented Aug 9, 2022

sjsprecious commented Aug 9, 2022

andrewgettelman commented Aug 10, 2022

Katetc left a comment

Choose a reason for hiding this comment

nusbaume left a comment

Choose a reason for hiding this comment

nusbaume Dec 6, 2022

Choose a reason for hiding this comment

cacraigucar Dec 7, 2022

Choose a reason for hiding this comment

nusbaume Dec 7, 2022

Choose a reason for hiding this comment

fvitt Dec 7, 2022

Choose a reason for hiding this comment

nusbaume Dec 6, 2022

Choose a reason for hiding this comment

fvitt Dec 7, 2022

Choose a reason for hiding this comment

cacraigucar commented Aug 3, 2022 •

edited

Loading