Adding version-specific run.ver files and also removing ccpa directories

[PerryShafran-NOAA]

Pull Request Testing

• Describe testing already performed for this Pull Request:
  

The global_ens has been running with a run.ver (run.ver.metplus4.1.4) that refers to older versions of MET/METplus. That works fine.
Developers will need to update run.ver file for the METplusv5.0.1 with the EVS Virtual Environment (VE).
The version run.ver.metplus5.0.0 is identical to the current run.ver file in the repository for the current versions running in the parallel.

Also removes the ccpa directories from the repository.

• Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
  

No need to run anything here. This makes available all versions of run.ver in the version directory.

• Has the code been checked to ensure that no errors occur during the execution? [Yes or No]

• Do these updates/additions include sufficient testing updates? [Yes or No]

• Please complete this pull request review by [Fill in date].
  

Complete by 4/28/2023.

Pull Request Checklist

• Review the source issue metadata (required labels, projects, and milestone).

• Complete the PR description above.

• Ensure the PR title matches the feature branch name.

• Check the following:

• Instructions provided on how to run

• Developer's name is replaced by ${user} where necessary throughout the code

• Check that the ecf file has all the proper definitions of variables

• Check that the jobs file has all the proper settings of COMIN and COMOUT and other input variables

• Check to see that the output directory structure is followed

• Be sure that you are not using MET utilities outside the METplus wrapper structure

• After submitting the PR, select Development issue with the original issue number.

• After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.

• Close the linked issue.

[ShelleyMelchior-NOAA]

Should this PR also update run.ver to use python ve's met and metplus versions?

[PerryShafran-NOAA]

Probably? I haven't tested the VE yet myself, so I would guess that the developer would make that change. I could do it in this PR if I could see an example run.ver that successfully tested the VE.

[LoganDawson-NOAA]

@ShelleyMelchior-NOAA is correct. The run.ver file should include these two updates:

export met_ver=11.0.1
export metplus_ver=5.0.1

Developers will make some updates to the list of module loads in their ecf scripts, but they shouldn't have to update run.ver to use the specific versions that will be final for EVSv1.

[ShelleyMelchior-NOAA]

Valid point. @OliviaOstwald-NOAA , can you point Perry to your run.ver that you successfully modified for met-11.0.1 and metplus-5.0.1?
@PerryShafran-NOAA , I think the only changes to run.ver are for met and metplus. The other changes for loading the python ve and the various other module load modifications are all contained in the ecf scripts. If it helps, I threw this together as a guide for Olivia: https://docs.google.com/document/d/1Fk5aRw05SdLSBZ491llDLU0XTiO0cXd-_Nf3riiTVv4/edit?usp=sharing

[OliviaOstwald-NOAA]

/lfs/h2/emc/vpppg/noscrub/olivia.ostwald/Fork/EVS/versions/run.ver

[PerryShafran-NOAA]

Updated run.ver to use the newest versions. I didn't update the python version - should I?

[ShelleyMelchior-NOAA]

I think ultimately the python_ver doesn't need to be in the run.ver, but it doesn't hurt being there. I'll let Logan be that authority.

[AliciaBentley-NOAA]

Hi all. I checked and export ve_evs_ver=1.0 is already included in run.ver, so that looks good. Python still says export python_ver=3.8.6, which will need to be updated to 3.10.4. Thanks!

…

___________________________________ *Alicia M. Bentley, Ph.D.* Global Verification Project Lead Model Evaluation Group (MEG) <https://www.emc.ncep.noaa.gov/users/meg/home/> NOAA/NWS/NCEP/EMC (VPPPG Branch) Work Phone: ‪(301) 683-3783‬

On Mon, Apr 24, 2023 at 2:07 PM PerryShafran-NOAA ***@***.***> wrote: Updated run.ver to use the newest versions. I didn't update the python version - should I? — Reply to this email directly, view it on GitHub <#103 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APUWOBFBXRE6JL7LYZJAM5TXC26PHANCNFSM6AAAAAAXJ4F6FI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[AliciaBentley-NOAA]

I request that we keep python_ver in run.ver and update it to 3.10.4. Developers might be loading python/3.10.4 to produce various EVS plots (where loading MET/METplus may not be required).

…

___________________________________ *Alicia M. Bentley, Ph.D.* Global Verification Project Lead Model Evaluation Group (MEG) <https://www.emc.ncep.noaa.gov/users/meg/home/> NOAA/NWS/NCEP/EMC (VPPPG Branch) Work Phone: ‪(301) 683-3783‬

On Mon, Apr 24, 2023 at 2:10 PM Shelley Melchior ***@***.***> wrote: I think ultimately the python_ver doesn't need to be in the run.ver, but it doesn't hurt being there. I'll let Logan be that authority. — Reply to this email directly, view it on GitHub <#103 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APUWOBBY5QZCJ55E32CL7N3XC26ZZANCNFSM6AAAAAAXJ4F6FI> . You are receiving this because you commented.Message ID: ***@***.***>

[LoganDawson-NOAA]

Updating the python_ver also works, but it's technically not needed. As far as I know, it will never be used in a module load statement.

When ve/evs/${ve_evs_ver} is loaded, the python binary executable is sim linked to the binary executable here:

[logan.dawson@clogin05 versions] llh /apps/dev/ve/evs/1.0/bin/python
lrwxrwxrwx 1 hpc-user hpc-user 90 Nov  3 15:21 /apps/dev/ve/evs/1.0/bin/python -> /apps/spack/python/3.10.4/intel/19.1.3.304/xqft4d45h4dp4xnbz2ue3nbxv65i6bgp/bin/python3.10

[PerryShafran-NOAA]

Python version is changed in the run.ver.

They are kept the same (3.8.6) in the 4.1.4 and 5.0.0 versions though.

[LoganDawson-NOAA]

We should delete python_ver from the final run.ver file. If we include it and someone directly loads python/3.10.4, they will get a version of python that is missing critical packages like Cartopy, geos, and pandas that will be needed to run the plotting jobs. See the email I sent separately for lists of the difference in python package availability when loading ve/evs/1.0 vs loading python/3.10.4.

We should delete python_ver so there's no confusion about how python should be loaded in our final jobs.

[PerryShafran-NOAA]

Sounds good - I deleted python_ver from the final run.ver file. It remains in the 5.0.0 and 4.1.4 versions.

Perry

[ShelleyMelchior-NOAA]

Removal of ccpa directories looks good.
Updates to run.ver look correct.
Additional run.ver* files look good.