Run gen_yaml.sh (building a super JEDI YAML)

Background Info

The gen_yaml.sh script is designed to concatenate individual JEDI obspace YAML files. Currently, for efficient file management and streamlined development, these YAML files are divided into components based on subsets (e.g., aircar, aircft, msonet) and observation types (e.g., airTemperature, specificHumidity). While this approach results in a significant amount of redundant text across files, it greatly simplifies development in the long run, making it easier to detect bugs and create new YAML configurations. For instance, using vi -d aircft_airTemperature_130.yaml msonet_airTemperature_188.yaml, you’ll find only about 13 lines that differ out of 140, highlighting how minimal the variations are between files.

aircar_airTemperature_133.yaml
aircar_specificHumidity_133.yaml
aircar_uv_233.yaml
aircft_airTemperature_130.yaml
aircft_airTemperature_131.yaml
aircft_airTemperature_134.yaml
aircft_airTemperature_135.yaml
aircft_specificHumidity_134.yaml
aircft_uv_230.yaml
aircft_uv_231.yaml
aircft_uv_234.yaml
aircft_uv_235.yaml
msonet_airTemperature_188.yaml
msonet_specificHumidity_188.yaml
msonet_stationPressure_188.yaml
msonet_uv_288.yaml

Running the script

cd RDASApp/rrfs-test/validated_yamls
./gen_yaml.sh

And that's it! It will create an intermediate yaml (temp.yaml), but the main yaml of interest is the conv.yaml.

Optional configuration

You may be wondering how to build the yaml with only one subset and one obtype. For example, let's use aircft_airTemperature_130.yaml. There are just two simple steps:

1. Comment (#) all the other *.yaml filenames in the aircft_obtype_configs.

# Define the aircft observation type configs as an array
aircft_obtype_configs=(
    "aircft_airTemperature_130.yaml"
    #"aircft_airTemperature_131.yaml"
    #"aircft_airTemperature_134.yaml"
    #"aircft_airTemperature_135.yaml"
    #"aircft_specificHumidity_134.yaml"
    #"aircft_uv_230.yaml"
    #"aircft_uv_231.yaml"
    #"aircft_uv_234.yaml"
    #"aircft_uv_235.yaml"
)

2. Comment out the unwanted process_obtypes:

# Concatenate all obtypes into the super yaml
#process_obtypes "aircar_obtype_configs[@]" "data/obs/ioda_aircar_dc.nc" "$temp_yaml"
process_obtypes "aircft_obtype_configs[@]" "data/obs/ioda_aircft_dc.nc" "$temp_yaml"
#process_obtypes "msonet_obtype_configs[@]" "data/obs/ioda_msonet_dc.nc" "$temp_yaml"

Now the script is configured to only process the aircft yamls and there is only aircft_airTemperature_130.yaml in that list.

Adding Additional Obs

Simply add the name of your yaml to the obtype_configs array or create a new array if the array for your subset doesn't already exist. Add the process_obtypes line below since each subset will have it's own ioda file if the process line doesn't already exist for your subset type.