Skip to content

Conversation

@chengzhuzhang
Copy link
Collaborator

@chengzhuzhang chengzhuzhang commented Dec 16, 2025

Summary

This PR fixes a KeyError: 'year' that occurs when running original plots that don't include any atmospheric variables (e.g., plots_original = "change_ohc,max_moc,change_sea_level"). The error happens because the "year" key in exp["annual"] only gets populated as a side effect of processing atmospheric variables through the standard set_var() pipeline, but some plots rely solely on manually loaded ocean data (ohc, volume) that bypassed this mechanism.

Error log:

Traceback (most recent call last):
  File "/lcrc/soft/climate/e3sm-unified/e3smu_1_12_0/chrysalis/conda/envs/e3sm_unified_1.12.0_login/lib/python3.13/site-packages/zppy_interfaces/global_time_series/coupled_global/mode_pdf.py", line 77, in assemble_cumulative_pdf
    plot_function(ax, xlim, exps, rgn)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/lcrc/soft/climate/e3sm-unified/e3smu_1_12_0/chrysalis/conda/envs/e3sm_unified_1.12.0_login/lib/python3.13/site-packages/zppy_interfaces/global_time_series/coupled_global/mix_pdf_original.py", line 171, in plot_change_ohc
    plot(ax, xlim, exps, param_dict, rgn)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lcrc/soft/climate/e3sm-unified/e3smu_1_12_0/chrysalis/conda/envs/e3sm_unified_1.12.0_login/lib/python3.13/site-packages/zppy_interfaces/global_time_series/coupled_global/plotting.py", line 38, in plot
    year = np.array(exp["annual"]["year"]) + exp["yoffset"]
                    ~~~~~~~~~~~~~^^^^^^^^
KeyError: 'year'
2025-12-15 17:53:28,229 [ERROR]: mode_pdf.py(assemble_cumulative_pdf:82) >> Failed plot_function for change_ohc. Check that [] are available.
2025-12-15 17:53:28,229 [INFO]: mode_pdf.py(assemble_cumulative_pdf:59) >> Plotting plot 1 on page 0. This is plot 1 in total.
2025-12-15 17:53:28,233 [INFO]: mix_pdf_original.py(plot_max_moc:176) >> Plot 6: plot_max_moc
2025-12-15 17:53:28,234 [INFO]: plotting.py(getmoc:116) >> /lcrc/group/e3sm/public_html/diagnostic_output/ac.zhang40/tests/zi_no_atm_unified1.2.0/ocn/glb/ts/monthly/5yr/ 1 moc files in total
2025-12-15 17:53:28,405 [INFO]: plotting.py(add_trend:175) >> [ 0.14595268 -8.02897892]
2025-12-15 17:53:28,408 [INFO]: mode_pdf.py(assemble_cumulative_pdf:59) >> Plotting plot 2 on page 0. This is plot 2 in total.
2025-12-15 17:53:28,413 [INFO]: mix_pdf_original.py(plot_change_sea_level:205) >> Plot 7: plot_change_sea_level
Traceback (most recent call last):
  File "/lcrc/soft/climate/e3sm-unified/e3smu_1_12_0/chrysalis/conda/envs/e3sm_unified_1.12.0_login/lib/python3.13/site-packages/zppy_interfaces/global_time_series/coupled_global/mode_pdf.py", line 77, in assemble_cumulative_pdf
    plot_function(ax, xlim, exps, rgn)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/lcrc/soft/climate/e3sm-unified/e3smu_1_12_0/chrysalis/conda/envs/e3sm_unified_1.12.0_login/lib/python3.13/site-packages/zppy_interfaces/global_time_series/coupled_global/mix_pdf_original.py", line 233, in plot_change_sea_level
    plot(ax, xlim, exps, param_dict, rgn)
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lcrc/soft/climate/e3sm-unified/e3smu_1_12_0/chrysalis/conda/envs/e3sm_unified_1.12.0_login/lib/python3.13/site-packages/zppy_interfaces/global_time_series/coupled_global/plotting.py", line 38, in plot
    year = np.array(exp["annual"]["year"]) + exp["yoffset"]

Root Cause

The issue stems from an order dependency in how the "year" key gets populated:

  1. Standard workflow: Atmospheric variables (RESTOM, TREFHT, etc.) get processed via set_var(), which populates both the variable data AND the "year" key (lines 895-899 in utils.py)
  2. Manual loading: Ocean variables (ohc, volume) get loaded manually in plots_original.py without populating the year key
  3. Plotting failure: When plotting.py:38 tries to access exp["annual"]["year"], it fails if no atmospheric variables were processed first

What this PR changes

  • File: zppy_interfaces/global_time_series/coupled_global/plots_original.py
  • Lines 47-57: Added year key population after manual ohc loading
  • Lines 60-72: Added year key population after manual volume loading

Select one: This pull request is...

  • a bug fix: increment the patch version
  • a small improvement: increment the minor version
  • a new feature: increment the minor version
  • an incompatible (non-backwards compatible) API change: increment the major version

Small Change

  • To merge, I will use "Squash and merge". That is, this change should be a single commit.
  • Logic: I have visually inspected the entire pull request myself.
  • Pre-commit checks: All the pre-commits checks have passed.

@chengzhuzhang
Copy link
Collaborator Author

chengzhuzhang commented Dec 16, 2025

Testing script is posted here:

from zppy_interfaces.global_time_series.__main__ import main
import sys
import time

sys.argv.extend([
    "--use_ocn", "True",
    "--input", "case directory",
    "--input_subdir", "archive/ocn/hist",
    "--moc_file", "mocTimeSeries_0166-0175.nc",
    "--case_dir", "/gpfs/fs0/globalscratch/ac.zhang40/20251107.v3.HR.piControl-Spinup/",
    "--experiment_name", "20250906.v3-HR.piControl_test6.1",
    "--figstr", "20250906.wcycl1850.ne120pg2_r025_RRSwISC6to18E3r5.test6.1.chrysalis",
    "--color", "Blue",
    "--ts_num_years", "5",
    # Test the problematic plot combination that causes KeyError
    "--plots_original", "change_ohc,max_moc,change_sea_level",
    # Alternatively, test the full working sequence:
    #"--plots_original", "net_toa_flux_restom,global_surface_air_temperature,toa_radiation,net_atm_energy_imbalance,change_ohc,max_moc,change_sea_level,net_atm_water_imbalance",
    #"--plots_original", "None",
    "--plots_atm", "None",
    "--plots_ice", "None",
    "--plots_lnd", "None",
    #"--plots_lnd", "all",
    "--plots_ocn", "None",
    "--nrows", "4",
    "--ncols", "2",
    "--results_dir", "/lcrc/group/e3sm/public_html/diagnostic_output/ac.zhang40/tests/zi_no_atm_unified1.2.0",
    "--regions", "glb,n,s",
    "--make_viewer", "True",
    "--start_yr", "166",
    "--end_yr", "175"
])


start_time = time.time()
main()
end_time = time.time()

print(f"Execution time: {end_time - start_time:.2f} seconds")

With zppy-interfaces from unified 1.12.0, result as shown below:
image

With this PR fix, results as shown below:
image

Copy link
Collaborator

@forsyth2 forsyth2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chengzhuzhang Thanks, this looks right from visual inspection. This is associated with the zppy PR E3SM-Project/zppy#765, correct? I can test this change too when I do integration testing for that zppy PR.

@forsyth2
Copy link
Collaborator

forsyth2 commented Dec 16, 2025

associated with the zppy PR

Nevermind, I missed your comment here confirming that. Yes, I will do combined integration testing then.

@forsyth2 forsyth2 added the semver: bug Bug fix (will increment patch version) label Dec 17, 2025
Copy link
Collaborator

@forsyth2 forsyth2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving based on testing done on the zppy PR

@forsyth2 forsyth2 merged commit 7faec4b into main Dec 17, 2025
5 checks passed
@forsyth2 forsyth2 deleted the fix-missing-year-key-original-plots branch December 17, 2025 21:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

semver: bug Bug fix (will increment patch version)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants