Conversation
…, fix for start_lpjml to not close coupled program to early, deprecation warning function
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #17 +/- ##
==========================================
+ Coverage 78.47% 81.43% +2.96%
==========================================
Files 7 7
Lines 1640 1988 +348
==========================================
+ Hits 1287 1619 +332
- Misses 353 369 +16 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR updates pycoupler for compatibility with newer pycopanlpjml/LPJmL coupling expectations, including a new default coupled model name, improved Slurm coupled-job handling, additional port/process cleanup, and expanded test coverage for utilities, run submission, and NetCDF export helpers.
Changes:
- Default coupled model name updated to
copan, with support for overriding via a coupled config. - Slurm coupled-job submission now patches
slurm.jcfto ensure the coupler process is waited on (and submits viasbatchwhen needed). - Adds NetCDF writing + grid transform utilities for
LPJmLData/LPJmLDataSet, plus substantial new tests.
Reviewed changes
Copilot reviewed 16 out of 17 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
pycoupler/config.py |
Adjusts coupled config defaults/behavior (model default + coupled config support). |
pycoupler/coupler.py |
Adds port cleanup utilities; fixes historic years handling; improves input sending conversion logic. |
pycoupler/run.py |
Introduces start_lpjml and deprecated alias; patches Slurm submission flow for coupled runs. |
pycoupler/utils.py |
Adds standardized deprecation warning helper. |
pycoupler/data.py |
Implements transform() and NetCDF export helpers for LPJmL data structures. |
tests/test_config.py |
Updates expectations for new default coupled model. |
tests/test_couple.py |
Adjusts time assignment and historic years expectation in coupling test. |
tests/test_run.py |
Extends Slurm submission tests to cover slurm.jcf patching and sbatch flow. |
tests/test_run_additional.py |
Adds additional run-related tests incl. deprecated alias warning behavior. |
tests/test_utils.py |
Extends detect_io_type test coverage for invalid UTF-8 bytes. |
tests/test_utils_additional.py |
Adds coverage for create_subdirs and read_json. |
tests/test_data.py |
Adds comprehensive coverage for grid transforms and NetCDF writing. |
tests/test_coupler_utils.py |
Adds coverage for new port cleanup utilities. |
tests/data/invalid_utf8.bin |
Adds a sample binary file (currently appears redundant with the updated test). |
pycoupler/release.py |
Clarifies release script instructions. |
pycoupler/__init__.py |
Minor formatting cleanup. |
CITATION.cff |
Bumps version to 1.7.0. |
Comments suppressed due to low confidence (1)
pycoupler/run.py:265
slurm_jcf_diris documented/used as the directory whereslurm.jcfshould be written, but thelpjsubmitsubprocess is still executed withoutcwd=.... Iflpjsubmitwritesslurm.jcfto its working directory (typical),_patch_slurm_and_submit()will look inslurm_jcf_dir/slurm.jcfand fail even though the file exists elsewhere. Consider ensuring the directory exists and runninglpjsubmitwithcwd=slurm_jcf_dir(or otherwise directinglpjsubmitoutput) soslurm_jcf_pathmatches where the file is actually created.
# run in coupled mode and pass coupling program/model
needs_coupler_wait = bool(couple_to)
if slurm_jcf_dir is None:
slurm_jcf_dir = os.getcwd()
slurm_jcf_path = Path(slurm_jcf_dir) / "slurm.jcf"
couple_file = None
if couple_to:
python_path = "python3"
if venv_path:
python_path = os.path.join(venv_path, "bin/python")
if not os.path.isfile(python_path):
raise FileNotFoundError(
f"venv path contains no python binary at '{python_path}'."
)
bash_script = f"""#!/bin/bash
# Define the path to the config file
config_file="{config_file}"
# Call the Python script with the config file as an argument
{python_path} {couple_to} $config_file
"""
couple_file = f"{output_path}/copan_lpjml.sh"
with open(couple_file, "w") as file:
file.write(bash_script)
# Change the permissions of the file to make it executable
run(["chmod", "+x", couple_file])
cmd.extend(["-couple", couple_file])
if needs_coupler_wait:
cmd.append("-norun")
cmd.extend([str(ntasks), config_file])
# Intialize submit_status in higher scope
submit_status = None
# set LPJROOT to model_path to be able to call lpjsubmit
try:
os.environ["LPJROOT"] = config.model_path
# call lpjsubmit via subprocess and return status if successfull
submit_status = run(cmd, capture_output=True)
except Exception as e:
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This MR introduces new features for the compatibility with the latest pycopanlpjml version: