Skip to content

update for pycopanlpjml #17

Open
jnnsbrr wants to merge 17 commits intomainfrom
docs
Open

update for pycopanlpjml #17
jnnsbrr wants to merge 17 commits intomainfrom
docs

Conversation

@jnnsbrr
Copy link
Collaborator

@jnnsbrr jnnsbrr commented Feb 5, 2026

This MR introduces new features for the compatibility with the latest pycopanlpjml version:

  • default model name set to copan, instead of copan:CORE - if provided via CoupledConfig it can be set
  • functionality to kill remaining processes on coupled port to avoid blocked port in the next simulation
  • debug historic years reading for coupling
  • adjust slurm.jcf file creation for older LPJmL versions to avoid slurm ending the job before coupled program has finished
  • deprecation warning for old objects names, attributes

@codecov
Copy link

codecov bot commented Feb 6, 2026

Codecov Report

❌ Patch coverage is 82.87037% with 74 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.43%. Comparing base (9490c5e) to head (bbf7b67).

Files with missing lines Patch % Lines
pycoupler/coupler.py 70.65% 27 Missing ⚠️
pycoupler/data.py 87.36% 24 Missing ⚠️
pycoupler/run.py 85.71% 16 Missing ⚠️
pycoupler/config.py 65.00% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #17      +/-   ##
==========================================
+ Coverage   78.47%   81.43%   +2.96%     
==========================================
  Files           7        7              
  Lines        1640     1988     +348     
==========================================
+ Hits         1287     1619     +332     
- Misses        353      369      +16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jnnsbrr jnnsbrr requested review from Copilot and zner0L and removed request for Copilot February 6, 2026 12:41
@jnnsbrr jnnsbrr changed the title pycopanlpjml update update for pycopanlpjml Feb 6, 2026
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates pycoupler for compatibility with newer pycopanlpjml/LPJmL coupling expectations, including a new default coupled model name, improved Slurm coupled-job handling, additional port/process cleanup, and expanded test coverage for utilities, run submission, and NetCDF export helpers.

Changes:

  • Default coupled model name updated to copan, with support for overriding via a coupled config.
  • Slurm coupled-job submission now patches slurm.jcf to ensure the coupler process is waited on (and submits via sbatch when needed).
  • Adds NetCDF writing + grid transform utilities for LPJmLData/LPJmLDataSet, plus substantial new tests.

Reviewed changes

Copilot reviewed 16 out of 17 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
pycoupler/config.py Adjusts coupled config defaults/behavior (model default + coupled config support).
pycoupler/coupler.py Adds port cleanup utilities; fixes historic years handling; improves input sending conversion logic.
pycoupler/run.py Introduces start_lpjml and deprecated alias; patches Slurm submission flow for coupled runs.
pycoupler/utils.py Adds standardized deprecation warning helper.
pycoupler/data.py Implements transform() and NetCDF export helpers for LPJmL data structures.
tests/test_config.py Updates expectations for new default coupled model.
tests/test_couple.py Adjusts time assignment and historic years expectation in coupling test.
tests/test_run.py Extends Slurm submission tests to cover slurm.jcf patching and sbatch flow.
tests/test_run_additional.py Adds additional run-related tests incl. deprecated alias warning behavior.
tests/test_utils.py Extends detect_io_type test coverage for invalid UTF-8 bytes.
tests/test_utils_additional.py Adds coverage for create_subdirs and read_json.
tests/test_data.py Adds comprehensive coverage for grid transforms and NetCDF writing.
tests/test_coupler_utils.py Adds coverage for new port cleanup utilities.
tests/data/invalid_utf8.bin Adds a sample binary file (currently appears redundant with the updated test).
pycoupler/release.py Clarifies release script instructions.
pycoupler/__init__.py Minor formatting cleanup.
CITATION.cff Bumps version to 1.7.0.
Comments suppressed due to low confidence (1)

pycoupler/run.py:265

  • slurm_jcf_dir is documented/used as the directory where slurm.jcf should be written, but the lpjsubmit subprocess is still executed without cwd=.... If lpjsubmit writes slurm.jcf to its working directory (typical), _patch_slurm_and_submit() will look in slurm_jcf_dir/slurm.jcf and fail even though the file exists elsewhere. Consider ensuring the directory exists and running lpjsubmit with cwd=slurm_jcf_dir (or otherwise directing lpjsubmit output) so slurm_jcf_path matches where the file is actually created.
    # run in coupled mode and pass coupling program/model
    needs_coupler_wait = bool(couple_to)
    if slurm_jcf_dir is None:
        slurm_jcf_dir = os.getcwd()
    slurm_jcf_path = Path(slurm_jcf_dir) / "slurm.jcf"

    couple_file = None

    if couple_to:
        python_path = "python3"
        if venv_path:
            python_path = os.path.join(venv_path, "bin/python")
            if not os.path.isfile(python_path):
                raise FileNotFoundError(
                    f"venv path contains no python binary at '{python_path}'."
                )

        bash_script = f"""#!/bin/bash

# Define the path to the config file
config_file="{config_file}"

# Call the Python script with the config file as an argument
{python_path} {couple_to} $config_file
"""

        couple_file = f"{output_path}/copan_lpjml.sh"

        with open(couple_file, "w") as file:
            file.write(bash_script)

        # Change the permissions of the file to make it executable
        run(["chmod", "+x", couple_file])

        cmd.extend(["-couple", couple_file])

    if needs_coupler_wait:
        cmd.append("-norun")

    cmd.extend([str(ntasks), config_file])

    # Intialize submit_status in higher scope
    submit_status = None
    # set LPJROOT to model_path to be able to call lpjsubmit
    try:
        os.environ["LPJROOT"] = config.model_path
        # call lpjsubmit via subprocess and return status if successfull
        submit_status = run(cmd, capture_output=True)
    except Exception as e:

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments