Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modify LSFEnvironment to use more reliable environment variable #10825

Merged
merged 46 commits into from
Jan 5, 2022
Merged

Modify LSFEnvironment to use more reliable environment variable #10825

merged 46 commits into from
Jan 5, 2022

Commits on Nov 29, 2021

  1. Update LSFEnvironment 

    - Use LSB_DJOB_RANKFILE to determine node rank and master address
- Update tests
    @ajtritt
    ajtritt committed Nov 29, 2021
    Configuration menu
    Copy the full SHA
    8e3d633 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Nov 29, 2021
    Configuration menu
    Copy the full SHA
    f970ee4 View commit details
    Browse the repository at this point in the history

  3. Cleanup for review 

    - Update CHANGELOG
- Fix PEP8 violations
    @ajtritt
    ajtritt committed Nov 29, 2021
    Configuration menu
    Copy the full SHA
    d0b6796 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2021

  1. Work out kinks 

    - Change master_address and master_port to main_address and main_port
- Fix errors in mocks and tests
    @ajtritt
    ajtritt committed Nov 30, 2021
    Configuration menu
    Copy the full SHA
    480c23b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'bugfix/lsf_env' of github.com:ajtritt/pytorch-lightning… 

    … into bugfix/lsf_env
    @ajtritt
    ajtritt committed Nov 30, 2021
    Configuration menu
    Copy the full SHA
    cd45898 View commit details
    Browse the repository at this point in the history

  3. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Nov 30, 2021
    Configuration menu
    Copy the full SHA
    d9440ce View commit details
    Browse the repository at this point in the history

  4. Fix deprecation test

    @ajtritt
    ajtritt committed Nov 30, 2021
    Configuration menu
    Copy the full SHA
    ba3a4a8 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'bugfix/lsf_env' of github.com:ajtritt/pytorch-lightning… 

    … into bugfix/lsf_env
    @ajtritt
    ajtritt committed Nov 30, 2021
    Configuration menu
    Copy the full SHA
    ae740ed View commit details
    Browse the repository at this point in the history

  6. Remove is_using_lsf from deprecated tests 

    - this method never existed on ClusterEnvironment
    @ajtritt
    ajtritt committed Nov 30, 2021
    Configuration menu
    Copy the full SHA
    8180e15 View commit details
    Browse the repository at this point in the history

  7. fix pep8 errors

    @ajtritt
    ajtritt committed Nov 30, 2021
    Configuration menu
    Copy the full SHA
    87a8234 View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2021

  1. Add is_using_lsf deprecation warning

    @ajtritt
    ajtritt committed Dec 1, 2021
    Configuration menu
    Copy the full SHA
    4d376c0 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'bugfix/lsf_env' of github.com:ajtritt/pytorch-lightning… 

    … into bugfix/lsf_env
    @ajtritt
    ajtritt committed Dec 1, 2021
    Configuration menu
    Copy the full SHA
    2302c5c View commit details
    Browse the repository at this point in the history

  3. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Dec 1, 2021
    Configuration menu
    Copy the full SHA
    2f90f13 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2021

  1. Update pytorch_lightning/plugins/environments/lsf_environment.py 

    Co-authored-by: Andrew Tritt <ajtritt@lbl.gov>
    @tchaton @ajtritt
    tchaton and ajtritt authored Dec 6, 2021
    Configuration menu
    Copy the full SHA
    b610ebe View commit details
    Browse the repository at this point in the history

  2. Apply suggestions from code review 

    Apply suggestions from @tchaton and add comments where requested

Co-authored-by: thomas chaton <thomas@grid.ai>
    @ajtritt @tchaton
    ajtritt and tchaton authored Dec 6, 2021
    Configuration menu
    Copy the full SHA
    a7e2638 View commit details
    Browse the repository at this point in the history

  3. Two things 

    - Reorder methods to better match what existed
- Remove check for 'batch' prefix in launch node names. Rely on position now.
    @ajtritt
    ajtritt committed Dec 6, 2021
    Configuration menu
    Copy the full SHA
    f2dd684 View commit details
    Browse the repository at this point in the history

  4. Clean up copy-paste garbage format docstrings

    @ajtritt
    ajtritt committed Dec 6, 2021
    Configuration menu
    Copy the full SHA
    bafce28 View commit details
    Browse the repository at this point in the history

Commits on Dec 7, 2021

  1. Update pytorch_lightning/plugins/environments/lsf_environment.py 

    Co-authored-by: thomas chaton <thomas@grid.ai>
    @ajtritt @tchaton
    ajtritt and tchaton authored Dec 7, 2021
    Configuration menu
    Copy the full SHA
    3782d95 View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2021

  1. Merge branch 'master' into bugfix/lsf_env

    @ajtritt
    ajtritt authored Dec 8, 2021
    Configuration menu
    Copy the full SHA
    93a5446 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Dec 8, 2021
    Configuration menu
    Copy the full SHA
    e261a2c View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2021

  1. Fix PEP8 errors and mypy errors

    @ajtritt
    ajtritt committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    e243b9d View commit details
    Browse the repository at this point in the history

  2. Fix mypy errors

    @ajtritt
    ajtritt committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    018b8ca View commit details
    Browse the repository at this point in the history

  3. add missing import

    @ajtritt
    ajtritt committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    5b03832 View commit details
    Browse the repository at this point in the history

  4. Move port type conversion

    @ajtritt
    ajtritt committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    a5b5c46 View commit details
    Browse the repository at this point in the history

  5. Fix mypy errors

    @ajtritt
    ajtritt committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    2f2c944 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' into bugfix/lsf_env

    @ajtritt
    ajtritt authored Dec 9, 2021
    Configuration menu
    Copy the full SHA
    10c706e View commit details
    Browse the repository at this point in the history

  7. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Dec 9, 2021
    Configuration menu
    Copy the full SHA
    a25179f View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2021

  1. Merge branch 'master' into bugfix/lsf_env

    @carmocca
    carmocca authored Dec 13, 2021
    Configuration menu
    Copy the full SHA
    f920a52 View commit details
    Browse the repository at this point in the history

  2. Apply suggestions from code review 

    Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
    @ajtritt @carmocca
    ajtritt and carmocca authored Dec 13, 2021
    Configuration menu
    Copy the full SHA
    8b536fd View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2021

  1. merge master

    @awaelchli
    awaelchli committed Dec 14, 2021
    Configuration menu
    Copy the full SHA
    5d2ca92 View commit details
    Browse the repository at this point in the history

  2. Update tests/plugins/environments/test_lsf_environment.py 

    Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
    @ajtritt @awaelchli
    ajtritt and awaelchli authored Dec 14, 2021
    Configuration menu
    Copy the full SHA
    9987ae3 View commit details
    Browse the repository at this point in the history

Commits on Dec 17, 2021

  1. Merge branch 'master' into bugfix/lsf_env

    @tchaton
    tchaton authored Dec 17, 2021
    Configuration menu
    Copy the full SHA
    34dd9f9 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into bugfix/lsf_env

    @Borda
    Borda authored Dec 17, 2021
    Configuration menu
    Copy the full SHA
    920a2e5 View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2021

  1. minimize diff 

    reduce diff
update tests
    @awaelchli
    awaelchli committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    1907f92 View commit details
    Browse the repository at this point in the history

  2. minimize diff 

    reduce diff
update tests
fix typo
    @awaelchli
    awaelchli committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    2e02660 View commit details
    Browse the repository at this point in the history

  3. reuse make_rank_file

    @awaelchli
    awaelchli committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    e22cc4d View commit details
    Browse the repository at this point in the history

  4. add missing property

    @awaelchli
    awaelchli committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    5198f7a View commit details
    Browse the repository at this point in the history

  5. update docstrings

    @awaelchli
    awaelchli committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    7e39a34 View commit details
    Browse the repository at this point in the history

  6. compress message

    @awaelchli
    awaelchli committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    943fcaa View commit details
    Browse the repository at this point in the history

  7. merge

    @awaelchli
    awaelchli committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    71314b9 View commit details
    Browse the repository at this point in the history

  8. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Dec 20, 2021
    Configuration menu
    Copy the full SHA
    45fb87c View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2022

  1. drop string representation

    @awaelchli
    awaelchli committed Jan 5, 2022
    Configuration menu
    Copy the full SHA
    a1d5f49 View commit details
    Browse the repository at this point in the history

  2. move environment variable lookup to property getters

    @awaelchli
    awaelchli committed Jan 5, 2022
    Configuration menu
    Copy the full SHA
    dd88292 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into bugfix/lsf_env

    @awaelchli
    awaelchli committed Jan 5, 2022
    Configuration menu
    Copy the full SHA
    d156ffa View commit details
    Browse the repository at this point in the history

  4. use fsspec io to open file

    @awaelchli
    awaelchli committed Jan 5, 2022
    Configuration menu
    Copy the full SHA
    9888945 View commit details
    Browse the repository at this point in the history

  5. set read mode for fs.open

    @awaelchli
    awaelchli committed Jan 5, 2022
    Configuration menu
    Copy the full SHA
    630dd6f View commit details
    Browse the repository at this point in the history