Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update DDP for torch.distributed.run with gloo backend #3680

Merged
merged 35 commits into from
Jun 19, 2021
Merged

Update DDP for torch.distributed.run with gloo backend #3680

merged 35 commits into from
Jun 19, 2021

Commits on Jun 18, 2021

  1. Update DDP for torch.distributed.run

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    007902e View commit details
    Browse the repository at this point in the history

  2. Add LOCAL_RANK

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    9bcb4ad View commit details
    Browse the repository at this point in the history

  3. remove opt.local_rank

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    b32bae0 View commit details
    Browse the repository at this point in the history

  4. backend="gloo|nccl"

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    b467501 View commit details
    Browse the repository at this point in the history

  5. print

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    c886538 View commit details
    Browse the repository at this point in the history

  6. print

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    5d847dc View commit details
    Browse the repository at this point in the history

  7. debug

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    26d0ecf View commit details
    Browse the repository at this point in the history

  8. debug

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    832ba4c View commit details
    Browse the repository at this point in the history

  9. os.getenv

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    9a1bb01 View commit details
    Browse the repository at this point in the history

  10. gloo

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    0e912df View commit details
    Browse the repository at this point in the history

  11. gloo

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    5f5e428 View commit details
    Browse the repository at this point in the history

  12. gloo

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    e8493c6 View commit details
    Browse the repository at this point in the history

  13. cleanup

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    fb342fc View commit details
    Browse the repository at this point in the history

  14. fix getenv

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    1 Configuration menu
    Copy the full SHA
    382ce4f View commit details
    Browse the repository at this point in the history

  15. cleanup

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    b09b415 View commit details
    Browse the repository at this point in the history

  16. cleanup destroy

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    9c4ac05 View commit details
    Browse the repository at this point in the history

  17. try nccl

    @glenn-jocher
    glenn-jocher committed Jun 18, 2021
    Configuration menu
    Copy the full SHA
    8ae9ea1 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2021

  1. merge master

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    a18f933 View commit details
    Browse the repository at this point in the history

  2. return opt

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    2435775 View commit details
    Browse the repository at this point in the history

  3. add --local_rank

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    56a4ab4 View commit details
    Browse the repository at this point in the history

  4. add timeout

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    c4d839b View commit details
    Browse the repository at this point in the history

  5. add init_method

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    0584e7e View commit details
    Browse the repository at this point in the history

  6. gloo

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    d917341 View commit details
    Browse the repository at this point in the history

  7. move destroy

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    6a1cc64 View commit details
    Browse the repository at this point in the history

  8. move destroy

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    3581c76 View commit details
    Browse the repository at this point in the history

  9. move print(opt) under if RANK

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    5f5d122 View commit details
    Browse the repository at this point in the history

  10. destroy only RANK 0

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    5451fc2 View commit details
    Browse the repository at this point in the history

  11. move destroy inside train()

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    9aa229e View commit details
    Browse the repository at this point in the history

  12. restore destroy outside train()

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    94363ce View commit details
    Browse the repository at this point in the history

  13. update print(opt)

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    9647379 View commit details
    Browse the repository at this point in the history

  14. merge master

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    cb8395d View commit details
    Browse the repository at this point in the history

  15. cleanup

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    96686fd View commit details
    Browse the repository at this point in the history

  16. nccl

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    446c610 View commit details
    Browse the repository at this point in the history

  17. gloo with 60 second timeout

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    49bb0b7 View commit details
    Browse the repository at this point in the history

  18. update namespace printing

    @glenn-jocher
    glenn-jocher committed Jun 19, 2021
    Configuration menu
    Copy the full SHA
    b5decde View commit details
    Browse the repository at this point in the history