Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DeepSpeed Communication Profiling and Logging #2012

Merged
merged 54 commits into from
Jul 25, 2022
Merged

DeepSpeed Communication Profiling and Logging #2012

Changes from 1 commit
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
867a853
Staging comms v1 (#301)
Quentin-Anthony May 27, 2022
c93fcfe
Delete stage1.py
awan-10 May 27, 2022
7f8ca01
Delete distributed.py
awan-10 May 27, 2022
977ee32
revert deepspeed/__init__.py logging calls
Quentin-Anthony May 28, 2022
68eb9f4
Delete test.py
Quentin-Anthony May 28, 2022
54796bb
Update comments and move custom comm ops to internal functions
Quentin-Anthony May 28, 2022
c06c72d
Merge branch 'staging-comms-next' of https://github.com/microsoft/Dee…
Quentin-Anthony May 28, 2022
f070a0c
Remove unnecessary print and update backend description
Quentin-Anthony May 28, 2022
9976681
Relax assertion to allow Megatron-DeepSpeed MoE to use ZeRO 1
Quentin-Anthony May 31, 2022
09063a3
Simplify ZeRO stage 1 check for previous commit
Quentin-Anthony May 31, 2022
656b415
Remove misleading world_size prints
Quentin-Anthony May 31, 2022
2e7129c
Add commslogger class, and introduce rough prototype comms logging
Quentin-Anthony Jun 1, 2022
0023b3e
Clean up logger
Quentin-Anthony Jun 1, 2022
e55c8e9
Add more robust arg checks
Quentin-Anthony Jun 3, 2022
31c7dcf
Add labels to common collective calls for logger
Quentin-Anthony Jun 3, 2022
8e23f50
Add more annotations
Quentin-Anthony Jun 3, 2022
7998350
Fix up log_summary_new and fix logging bug for barrier
Quentin-Anthony Jun 7, 2022
227874e
Clean up arg sweep logic and add isend/irecv
Quentin-Anthony Jun 7, 2022
27c38f9
Merge branch 'master' into staging-comms-logging-v1
Quentin-Anthony Jun 13, 2022
26e15ae
Clean up logging branch
Quentin-Anthony Jun 13, 2022
3aa3e38
Unify naming and fix circular import
Quentin-Anthony Jun 13, 2022
d2561dc
Fix deepspeed comm imports for logging.py
Quentin-Anthony Jun 13, 2022
c85f3c1
Added comms config support, removed some log names
Quentin-Anthony Jun 14, 2022
f70addb
Add comms config file
Quentin-Anthony Jun 14, 2022
a153331
Add pydantic to requirements
Quentin-Anthony Jun 14, 2022
351f384
Add configure non-op to old torch
Quentin-Anthony Jun 14, 2022
bcb3afd
Update logging call for old torch
Quentin-Anthony Jun 14, 2022
2f8320a
Add log_name placeholder args for old torch
Quentin-Anthony Jun 14, 2022
95aa7d8
Add basic verbosity setup
Quentin-Anthony Jun 15, 2022
93d1a31
Complete verbosity setup
Quentin-Anthony Jun 18, 2022
4a6236d
move comms logging to separate file and clean up
Quentin-Anthony Jun 18, 2022
393c90a
Change debug message design
Quentin-Anthony Jun 25, 2022
527d1c8
refactor debug helper and clean up
Quentin-Anthony Jun 25, 2022
40482a8
Refactor a bit and clean up prints
Quentin-Anthony Jun 25, 2022
a6beecf
Merge branch 'master' into staging-comms-logging-v1
Quentin-Anthony Jun 25, 2022
9343f87
config docs, remove old log_summary func, fix imports
Quentin-Anthony Jun 25, 2022
c07bc13
Finished docs, added import, fixed non-debug calls
Quentin-Anthony Jun 25, 2022
f5fd1f2
Ran pre-commit
Quentin-Anthony Jun 25, 2022
1b31798
Removed old comments
Quentin-Anthony Jun 25, 2022
298349d
Updated fn signatures for torch1.2
Quentin-Anthony Jun 27, 2022
102ae1d
Remove lingering prof arg
Quentin-Anthony Jun 27, 2022
2185f16
Merge branch 'master' into staging-comms-logging-v1
jeffra Jun 29, 2022
4faf3b9
Update logging tutorial
Quentin-Anthony Jun 29, 2022
6381187
Removed unnecessary imports and cleaned up comments
Quentin-Anthony Jun 30, 2022
56dbd71
Take master's cleaner comms init logic
Quentin-Anthony Jun 30, 2022
ae524f0
Fixed bw calculations and made all logging calls blocking
Quentin-Anthony Jul 20, 2022
19bcf79
Added comms logging synch disclaimer
Quentin-Anthony Jul 20, 2022
b9cb4d3
Merge branch 'master' into staging-comms-logging-v1
Quentin-Anthony Jul 21, 2022
c6925a1
Added using_mpi flag for logging
Quentin-Anthony Jul 22, 2022
5a0715c
Formatting
Quentin-Anthony Jul 22, 2022
b4449a2
Merge branch 'master' of https://github.com/microsoft/DeepSpeed into …
Quentin-Anthony Jul 22, 2022
b648979
Merge branch 'master' into staging-comms-logging-v1
Quentin-Anthony Jul 22, 2022
9357a16
Merge branch 'master' into staging-comms-logging-v1
Quentin-Anthony Jul 25, 2022
c85e323
Merge branch 'master' into staging-comms-logging-v1
Quentin-Anthony Jul 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Delete distributed.py
  • Loading branch information
@awan-10
awan-10 authored May 27, 2022
commit 7f8ca0131a952e3b568acddabcf7045d00fb6f27
173 changes: 0 additions & 173 deletions deepspeed/utils/distributed.py
View file
Open in desktop

This file was deleted.