Add local to global balanced override env var #7981
Conversation
…feat-add_is_balanced_arg_in_to_global
…add_local_to_global_balanced_override_env_var
| } | ||
|
|
||
| bool GetLocalToGlobalIsBalanced(bool is_balanced) { | ||
| thread_local Optional<bool> eager_local_to_global_balanced_override = |
There was a problem hiding this comment.
看起来这个会一个环境变量打开所有的 local to global 的 is_balanced ？
There was a problem hiding this comment.
看起来这个会一个环境变量打开所有的 local to global 的 is_balanced ？
是的，这个环境变量有一键配置的作用，对整个网络生效
| @@ -53,6 +53,21 @@ namespace impl { | |||
|
|
|||
| namespace { | |||
|
|
|||
| Optional<bool> ParseEagerLocalToGlobalBalancedOverride() { | |||
| const char* env_p = std::getenv("ONEFLOW_EAGER_LOCAL_TO_GLOBAL_BALANCED_OVERRIDE"); | |||
There was a problem hiding this comment.
引入一个新的环境变量，至少源码层级得有一点注释或者解释。
There was a problem hiding this comment.
引入一个新的环境变量，至少源码层级得有一点注释或者解释。
已添加注释
|
Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally. |
…://github.com/Oneflow-Inc/oneflow into add_local_to_global_balanced_override_env_var
clackhan
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
clackhan
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
|
View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/7981/ |
clackhan
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
|
View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/7981/ |
|
View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/7981/ |
clackhan
requested review from
oneflow-ci-bot
and removed request for
oneflow-ci-bot
|
View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/7981/ |
Speed stats: |
添加
ONEFLOW_EAGER_LOCAL_TO_GLOBAL_BALANCED_OVERRIDE环境变量，该环境变量用于指示local to global时输入数据的形状是否是一样的，如果设置为true，则不需要同步各rank形状，本地计算逻辑形状。