[Auto Parallel] Add spmd rule No.9 for group_norm and group_norm_grad ops. #72946
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
jeff41404
reviewed
Jun 10, 2025
Comment on lines
120
to
145
| VLOG(4) << "shape size of x: 3~5"; | ||
| VLOG(4) << "Einsum Notation: " << x_axes << "," << scale_axes << "," | ||
| << bias_axes << "-->" << out_axes << "," << mean_axes << "," | ||
| << variance_axes; | ||
| VLOG(4) << "X" | ||
| << " shape: [" << str_join(x_shape) << "] " | ||
| << "src_dims_mapping: [" << str_join(x_dist_attr_src.dims_mapping()) | ||
| << "] " | ||
| << "dst_dims_mapping: [" << str_join(x_dims_mapping) << "]"; | ||
| VLOG(4) << "Scale" | ||
| << " shape: [" << str_join(scale_shape) << "] " | ||
| << "src_dims_mapping: [" << str_join(scale_dims_mapping) << "] " | ||
| << "dst_dims_mapping: [" | ||
| << str_join(scale_dist_attr_dst.dims_mapping()) << "]"; | ||
| VLOG(4) << "Bias" | ||
| << " shape: [" << str_join(bias_shape) << "] " | ||
| << "src_dims_mapping: [" << str_join(bias_dims_mapping) << "] " | ||
| << "dst_dims_mapping: [" | ||
| << str_join(bias_dist_attr_dst.dims_mapping()) << "]"; | ||
| VLOG(4) << "Out dims mapping: [" << str_join(out_dist_attr.dims_mapping()) | ||
| << "]"; | ||
| VLOG(4) << "Mean dims mapping: [" << str_join(mean_dist_attr.dims_mapping()) | ||
| << "]"; | ||
| VLOG(4) << "Variance dims mapping: [" | ||
| << str_join(variance_dist_attr.dims_mapping()) << "]"; | ||
| VLOG(4) << std::endl; |
There was a problem hiding this comment.
shall we use macro LOG_SPMD_INPUT or LOG_SPMD_OUTPUT to simplify log code
There was a problem hiding this comment.
Thanks, I will use it to simplify log code in next commit.
jeff41404
reviewed
Jun 10, 2025
Comment on lines
330
to
367
| VLOG(4) << "Scale" | ||
| << " shape: [" << str_join(scale_shape) << "] " | ||
| << "src_dims_mapping: [" << str_join(scale_dims_mapping) << "] " | ||
| << "dst_dims_mapping: [" | ||
| << str_join(scale_dist_attr_dst.dims_mapping()) << "]"; | ||
| VLOG(4) << "Bias" | ||
| << " shape: [" << str_join(bias_shape) << "] " | ||
| << "src_dims_mapping: [" << str_join(bias_dims_mapping) << "] " | ||
| << "dst_dims_mapping: [" | ||
| << str_join(bias_dist_attr_dst.dims_mapping()) << "]"; | ||
| VLOG(4) << "Y" | ||
| << " shape: [" << str_join(y_shape) << "] " | ||
| << "src_dims_mapping: [" << str_join(y_dims_mapping) << "] " | ||
| << "dst_dims_mapping: [" << str_join(y_dist_attr_dst.dims_mapping()) | ||
| << "]"; | ||
| VLOG(4) << "Mean" | ||
| << " shape: [" << str_join(mean_shape) << "] " | ||
| << "src_dims_mapping: [" << str_join(mean_dims_mapping) << "] " | ||
| << "dst_dims_mapping: [" | ||
| << str_join(mean_dist_attr_dst.dims_mapping()) << "]"; | ||
| VLOG(4) << "Variance" | ||
| << " shape: [" << str_join(variance_shape) << "] " | ||
| << "src_dims_mapping: [" << str_join(variance_dims_mapping) << "] " | ||
| << "dst_dims_mapping: [" | ||
| << str_join(variance_dist_attr_dst.dims_mapping()) << "]"; | ||
| VLOG(4) << "Y_grad" | ||
| << " shape: [" << str_join(y_grad_shape) << "] " | ||
| << "src_dims_mapping: [" << str_join(y_grad_dims_mapping) << "] " | ||
| << "dst_dims_mapping: [" | ||
| << str_join(y_grad_dist_attr_dst.dims_mapping()) << "]"; | ||
|
|
||
| VLOG(4) << "X_grad dims mapping: [" | ||
| << str_join(x_grad_dist_attr.dims_mapping()) << "]"; | ||
| VLOG(4) << "Scale_grad dims mapping: [" | ||
| << str_join(scale_grad_dist_attr.dims_mapping()) << "]"; | ||
| VLOG(4) << "Bias_grad dims mapping: [" | ||
| << str_join(bias_grad_dist_attr.dims_mapping()) << "]"; | ||
| VLOG(4) << std::endl; |
There was a problem hiding this comment.
shall we use macro LOG_SPMD_INPUT or LOG_SPMD_OUTPUT to simplify log code
There was a problem hiding this comment.
shall we use macro
LOG_SPMD_INPUTorLOG_SPMD_OUTPUTto simplify log code
Thanks, I will use it to simplify log code in next commit.
|
Sorry to inform you that f82d50d's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
|
@Glencsa 每条评审意见需要都回复 |
|
/re-run all-failed |
zyfncg
approved these changes
Jun 13, 2025
DrRyanHuang
pushed a commit
to DrRyanHuang/Paddle
that referenced
this pull request
Jun 14, 2025
… ops. (PaddlePaddle#72946) * add unary ops which have spmd_rule but not add in yaml file. * Add spmd_rule for group_norm ops. * Add spmd_rule for group_norm ops. * add CI test for group_norm. * add CI test for group_norm. * fix bug. * fix bug(PD_REGISTER_SPMD_RULE not surport string) * PD_REGISTER_SPMD_RULE need less than 5? * fix bug. * fix bug. * fix bug. * fix bug. * fix bug. * fix bug. * add partial status. * fix bug. * fix bug.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Category
Auto Parallel
PR Types
New features
Description