Skip to content

Conversation

cyx-6
Copy link
Contributor

@cyx-6 cyx-6 commented Jul 15, 2023

This PR introduces the operator root mean square, rms_norm, into TOPI.

@tvm-bot
Copy link
Collaborator

tvm-bot commented Jul 15, 2023

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

  • No users to tag found in teams: op See #10317 for details

Generated by tvm-bot

Copy link
Contributor

@sunggg sunggg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the nice work! A few minor comments.


# only test on llvm because schedule is missing
@tvm.testing.parametrize_targets("llvm")
@pytest.mark.parametrize("shape,axis", [([4, 16], (1,)), ([4, 16, 16], (1, 2))])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it also support symbolic shape, would you add a testcase?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, the symbolic shape test has been added.

<< "rms_norm: only support float32 and float16 for now";
bool is_float16 = data_type == DataType::Float(16);

auto x = is_float16 ? cast(data, DataType::Float(32)) : data;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need casting here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually no, and all the cast have been removed.

cyx-6 added 2 commits July 17, 2023 21:17
This PR introduces the operator root mean square, `rms_norm`, into TOPI.
@junrushao junrushao merged commit a13b56a into apache:main Jul 18, 2023
junrushao pushed a commit to junrushao/tvm that referenced this pull request Jul 24, 2023
This PR introduces the operator root mean square, `rms_norm`, into TOPI.
junrushao pushed a commit to junrushao/tvm that referenced this pull request Jul 27, 2023
This PR introduces the operator root mean square, `rms_norm`, into TOPI.
junrushao pushed a commit to junrushao/tvm that referenced this pull request Jul 30, 2023
This PR introduces the operator root mean square, `rms_norm`, into TOPI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants