Skip to content

5853: GEGLU activation function for the MLP Block. #5856

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Jan 19, 2023
Merged

5853: GEGLU activation function for the MLP Block. #5856

merged 7 commits into from
Jan 19, 2023

Conversation

@f-schnabel
Copy link
Contributor

@f-schnabel f-schnabel commented Jan 16, 2023 •
edited
Loading

Fixes #5853.

Description

Add GEGLU activation function for the MLP Block.

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • New tests added to cover the changes.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@f-schnabel f-schnabel force-pushed the issue/5853 branch 2 times, most recently from 412cb09 to 10f8a58 Compare January 19, 2023 18:20
@f-schnabel
Copy link
Contributor Author

f-schnabel commented Jan 19, 2023

Do I need to add tests for this?
For the other activation functions, I couldn't find any tests, so I'm unsure how to do that.

@f-schnabel f-schnabel marked this pull request as ready for review January 19, 2023 18:27
@f-schnabel
First draft of the GEGLU activation function for the MLP Block.
14d91f2 
Signed-off-by: Felix Schnabel <f.schnabel@tum.de>
@f-schnabel
Formatting
f686f39 
Signed-off-by: Felix Schnabel <f.schnabel@tum.de>
@f-schnabel
Add documentation
8d5e721 
Signed-off-by: Felix Schnabel <f.schnabel@tum.de>
@f-schnabel
Add GEGLU to documentation
35cba63 
Signed-off-by: Felix Schnabel <f.schnabel@tum.de>
@f-schnabel
Fix formatting and export GEGLU in __init__
11059b0 
Signed-off-by: Felix Schnabel <f.schnabel@tum.de>
@f-schnabel
Add testcase
2da7ad3 
Signed-off-by: Felix Schnabel <f.schnabel@tum.de>
@f-schnabel
Fix formatting
9ca029d 
Signed-off-by: Felix Schnabel <f.schnabel@tum.de>
wyli
wyli approved these changes Jan 19, 2023
View reviewed changes
Copy link
Contributor

@wyli wyli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, it looks good to me.

@wyli
Copy link
Contributor

wyli commented Jan 19, 2023

/build

@wyli wyli enabled auto-merge (squash) January 19, 2023 20:16
@wyli wyli merged commit 5c778ad into Project-MONAI:dev Jan 19, 2023
@f-schnabel f-schnabel deleted the issue/5853 branch January 23, 2023 12:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wyli wyli wyli approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Make MLPBlock compatible with diffusion models

2 participants

@f-schnabel @wyli