Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add vista network #7987

Merged
merged 42 commits into from
Aug 15, 2024
Merged

Add vista network #7987

merged 42 commits into from
Aug 15, 2024

Conversation

heyufan1995
Copy link
Member

Fixes # .

Description

Add VISTA3D model architecture to MONAI core

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@heyufan1995
Add vista network
129a300 
Signed-off-by: heyufan1995 <heyufan1995@gmail.com>
ericspod
ericspod reviewed Aug 2, 2024
View reviewed changes
monai/transforms/utils.py Outdated Show resolved Hide resolved
ericspod
ericspod reviewed Aug 2, 2024
View reviewed changes
monai/transforms/utils.py Outdated Show resolved Hide resolved
ericspod
ericspod reviewed Aug 2, 2024
View reviewed changes
monai/transforms/utils.py Outdated Show resolved Hide resolved
@ericspod
Copy link
Member

ericspod commented Aug 2, 2024

Hi @heyufan1995 Thanks for the contribution from the VISTA team! I've added a few comments on style and such to try improving readability and number of lines. We will also need unit tests for anything added. You can also run the auto formatting fixes with runtests.sh.

KumoLiu
KumoLiu reviewed Aug 5, 2024
View reviewed changes
monai/networks/nets/segresnet_ds.py Outdated Show resolved Hide resolved
monai/networks/nets/segresnet_ds.py Outdated Show resolved Hide resolved
monai/networks/nets/vista3d.py Outdated Show resolved Hide resolved
@yiheng-wang-nv
Copy link
Contributor

HI @heyufan1995 , I suggest we split this PR into three small PRs:

  1. add util functions
  2. add SegResNetDS2
  3. add vista3d

To merge int MONAI, there are some basic requirements:

  1. correct code format
  2. enough unit tests
  3. enough doc strings

It would be more efficient to handle each parts separately

heyufan1995 and others added 2 commits August 5, 2024 15:36
@heyufan1995
Fix comments
66408f5 
Signed-off-by: Yufan He <heyufan1995@gmail.com>
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
91815bb 
for more information, see https://pre-commit.ci
This was referenced Aug 6, 2024
Merged
Merged
@yiheng-wang-nv
Copy link
Contributor

HI @heyufan1995 , I suggest we split this PR into three small PRs:

  1. add util functions
  2. add SegResNetDS2
  3. add vista3d

To merge int MONAI, there are some basic requirements:

  1. correct code format
  2. enough unit tests
  3. enough doc strings

It would be more efficient to handle each parts separately

for utils functions, I copy the content and submit a draft PR: #7999
Will address issues there

yiheng-wang-nv added a commit that referenced this pull request Aug 7, 2024
@yiheng-wang-nv @KumoLiu
7994-enhance-mlpblock (#7995)
49a1e34 
Fixes #7994  .

### Description
The current implementation does not support tuple input of "GEGLU" since
it only change the out features of the first linear layer when the input
is a string of "GEGLU".

This PR enhances it, and also enable "vista3d" mode to support #7987 
Tests are added to cover the changes.

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [x] New tests added to cover the changes.
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [ ] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [x] In-line docstrings updated.
- [ ] Documentation updated, tested `make html` command in the `docs/`
folder.

---------

Signed-off-by: Yiheng Wang <vennw@nvidia.com>
Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>
Co-authored-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>
@yiheng-wang-nv
Copy link
Contributor

other discussed topics:

  1. TwoWayTransformer, TwoWayAttentionBlock, PositionEmbeddingRandom and other blocks may move to other files.
  2. SABlock, may need to adjust to use different linear layers for q, k, v respectively

@yiheng-wang-nv
fix Tensor tensor issue
6c46d47 
Signed-off-by: Yiheng Wang <vennw@nvidia.com>
@yiheng-wang-nv
add vista3d doc
979bbe7 
Signed-off-by: Yiheng Wang <vennw@nvidia.com>
yiheng-wang-nv and others added 9 commits August 12, 2024 22:34
KumoLiu
KumoLiu reviewed Aug 13, 2024
View reviewed changes
monai/networks/nets/segresnet_ds.py Show resolved Hide resolved
monai/networks/nets/segresnet_ds.py Outdated Show resolved Hide resolved
monai/networks/nets/vista3d.py Outdated Show resolved Hide resolved
monai/networks/nets/vista3d.py Outdated Show resolved Hide resolved
monai/networks/nets/vista3d.py Outdated Show resolved Hide resolved
monai/networks/nets/vista3d.py Outdated Show resolved Hide resolved
monai/networks/nets/vista3d.py Show resolved Hide resolved
KumoLiu and others added 2 commits August 13, 2024 16:18
@KumoLiu
Merge branch 'dev' into add-vista3d-network
3b58ca7
@yiheng-wang-nv @KumoLiu
Update monai/networks/nets/vista3d.py
5d6e7b1 
Co-authored-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>
Signed-off-by: Yiheng Wang <68361391+yiheng-wang-nv@users.noreply.github.com>
heyufan1995 and others added 2 commits August 13, 2024 13:55
@heyufan1995
Update docstring and removed unused layer
6b53216 
Signed-off-by: heyufan1995 <heyufan1995@gmail.com>
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
cddcd30 
for more information, see https://pre-commit.ci
KumoLiu
KumoLiu approved these changes Aug 14, 2024
View reviewed changes
Copy link
Contributor

@KumoLiu KumoLiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick update, LGTM now.

@KumoLiu
Copy link
Contributor

KumoLiu commented Aug 14, 2024

Hi @heyufan1995 and @yiheng-wang-nv, please help take a look at the ci issue.
https://github.com/Project-MONAI/MONAI/actions/runs/10374743344/job/28722878047?pr=7987#step:7:392
https://github.com/Project-MONAI/MONAI/actions/runs/10374743344/job/28722877791?pr=7987
Thanks.

@KumoLiu
Copy link
Contributor

KumoLiu commented Aug 15, 2024

/build

@KumoLiu KumoLiu merged commit 77304dd into Project-MONAI:dev Aug 15, 2024
28 checks passed
@KumoLiu KumoLiu added this to the VISTA 3D [P0 v1.4] milestone Aug 19, 2024
rcremese pushed a commit to rcremese/MONAI that referenced this pull request Sep 2, 2024
@yiheng-wang-nv @KumoLiu
@ericspod @rcremese
Add utils for vista3d (Project-MONAI#7999)
1b5402b 
This PR is a part of Project-MONAI#7987 

### Description

A few sentences describing the changes proposed in this pull request.

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [x] New tests added to cover the changes.
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [ ] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [x] In-line docstrings updated.
- [x] Documentation updated, tested `make html` command in the `docs/`
folder.

---------

Signed-off-by: Yiheng Wang <vennw@nvidia.com>
Signed-off-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>
Signed-off-by: Yiheng Wang <68361391+yiheng-wang-nv@users.noreply.github.com>
Co-authored-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>
Co-authored-by: Eric Kerfoot <17726042+ericspod@users.noreply.github.com>
rcremese pushed a commit to rcremese/MONAI that referenced this pull request Sep 2, 2024
@heyufan1995 @pre-commit-ci
@yiheng-wang-nv @KumoLiu @rcremese
Add vista network (Project-MONAI#7987)
0f23244 
Fixes # .

### Description

Add VISTA3D model architecture to MONAI core

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [ ] New tests added to cover the changes.
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [ ] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [ ] In-line docstrings updated.
- [ ] Documentation updated, tested `make html` command in the `docs/`
folder.

---------

Signed-off-by: heyufan1995 <heyufan1995@gmail.com>
Signed-off-by: Yufan He <heyufan1995@gmail.com>
Signed-off-by: Yiheng Wang <vennw@nvidia.com>
Signed-off-by: Yiheng Wang <68361391+yiheng-wang-nv@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Yiheng Wang <68361391+yiheng-wang-nv@users.noreply.github.com>
Co-authored-by: Yiheng Wang <vennw@nvidia.com>
Co-authored-by: YunLiu <55491388+KumoLiu@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yiheng-wang-nv yiheng-wang-nv yiheng-wang-nv left review comments

@KumoLiu KumoLiu KumoLiu approved these changes

@Nic-Ma Nic-Ma Awaiting requested review from Nic-Ma

@ericspod ericspod Awaiting requested review from ericspod

Assignees
No one assigned
Labels
None yet
Projects
No open projects
MONAI v1.4
Status: Done
Milestone
VISTA 3D [P0 v1.4]
Development

Successfully merging this pull request may close these issues.
4 participants
@heyufan1995 @ericspod @yiheng-wang-nv @KumoLiu