Integrating a Tailored Diffusion U-Net Model into Generative Model Application #7867

dongyang0122 · 2024-06-21T21:12:50Z

Integrating a tailored diffusion U-Net model into the generative model application to improve the production of high-dimensional 3D images, specifically sized at 512 x 512 x 512.

A few sentences describing the changes proposed in this pull request.

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
Breaking change (fix or new feature that would cause existing functionality to change).
New tests added to cover the changes.
Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
In-line docstrings updated.
Documentation updated, tested make html command in the docs/ folder.

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

for more information, see https://pre-commit.ci

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

for more information, see https://pre-commit.ci

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

for more information, see https://pre-commit.ci

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

…0122/MONAI into fix-issue-7858-add-unet

for more information, see https://pre-commit.ci

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

for more information, see https://pre-commit.ci

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

for more information, see https://pre-commit.ci

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

for more information, see https://pre-commit.ci

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

dongyang0122 · 2024-06-22T22:23:16Z

@KumoLiu @Nic-Ma @ericspod I am not clear how to pass these failed checks. Please share your suggestion. Thanks.

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

KumoLiu

Thanks for the PR, few comments inline.

monai/apps/generation/maisi/networks/diffusion_model_unet_maisi.py

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

monai/apps/generation/maisi/networks/diffusion_model_unet_maisi.py

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

KumoLiu · 2024-07-01T15:19:37Z

/build

ericspod

I think this is good now. I'm not sure if referencing a specific commit in requirements-dev.txt is a good idea, we can do another "bug-fix" release of GenerativeModels later if this is a problem.

mingxin-zheng · 2024-07-08T06:53:26Z

monai/apps/generation/maisi/networks/diffusion_model_unet_maisi.py

+    "generative.networks.nets.diffusion_model_unet", name="get_timestep_embedding"
+)
+get_up_block, has_get_up_block = optional_import("generative.networks.nets.diffusion_model_unet", name="get_up_block")
+xformers, has_xformers = optional_import("xformers")


Hi @dongyang0122 @KumoLiu , I think we need to either add xformers into our requirements-dev, or remove this from the code.

P.S. As far as I know, the installation of xformers is pretty nasty with nvidia DLFW pytorch container. It need to build from source and the compilation will somehow break/restart the machine if it hardwares are low profile.

init

6ca3b1b

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

dongyang0122 self-assigned this Jun 21, 2024

dongyang0122 mentioned this pull request Jun 21, 2024

Adding Tailored Model Implementations into Generative Model Application #7858

Closed

2 tasks

pre-commit-ci bot and others added 26 commits June 21, 2024 21:14

[pre-commit.ci] auto fixes from pre-commit.com hooks

2989c50

for more information, see https://pre-commit.ci

update

39f6469

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

[pre-commit.ci] auto fixes from pre-commit.com hooks

f1eeeb3

for more information, see https://pre-commit.ci

update

76e3f4a

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

update

f75e513

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

update inferers

c1749e0

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

[pre-commit.ci] auto fixes from pre-commit.com hooks

0422908

for more information, see https://pre-commit.ci

update inferers

926eb7f

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

Merge branch 'fix-issue-7858-add-unet' of https://github.com/dongyang…

6acb848

…0122/MONAI into fix-issue-7858-add-unet

[pre-commit.ci] auto fixes from pre-commit.com hooks

f99f2f3

for more information, see https://pre-commit.ci

update inferers

8ec8848

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

update inferers

3953bfb

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

update inferers

744d829

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

[pre-commit.ci] auto fixes from pre-commit.com hooks

de414bb

for more information, see https://pre-commit.ci

update inferers

8fc994c

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

update inferers

59aacd9

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

update inferers

6638d17

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

add unit test cases

f6e5d2b

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

[pre-commit.ci] auto fixes from pre-commit.com hooks

346c645

for more information, see https://pre-commit.ci

fix format error

d035b73

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update diffusion unet

f21d77c

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update unet

f82bf6b

Signed-off-by: Dong Yang <yangd05@Dongs-MacBook-Pro.local>

[pre-commit.ci] auto fixes from pre-commit.com hooks

31d6795

for more information, see https://pre-commit.ci

update diffusion unet

d423cd3

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update diffusion unet

54fd068

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update

03d7de2

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

dongyang0122 marked this pull request as ready for review June 22, 2024 06:35

dongyang0122 added 2 commits June 22, 2024 06:52

update

5324e0e

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update unet

90d3961

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update

eb2ca47

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

KumoLiu reviewed Jun 25, 2024

View reviewed changes

monai/apps/generation/maisi/networks/diffusion_model_unet_maisi.py Outdated Show resolved Hide resolved

monai/apps/generation/maisi/networks/diffusion_model_unet_maisi.py Outdated Show resolved Hide resolved

dongyang0122 requested a review from mingxin-zheng June 25, 2024 13:38

dongyang0122 added 7 commits June 28, 2024 17:39

update

ec11055

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update xformers

867d0cd

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update print messages

38d41c4

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update tensor formating

8ad8ef8

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

refactor forward function

473ef4b

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

refactor embedding modules

4a53fc5

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

refactor embedding modules

fc3e828

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

ericspod reviewed Jun 29, 2024

View reviewed changes

monai/apps/generation/maisi/networks/diffusion_model_unet_maisi.py Outdated Show resolved Hide resolved

mingxin-zheng reviewed Jul 1, 2024

View reviewed changes

monai/apps/generation/maisi/networks/diffusion_model_unet_maisi.py Outdated Show resolved Hide resolved

mingxin-zheng reviewed Jul 1, 2024

View reviewed changes

monai/apps/generation/maisi/networks/diffusion_model_unet_maisi.py Outdated Show resolved Hide resolved

dongyang0122 and others added 6 commits July 1, 2024 13:13

refactor embedding modules

5ea43b9

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update

06852b8

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update

757dda4

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update

7a9cd10

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

update

22582bd

Signed-off-by: dongyang0122 <don.yang.mech@gmail.com>

Merge branch 'dev' into fix-issue-7858-add-unet

5dff148

KumoLiu requested review from ericspod and mingxin-zheng July 1, 2024 15:19

ericspod approved these changes Jul 2, 2024

View reviewed changes

KumoLiu merged commit 15d0771 into Project-MONAI:dev Jul 2, 2024
28 checks passed

mingxin-zheng mentioned this pull request Jul 2, 2024

The monai-generative in requirements-dev.txt points to a deleted branch #7894

Closed

mingxin-zheng reviewed Jul 8, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Integrating a Tailored Diffusion U-Net Model into Generative Model Application #7867

Integrating a Tailored Diffusion U-Net Model into Generative Model Application #7867

Uh oh!

dongyang0122 commented Jun 21, 2024

Uh oh!

dongyang0122 commented Jun 22, 2024

Uh oh!

KumoLiu left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KumoLiu commented Jul 1, 2024

Uh oh!

ericspod left a comment

Uh oh!

Uh oh!

mingxin-zheng Jul 8, 2024

Uh oh!

Uh oh!

Integrating a Tailored Diffusion U-Net Model into Generative Model Application #7867

Integrating a Tailored Diffusion U-Net Model into Generative Model Application #7867

Uh oh!

Conversation

dongyang0122 commented Jun 21, 2024

Types of changes

Uh oh!

dongyang0122 commented Jun 22, 2024

Uh oh!

KumoLiu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KumoLiu commented Jul 1, 2024

Uh oh!

ericspod left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mingxin-zheng Jul 8, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!