4855 lazy resampling impl -- Compose #5860

wyli · 2023-01-17T00:38:04Z

part of #4855

upgrade #4911 to use the latest dev API

Description

e.g., for a sequence of spatial transforms

xforms = [
    mt.LoadImageD(keys, ensure_channel_first=True),
    mt.Orientationd(keys, "RAS"),
    mt.SpacingD(keys, (1.5, 1.5, 1.5)),
    mt.CenterScaleCropD(keys, roi_scale=0.9),
    # mt.CropForegroundD(keys, source_key="seg", k_divisible=5),
    mt.RandRotateD(keys, prob=1.0, range_y=np.pi / 2, range_x=np.pi / 3),
    mt.RandSpatialCropD(keys, roi_size=(76, 87, 73)),
    mt.RandScaleCropD(keys, roi_scale=0.9),
    mt.Resized(keys, (30, 40, 60)),
    # mt.NormalizeIntensityd(keys),
    mt.ZoomD(keys, 1.3, keep_size=False),
    mt.FlipD(keys),
    mt.Rotate90D(keys),
    mt.RandAffined(keys),
    mt.ResizeWithPadOrCropd(keys, spatial_size=(32, 43, 54)),
    mt.DivisiblePadD(keys, k=3),
]
lazy_kwargs = dict(mode=("bilinear", 0), padding_mode=("border", "nearest"), dtype=(torch.float32, torch.uint8))
xform = mt.Compose(xforms, lazy_evaluation=True, overrides=lazy_kwargs, override_keys=keys)
xform.set_random_state(0)

lazy_evaluation=True preserves more details

compared with the regular compose

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
Breaking change (fix or new feature that would cause existing functionality to change).
New tests added to cover the changes.
Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
In-line docstrings updated.
Documentation updated, tested make html command in the docs/ folder.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

…-impl Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

…-impl Signed-off-by: Wenqi Li <wenqil@nvidia.com>

tests/test_integration_lazy_samples.py

Signed-off-by: Ben Murray <ben.murray@gmail.com>

monai/transforms/compose.py

Nic-Ma

Looks good to me, looking forward to more tests and some bundle benchmarks before merging.

Thanks.

monai/transforms/lazy/functional.py

tests/test_integration_lazy_samples.py

monai/transforms/compose.py

wyli · 2023-03-23T12:49:14Z

thanks, I'm working on this branch to address the review comments.

atbenmurray · 2023-03-23T12:57:44Z

thanks, I'm working on this branch to address the review comments.

Ahh, @wyli I was in the process of adding the missing keywords to the overrides namespace, but if you are already doing it I'll leave it

wyli · 2023-03-23T13:07:29Z

thanks, I'm working on this branch to address the review comments.

Ahh, @wyli I was in the process of adding the missing keywords to the overrides namespace, but if you are already doing it I'll leave it

sure, I'll make minor usability fixes and merge it soon..

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

wyli · 2023-03-23T18:15:28Z

/build

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

wyli · 2023-03-23T18:29:21Z

/build

part of Project-MONAI#4855 upgrade Project-MONAI#4911 to use the latest dev API ### Description Example usage: for a sequence of spatial transforms ```py xforms = [ mt.LoadImageD(keys, ensure_channel_first=True), mt.Orientationd(keys, "RAS"), mt.SpacingD(keys, (1.5, 1.5, 1.5)), mt.CenterScaleCropD(keys, roi_scale=0.9), # mt.CropForegroundD(keys, source_key="seg", k_divisible=5), mt.RandRotateD(keys, prob=1.0, range_y=np.pi / 2, range_x=np.pi / 3), mt.RandSpatialCropD(keys, roi_size=(76, 87, 73)), mt.RandScaleCropD(keys, roi_scale=0.9), mt.Resized(keys, (30, 40, 60)), # mt.NormalizeIntensityd(keys), mt.ZoomD(keys, 1.3, keep_size=False), mt.FlipD(keys), mt.Rotate90D(keys), mt.RandAffined(keys), mt.ResizeWithPadOrCropd(keys, spatial_size=(32, 43, 54)), mt.DivisiblePadD(keys, k=3), ] lazy_kwargs = dict(mode=("bilinear", 0), padding_mode=("border", "nearest"), dtype=(torch.float32, torch.uint8)) xform = mt.Compose(xforms, lazy_evaluation=True, overrides=lazy_kwargs, override_keys=keys) xform.set_random_state(0) ``` lazy_evaluation=True preserves more details ![Screenshot 2023-01-17 at 00 31 40](https://user-images.githubusercontent.com/831580/212784981-ea39833b-54ab-42fb-bc03-38b012281857.png) compared with the regular compose ![Screenshot 2023-01-17 at 00 31 43](https://user-images.githubusercontent.com/831580/212785016-ba3be8ff-f17f-47b4-8025-cd351a637a82.png) ### Types of changes  - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. --------- Signed-off-by: Wenqi Li <wenqil@nvidia.com> Signed-off-by: Yiheng Wang <vennw@nvidia.com> Signed-off-by: KumoLiu <yunl@nvidia.com> Signed-off-by: Ben Murray <ben.murray@gmail.com> Co-authored-by: Ben Murray <ben.murray@gmail.com> Co-authored-by: binliu <binliu@nvidia.com> Co-authored-by: Yiheng Wang <68361391+yiheng-wang-nv@users.noreply.github.com> Co-authored-by: YunLiu <55491388+KumoLiu@users.noreply.github.com> Co-authored-by: Yiheng Wang <vennw@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: KumoLiu <yunl@nvidia.com>

wyli mentioned this pull request Jan 17, 2023

[Prototype] 4855 compose with lazy resampling #4911

Closed

12 tasks

wyli force-pushed the 4855-lazy-resampling-impl branch from 690d075 to bd21ea0 Compare January 17, 2023 15:24

wyli added this to the Lazy resampling (+ cache-based dataset refactoring) [P1 v1.2] milestone Jan 17, 2023

wyli force-pushed the 4855-lazy-resampling-impl branch 3 times, most recently from 5e58297 to 42844e8 Compare January 24, 2023 00:38

wyli added 9 commits January 25, 2023 16:01

smaller copy

3083442

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

fixes tests

1e15fd1

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

fixe tests

88ab8ed

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

fixes push transform

79f368e

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

one of

edcd20f

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

push transform tensor

256267b

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

refactoring push transform

bd33d93

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

fixes tests

2c040be

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

default false lazy

7699634

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

wyli force-pushed the 4855-lazy-resampling-impl branch from 45631e0 to 1232a6b Compare January 25, 2023 18:38

fixes tests

aa0389e

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

wyli force-pushed the 4855-lazy-resampling-impl branch from 1232a6b to aa0389e Compare January 25, 2023 18:51

wyli added 3 commits January 25, 2023 21:08

update one_of/random_order tests

0fbbd78

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

evaluate cases

a2a60f1

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

no inplace meta change

fce044b

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

wyli force-pushed the 4855-lazy-resampling-impl branch from 9790148 to fce044b Compare January 26, 2023 00:52

wyli added 8 commits January 26, 2023 01:40

update samples

d6d8e9b

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

multisample cropping

4b062ef

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

fixes tests

c00b2f6

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Merge remote-tracking branch 'upstream/dev' into 4855-lazy-resampling…

1f63bbc

…-impl Signed-off-by: Wenqi Li <wenqil@nvidia.com>

style fix

526f1f7

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

fixes style

a61d3ef

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Merge remote-tracking branch 'upstream/dev' into 4855-lazy-resampling…

93784f4

…-impl Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Merge remote-tracking branch 'upstream/dev' into 4855-lazy-resampling…

937744c

…-impl Signed-off-by: Wenqi Li <wenqil@nvidia.com>

atbenmurray mentioned this pull request Mar 22, 2023

Improve Compose encapsulation #6224

Merged

7 tasks

Nic-Ma reviewed Mar 22, 2023

View reviewed changes

tests/test_integration_lazy_samples.py Outdated Show resolved Hide resolved

wyli and others added 2 commits March 22, 2023 15:45

Merge branch 'dev' into 4855-lazy-resampling-impl

349dc09

Fix for non-trivial mappings between override_keys and _keys (#21)

40932e0

Signed-off-by: Ben Murray <ben.murray@gmail.com>

KumoLiu reviewed Mar 23, 2023

View reviewed changes

monai/transforms/compose.py Outdated Show resolved Hide resolved

wyli commented Mar 23, 2023

View reviewed changes

monai/transforms/compose.py Outdated Show resolved Hide resolved

wyli commented Mar 23, 2023

View reviewed changes

monai/transforms/compose.py Show resolved Hide resolved

Nic-Ma approved these changes Mar 23, 2023

View reviewed changes

monai/transforms/lazy/functional.py Outdated Show resolved Hide resolved

tests/test_integration_lazy_samples.py Outdated Show resolved Hide resolved

monai/transforms/compose.py Outdated Show resolved Hide resolved

Merge branch 'dev' into 4855-lazy-resampling-impl

17e4134

wyli and others added 7 commits March 23, 2023 14:22

Merge branch 'dev' into 4855-lazy-resampling-impl

12821a5

update integration tests

9d1fe50

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

update device

cc73d6d

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

update

b8674bf

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

update

91adea2

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

update based on comments

732ecd0

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

update based on comments

28b60b9

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

wyli enabled auto-merge (squash) March 23, 2023 18:17

update default flag

c7ec452

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

wyli disabled auto-merge March 23, 2023 18:30

wyli changed the title ~~4855 lazy resampling impl~~ 4855 lazy resampling impl -- Compose Mar 23, 2023

wyli merged commit b87375f into Project-MONAI:dev Mar 23, 2023

wyli added this to the Lazy resampling (+ cache-based dataset refactoring) [P1 v1.2] milestone Mar 23, 2023

atbenmurray mentioned this pull request Apr 28, 2023

Lazy eval refactor #6257

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

4855 lazy resampling impl -- Compose #5860

4855 lazy resampling impl -- Compose #5860

Uh oh!

wyli commented Jan 17, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Nic-Ma left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wyli commented Mar 23, 2023

Uh oh!

atbenmurray commented Mar 23, 2023

Uh oh!

wyli commented Mar 23, 2023

Uh oh!

wyli commented Mar 23, 2023

Uh oh!

wyli commented Mar 23, 2023

Uh oh!

Uh oh!

4855 lazy resampling impl -- Compose #5860

4855 lazy resampling impl -- Compose #5860

Uh oh!

Conversation

wyli commented Jan 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Nic-Ma left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wyli commented Mar 23, 2023

Uh oh!

atbenmurray commented Mar 23, 2023

Uh oh!

wyli commented Mar 23, 2023

Uh oh!

wyli commented Mar 23, 2023

Uh oh!

wyli commented Mar 23, 2023

Uh oh!

Uh oh!

wyli commented Jan 17, 2023 •

edited

Loading