[Transforms] ApplySomeOf

[marcown]

🚀 The feature

Hey,

I found it useful to have a someof class e.g. in imgaug to randomly apply a subset of transformations out of a set of transformation with a parameter "max_transforms". The number of transformations in the subset is randomly sampled with random.randint(0, max_number).

I could not found something similar in the repo so here is an example class for v2:

import torchvision
from typing import Sequence, Callable, Any, Optional, List
import random

class ApplySomeOf(torchvision.transforms.v2.Transform):
    def __init__(
        self,
        transforms: Sequence[Callable],
        max_transforms: Optional[int] = None,
        rand_shuffle: Optional[bool] = False,
    ) -> None:
        if not isinstance(transforms, Sequence):
            raise TypeError("Argument transforms should be a sequence of callables")

        if max_transforms is None:
            max_transforms = len(transforms)

        super().__init__()

        self.transforms = transforms
        self.max_transforms = max_transforms
        self.rand_shuffle =rand_shuffle

    def forward(self, *inputs: Any) -> Any:
        sample = inputs if len(inputs) > 1 else inputs[0]
        selected_transforms = random.sample(self.transforms, k=random.randint(0, self.max_transforms))

        if self.rand_shuffle:
            random.shuffle(selected_transforms)

        for transform in selected_transforms:
            sample = transform(sample)

        return sample

What do you think? Could this be integrated? Or did I miss a similar functionality?

Motivation, pitch

Imgaug has it. It is quite useful.

Alternatives

No response

Additional context

No response

cc @vfdev-5

[NicolasHug]

Thanks for the suggestion @marcown . I think we should be able to support this functionality by extending RandomChoice with a new parameter:

• max_transforms=None (default) -> current behaviour, always pick one transform randomly
• max_transforms=N -> sample in [0, N]

(We can't have the default be 1 because that would allow zero transforms to be applied, which would break BC by default).

If we want we can also allow max_transforms to be a tuple of ints in which case we sample between [a, b], but I'm not sure it's super useful - I'd leave it out for now.

Any additional thoughts / concerns @pmeier @vfdev-5 ?

[pmeier]

+1 for extending RandomChoice rather than having a separate transform.

max_transforms=N -> sample in [0, N]

@marcown could you link the paper / reference implementation for this? Is 0, i.e. not applying any transform, actually a valid use case or should it be [1, N]?

[marcown]

Adding it to RandomChoice sounds good to me, with the options None (pick one) and N (pick a subset of maximum size N). Furthermore, applying zero transformations is useful I think, randomly having images that are near the real world use case is quite helpful for training

I don't know any paper that did a large scale test of this function, I just used it a lot in the imgaug library: https://github.com/aleju/imgaug

@pmeier @NicolasHug

[ambujpawar]

Can I help with submitting the PR for this new feature?

[NicolasHug]

Thanks for proposing to help @ambujpawar. Feel free to submit a PR. Let's start by including this feature in transforms.v2.RandomChoice first (

vision/torchvision/transforms/v2/_container.py

Line 116 in 62c2231

class RandomChoice(Transform):

). We can tackle the V1 version later if we want to.

Something I missed earlier is that the new max_features parameter may not interact well with the p parameter. So to keep things simple, we can make them mutually exclusive. I.e. we should enforce that at least one of them must be kept to None.

[ambujpawar]

Thanks for the heads up. I will work on it and let you know if I encounter some issues