Image augmentation #18

ravinkohli · 2020-10-25T16:03:52Z

Added:

image augmentations
ImageAugmenter that aggregates the various augmentations automatically and create a sequential
ImageDataLoader that can add image augmenter to transforms

…tations

…nd example

franchuterivera · 2020-10-26T14:17:38Z

autoPyTorch/pipeline/components/setup/early_preprocessor/EarlyPreprocessing.py

@@ -67,8 +67,8 @@ def get_hyperparameter_search_space(
    @staticmethod
    def get_properties(dataset_properties: Optional[Dict[str, Any]] = None) -> Dict[str, str]:
        return {
-            'shortname': 'Preprocessing',
-            'name': 'Preprocessing Node',
+            'shortname': 'EarlyPreprocessing',


haha my PR and your PR will class on this change. Depending on which get merged first, this will have to be updated

franchuterivera · 2020-10-26T14:22:20Z

autoPyTorch/pipeline/components/setup/augmentation/image/base_image_augmenter.py

+                                 " fitted yet".format(self.__class__.__name__))
+        return self.augmenter
+    #
+    # def check_requirements(self, X: Dict[str, Any], y: Any = None) -> None:


Why is this commented out :D?

I'm sorry I had copied first in case I need requirements. As augmenter has no requirements, it is not needed but I forgot to delete

franchuterivera · 2020-10-26T14:35:39Z

autoPyTorch/pipeline/components/setup/augmentation/image/ImageAugmenter.py

+        for name in available_augmenters.keys():
+            preprocessor_configuration_space = available_augmenters[name].\
+                get_hyperparameter_search_space(dataset_properties)
+            cs.add_configuration_space(name, preprocessor_configuration_space)


I fail to understand something -- let us say we don't want cropping for this pipeline because it hurts performance. Is there the possibility to not have cropping? Like normally I will expect a 'no cropping' or something similar. Maybe I missed something :)

So, the idea is for augmenters to have a probability of being applied as a hyperparameter. According to Lucas' comments, currently there is one just for RandomCutout. But it can be added for others as well.

I think we want a switch for the more fancy augmentations, such as use_cutout, which is what APT did before too, and then have all the hyperparameters as conditionals on that switch. I would say those would be all but flips and zero_pad_and_crop

autoPyTorch/pipeline/components/setup/augmentation/image/ImageAugmenter.py

franchuterivera · 2020-10-26T14:42:18Z

test/test_pipeline/components/test_setup_image_augmenter.py

+from autoPyTorch.pipeline.components.setup.augmentation.image.ImageAugmenter import ImageAugmenter
+
+
+class TestImageAugmenter(unittest.TestCase):


Can you add a test that actually takes an image and apply a transformation, to make sure it properly happen? It can be that it takes a random transformation on a manually created 1,H,W,C numpy array and when applied we make sure some change happened. This will also prevent bugs on theimplementation

franchuterivera · 2020-10-26T14:52:54Z

autoPyTorch/pipeline/components/training/metrics/utils.py

@@ -32,7 +32,7 @@ def get_components() -> Dict[str, autoPyTorchMetric]:
            as choices
    """
    components = OrderedDict()
-    components.update(_encoders)
+    components.update(_metrics)


Can you add a checker for the addition of new metrics? like a user-defined accuracy? Doesn't have to be in this PR, but create an issue for this?

Okay, but by checker you mean a test right?

franchuterivera

I left some comments that will likely request a change. Please take a look and see if they make sense

LMZimmer

Looking good, could you also add a resize as an augmentation?

autoPyTorch/pipeline/components/setup/augmentation/image/ImageAugmenter.py

LMZimmer · 2020-10-27T11:52:12Z

autoPyTorch/pipeline/components/setup/augmentation/image/ImageAugmenter.py

+        for name in available_augmenters.keys():
+            preprocessor_configuration_space = available_augmenters[name].\
+                get_hyperparameter_search_space(dataset_properties)
+            cs.add_configuration_space(name, preprocessor_configuration_space)


I think we want a switch for the more fancy augmentations, such as use_cutout, which is what APT did before too, and then have all the hyperparameters as conditionals on that switch. I would say those would be all but flips and zero_pad_and_crop

LMZimmer · 2020-10-27T11:56:34Z

requirements.txt

Lets just put this at >=0.4

LMZimmer · 2020-10-27T12:00:43Z

autoPyTorch/pipeline/components/training/data_loader/image_data_loader.py

+        # These can apply for both train/val/test, so no
+        # distinction is performed
+
+        if not X['is_small_preprocess']:


Could you add documentation what this means?

…augmented and added names to augmenters

…to-PyTorch_refactor into image_augmentation

LMZimmer

Just a few comments, once these are addressed I will merge it 👍

LMZimmer · 2020-10-30T12:29:17Z

autoPyTorch/pipeline/components/setup/augmentation/image/ImageAugmenter.py

+        Dict[str, BaseImageAugmenter]: all BaseImageAugmenter components available
+            as choices
+    """
+    # Error in implementation of CropToFixedSize augmenter in imgaug


Is this fixed? Then we don't need the comment anymore

LMZimmer · 2020-10-30T12:45:06Z

autoPyTorch/pipeline/components/setup/augmentation/image/RandomAffine.py

+        translate_percent_offset = UniformFloatHyperparameter('translate_percent_offset', lower=0, upper=0.99,
+                                                              default_value=0.3)
+        shear = UniformIntegerHyperparameter('shear', lower=0, upper=45, default_value=30)
+        rotate = UniformIntegerHyperparameter('rotate', lower=0, upper=360, default_value=45)


I would simplify this as follows:

Remove scale_min, translate_percent_min and hardcode them to 0.

scale_offset = UniformFloatHyperparameter('scale_offset', lower=0, upper=0.4, default_value=0.2) translate_percent_offset = UniformFloatHyperparameter('translate_percent_offset', lower=0, upper=0.4, default_value=0.2)

So I had made a mistake in interpreting what scale means. So, if we have scale as 1, that would mean scaled to 100% of original size(i.e, no change) else with 0.5 then 50% of the image. So I think scale offset should be 0 to 0.4(therefore scale_min) and it should go from scale_min to 1 and that should mean we scale it to anything between 60%(1-0.4) and 100%(1)

I would expect that if you specify the scaling percentage as e.g. (0, 0.4) that the augmentation scales the images from 60 % - 140 %, which it should. Could you double check that this is done now?

Hey Lucas, I checked and it is that whatever number we enter, its the percentage of zooming out(if greater than 1) or zooming in(if less than 1). So if we want something like 60-140%, we could have 1 - offset and 1+offset, or we could fix the offset and learn what to use as the centre point. Let me know what you think is more appropriate.

LMZimmer · 2020-10-30T12:55:14Z

autoPyTorch/pipeline/components/setup/augmentation/image/Resize.py

+    ) -> ConfigurationSpace:
+
+        cs = ConfigurationSpace()
+        interpolation = CategoricalHyperparameter('interpolation', choices=['nearest', 'linear', 'area', 'cubic'],


Let's fix this to one to reduce hyperparameters

autoPyTorch/pipeline/components/setup/augmentation/image/Resize.py

autoPyTorch/pipeline/components/setup/augmentation/image/ZeroPadAndCrop.py

LMZimmer · 2020-10-30T13:04:32Z

test/test_pipeline/components/test_setup_image_augmenter.py

+        image_augmenter = image_augmenter.set_hyperparameters(configuration=configuration)
+        X = dict(X_train=np.random.randint(0, 255, (8, 3, 16, 16), dtype=np.uint8), image_height=16, image_width=16)
+        for name, augmenter in image_augmenter.available_augmenters.items():
+            if not augmenter.use_augmenter:


To test every augmenter, there should be a configuration that has use_augmenter==True for all augmenters right?

LMZimmer · 2020-11-01T15:20:32Z

test/test_pipeline/components/test_setup_image_augmenter.py

@@ -10,12 +10,10 @@
 class TestImageAugmenter(unittest.TestCase):
    def test_every_augmenter(self):
        image_augmenter = ImageAugmenter()
-        configuration = image_augmenter.get_hyperparameter_search_space().sample_configuration()
+        configuration = image_augmenter.get_hyperparameter_search_space().get_default_configuration()


Does the default have every augmentation? I guess then this is fine, but perhaps add a comment here that this is implicitly assumed for this test

ravinkohli added 10 commits October 22, 2020 12:50

ImageAugmenter in progress

77a066b

FIX typo in metric utils

8f21e58

Completed image augmenter node and updated hyperparameters for augmen…

4df2a7a

…tations

ADD param to init of gaussian noise

604e306

ADD param to init of gaussian noise (2)

6bd75cd

Changed hps for gaussian noise, cutout, affine

0788b0d

Working augmenters and image augmenter with tests

55d3995

Fixed flake and mypy errors

88472c8

ADD Image data loader with tests

ced288a

ADD get_properties and str to augmenters and updated image pipeline a…

f6f3a99

…nd example

ravinkohli requested review from franchuterivera and LMZimmer October 25, 2020 16:04

ravinkohli added 4 commits October 26, 2020 13:47

FIX flake and requirements

38112db

FIX requirements

8465d4d

FIX requirements 2

2b92273

FIX requirements 3

9a1f9ec

franchuterivera reviewed Oct 26, 2020

View reviewed changes

autoPyTorch/pipeline/components/setup/augmentation/image/ImageAugmenter.py Outdated Show resolved Hide resolved

franchuterivera reviewed Oct 26, 2020

View reviewed changes

franchuterivera requested changes Oct 26, 2020

View reviewed changes

ravinkohli and others added 4 commits October 26, 2020 17:03

FIX requirements 4

aee63b9

Need imgaug 0.4.0

5868270

Removed tests for python 3.5

1e8ef8a

Merge branch 'development' into image_augmentation

343a876

LMZimmer requested changes Oct 27, 2020

View reviewed changes

ravinkohli added 2 commits October 27, 2020 16:35

ADD use_augmenter to augmenters, added test that checks if image was …

bed6f7e

…augmented and added names to augmenters

Changed requirement for imgaug

1816a4f

ravinkohli added 7 commits October 27, 2020 16:41

resolve merge conflict in travis

7717133

Merge branch 'image_augmentation' of https://github.com/ravinkohli/Au…

e461410

…to-PyTorch_refactor into image_augmentation

Added description about is_small_preprocess

f574ea4

fix flake 8 errors

56824b8

Added run_examples from fransisco's repo

600cc71

Fix error in run_examples

2f87dde

FIX for zero pad and crop

fa02ba0

LMZimmer requested changes Oct 30, 2020

View reviewed changes

Changed stuff in resize, affine, pnc, augmenter

d06338d

ravinkohli requested review from franchuterivera and LMZimmer October 30, 2020 15:16

LMZimmer reviewed Nov 1, 2020

View reviewed changes

Set each use_augmenter to True by default

e110256

LMZimmer merged commit dd6caf3 into LMZimmer:development Nov 2, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Image augmentation #18

Image augmentation #18

ravinkohli commented Oct 25, 2020

franchuterivera Oct 26, 2020

franchuterivera Oct 26, 2020

ravinkohli Oct 26, 2020

franchuterivera Oct 26, 2020

ravinkohli Oct 26, 2020

LMZimmer Oct 27, 2020

franchuterivera Oct 26, 2020

franchuterivera Oct 26, 2020

ravinkohli Oct 26, 2020 •

edited

Loading

franchuterivera Oct 27, 2020

franchuterivera left a comment

LMZimmer left a comment

LMZimmer Oct 27, 2020

LMZimmer Oct 27, 2020

LMZimmer Oct 27, 2020

ravinkohli Oct 27, 2020

LMZimmer left a comment

LMZimmer Oct 30, 2020

LMZimmer Oct 30, 2020

ravinkohli Oct 30, 2020

LMZimmer Nov 1, 2020

ravinkohli Nov 1, 2020

LMZimmer Oct 30, 2020

LMZimmer Oct 30, 2020

LMZimmer Nov 1, 2020

		from autoPyTorch.pipeline.components.setup.augmentation.image.ImageAugmenter import ImageAugmenter


		class TestImageAugmenter(unittest.TestCase):

Image augmentation #18

Image augmentation #18

Conversation

ravinkohli commented Oct 25, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ravinkohli Oct 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

franchuterivera left a comment

Choose a reason for hiding this comment

LMZimmer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LMZimmer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ravinkohli Oct 26, 2020 •

edited

Loading