Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image augmentation #18

Merged
merged 29 commits into from
Nov 2, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
77a066b
ImageAugmenter in progress
ravinkohli Oct 22, 2020
8f21e58
FIX typo in metric utils
ravinkohli Oct 22, 2020
4df2a7a
Completed image augmenter node and updated hyperparameters for augmen…
ravinkohli Oct 24, 2020
604e306
ADD param to init of gaussian noise
ravinkohli Oct 24, 2020
6bd75cd
ADD param to init of gaussian noise (2)
ravinkohli Oct 24, 2020
0788b0d
Changed hps for gaussian noise, cutout, affine
ravinkohli Oct 24, 2020
55d3995
Working augmenters and image augmenter with tests
ravinkohli Oct 25, 2020
88472c8
Fixed flake and mypy errors
ravinkohli Oct 25, 2020
ced288a
ADD Image data loader with tests
ravinkohli Oct 25, 2020
f6f3a99
ADD get_properties and str to augmenters and updated image pipeline a…
ravinkohli Oct 25, 2020
38112db
FIX flake and requirements
ravinkohli Oct 26, 2020
8465d4d
FIX requirements
ravinkohli Oct 26, 2020
2b92273
FIX requirements 2
ravinkohli Oct 26, 2020
9a1f9ec
FIX requirements 3
ravinkohli Oct 26, 2020
aee63b9
FIX requirements 4
ravinkohli Oct 26, 2020
5868270
Need imgaug 0.4.0
ravinkohli Oct 26, 2020
1e8ef8a
Removed tests for python 3.5
ravinkohli Oct 26, 2020
343a876
Merge branch 'development' into image_augmentation
LMZimmer Oct 27, 2020
bed6f7e
ADD use_augmenter to augmenters, added test that checks if image was …
ravinkohli Oct 27, 2020
1816a4f
Changed requirement for imgaug
ravinkohli Oct 27, 2020
7717133
resolve merge conflict in travis
ravinkohli Oct 27, 2020
e461410
Merge branch 'image_augmentation' of https://github.com/ravinkohli/Au…
ravinkohli Oct 27, 2020
f574ea4
Added description about is_small_preprocess
ravinkohli Oct 27, 2020
56824b8
fix flake 8 errors
ravinkohli Oct 27, 2020
600cc71
Added run_examples from fransisco's repo
ravinkohli Oct 27, 2020
2f87dde
Fix error in run_examples
ravinkohli Oct 27, 2020
fa02ba0
FIX for zero pad and crop
ravinkohli Oct 30, 2020
d06338d
Changed stuff in resize, affine, pnc, augmenter
ravinkohli Oct 30, 2020
e110256
Set each use_augmenter to True by default
ravinkohli Nov 1, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,6 @@ def get_components() -> Dict[str, BaseImageAugmenter]:
Dict[str, BaseImageAugmenter]: all BaseImageAugmenter components available
as choices
"""
# Error in implementation of CropToFixedSize augmenter in imgaug
# It returns a list of arrays instead of an array
# if 'ZeroPadAndCrop' in _augmenters:
# del _augmenters['ZeroPadAndCrop']
components = OrderedDict()
components.update(_augmenters)
components.update(_addons.components)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,13 @@


class RandomAffine(BaseImageAugmenter):
def __init__(self, use_augmenter: bool = True, scale_min: float = 0, scale_offset: float = 0.2,
translate_percent_min: float = 0, translate_percent_offset: float = 0.3,
shear: int = 30, rotate: int = 45, random_state: Optional[Union[int, np.random.RandomState]] = None):
def __init__(self, use_augmenter: bool = True, scale_offset: float = 0.2,
translate_percent_offset: float = 0.3, shear: int = 30,
rotate: int = 45, random_state: Optional[Union[int, np.random.RandomState]] = None):
super().__init__(use_augmenter=use_augmenter)
self.random_state = random_state
self.scale = (scale_min, scale_min + scale_offset)
self.translate_percent = (translate_percent_min, translate_percent_min + translate_percent_offset)
self.scale = (1, 1 - scale_offset)
self.translate_percent = (0, translate_percent_offset)
self.shear = (-shear, shear)
self.rotate = (-rotate, rotate)

Expand All @@ -41,24 +41,20 @@ def get_hyperparameter_search_space(
) -> ConfigurationSpace:

cs = ConfigurationSpace()
scale_min = UniformFloatHyperparameter('scale_min', lower=0, upper=0.99, default_value=0)
scale_offset = UniformFloatHyperparameter('scale_offset', lower=0, upper=0.99, default_value=0.2)
translate_percent_min = UniformFloatHyperparameter('translate_percent_min', lower=0, upper=0.99,
default_value=0)
translate_percent_offset = UniformFloatHyperparameter('translate_percent_offset', lower=0, upper=0.99,
default_value=0.3)
scale_offset = UniformFloatHyperparameter('scale_offset', lower=0, upper=0.4, default_value=0.2)

translate_percent_offset = UniformFloatHyperparameter('translate_percent_offset', lower=0, upper=0.4,
default_value=0.2)
shear = UniformIntegerHyperparameter('shear', lower=0, upper=45, default_value=30)
rotate = UniformIntegerHyperparameter('rotate', lower=0, upper=360, default_value=45)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would simplify this as follows:

Remove scale_min, translate_percent_min and hardcode them to 0.

scale_offset = UniformFloatHyperparameter('scale_offset', lower=0, upper=0.4, default_value=0.2)
translate_percent_offset = UniformFloatHyperparameter('translate_percent_offset', lower=0, upper=0.4, default_value=0.2)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I had made a mistake in interpreting what scale means. So, if we have scale as 1, that would mean scaled to 100% of original size(i.e, no change) else with 0.5 then 50% of the image. So I think scale offset should be 0 to 0.4(therefore scale_min) and it should go from scale_min to 1 and that should mean we scale it to anything between 60%(1-0.4) and 100%(1)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect that if you specify the scaling percentage as e.g. (0, 0.4) that the augmentation scales the images from 60 % - 140 %, which it should. Could you double check that this is done now?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Lucas, I checked and it is that whatever number we enter, its the percentage of zooming out(if greater than 1) or zooming in(if less than 1). So if we want something like 60-140%, we could have 1 - offset and 1+offset, or we could fix the offset and learn what to use as the centre point. Let me know what you think is more appropriate.


use_augmenter = CategoricalHyperparameter('use_augmenter', choices=[True, False])

cs.add_hyperparameters([scale_min, scale_offset, translate_percent_min, translate_percent_offset])
cs.add_hyperparameters([scale_offset, translate_percent_offset])
cs.add_hyperparameters([shear, rotate, use_augmenter])

# only add hyperparameters to configuration space if we are using the augmenter
cs.add_condition(CS.EqualsCondition(scale_min, use_augmenter, True))
cs.add_condition(CS.EqualsCondition(scale_offset, use_augmenter, True))
cs.add_condition(CS.EqualsCondition(translate_percent_min, use_augmenter, True))
cs.add_condition(CS.EqualsCondition(translate_percent_offset, use_augmenter, True))
cs.add_condition(CS.EqualsCondition(shear, use_augmenter, True))
cs.add_condition(CS.EqualsCondition(rotate, use_augmenter, True))
Expand Down
13 changes: 3 additions & 10 deletions autoPyTorch/pipeline/components/setup/augmentation/image/Resize.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
from typing import Any, Dict, Optional, Union

import ConfigSpace as CS
from ConfigSpace.configuration_space import ConfigurationSpace
from ConfigSpace.hyperparameters import (
CategoricalHyperparameter,
Expand All @@ -15,17 +14,16 @@


class Resize(BaseImageAugmenter):
def __init__(self, interpolation: str = 'linear', use_augmenter: bool = True,
def __init__(self, use_augmenter: bool = True,
random_state: Optional[Union[int, np.random.RandomState]] = None):
super().__init__(use_augmenter=use_augmenter)
self.random_state = random_state
self.interpolation = interpolation

def fit(self, X: Dict[str, Any], y: Any = None) -> BaseImageAugmenter:
self.check_requirements(X, y)
if self.use_augmenter:
self.augmenter: Augmenter = iaa.Resize(size=(X['image_height'], X['image_width']),
interpolation=self.interpolation, name=self.get_properties()['name'])
interpolation='linear', name=self.get_properties()['name'])

return self

Expand Down Expand Up @@ -53,13 +51,8 @@ def get_hyperparameter_search_space(
) -> ConfigurationSpace:

cs = ConfigurationSpace()
interpolation = CategoricalHyperparameter('interpolation', choices=['nearest', 'linear', 'area', 'cubic'],
default_value='linear')
use_augmenter = CategoricalHyperparameter('use_augmenter', choices=[True, False])
LMZimmer marked this conversation as resolved.
Show resolved Hide resolved
cs.add_hyperparameters([interpolation, use_augmenter])

# only add hyperparameters to configuration space if we are using the augmenter
cs.add_condition(CS.EqualsCondition(interpolation, use_augmenter, True))
cs.add_hyperparameters([use_augmenter])

return cs

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,20 +14,18 @@


class ZeroPadAndCrop(BaseImageAugmenter):
def __init__(self, x_position: float = 0, y_position: float = 0,
percent: float = 0.1, random_state: Optional[Union[int, np.random.RandomState]] = None):
def __init__(self, percent: float = 0.1,
random_state: Optional[Union[int, np.random.RandomState]] = None):
super().__init__()
self.random_state = random_state
self.percent = percent
self.position = (x_position, y_position)
self.pad_augmenter: Optional[Augmenter] = None
self.crop_augmenter: Optional[Augmenter] = None

def fit(self, X: Dict[str, Any], y: Any = None) -> BaseImageAugmenter:
self.check_requirements(X, y)
self.pad_augmenter = iaa.Pad(percent=self.percent, keep_size=False)
self.crop_augmenter = iaa.CropToFixedSize(height=X['image_height'], width=X['image_width'],
position=self.position)
self.crop_augmenter = iaa.CropToFixedSize(height=X['image_height'], width=X['image_width'])
self.augmenter: Augmenter = iaa.Sequential([
self.pad_augmenter,
self.crop_augmenter
Expand Down Expand Up @@ -59,10 +57,8 @@ def get_hyperparameter_search_space(
) -> ConfigurationSpace:

cs = ConfigurationSpace()
x_position = UniformFloatHyperparameter('x_position', lower=0, upper=1, default_value=0)
y_position = UniformFloatHyperparameter('y_position', lower=0, upper=1, default_value=0)
percent = UniformFloatHyperparameter('percent', lower=0, upper=0.5, default_value=0.1)
cs.add_hyperparameters([x_position, y_position, percent])
cs.add_hyperparameters([percent])
return cs

@staticmethod
Expand Down
4 changes: 1 addition & 3 deletions test/test_pipeline/components/test_setup_image_augmenter.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,10 @@
class TestImageAugmenter(unittest.TestCase):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a test that actually takes an image and apply a transformation, to make sure it properly happen? It can be that it takes a random transformation on a manually created 1,H,W,C numpy array and when applied we make sure some change happened. This will also prevent bugs on theimplementation

def test_every_augmenter(self):
image_augmenter = ImageAugmenter()
configuration = image_augmenter.get_hyperparameter_search_space().sample_configuration()
configuration = image_augmenter.get_hyperparameter_search_space().get_default_configuration()
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the default have every augmentation? I guess then this is fine, but perhaps add a comment here that this is implicitly assumed for this test

image_augmenter = image_augmenter.set_hyperparameters(configuration=configuration)
X = dict(X_train=np.random.randint(0, 255, (8, 3, 16, 16), dtype=np.uint8), image_height=16, image_width=16)
for name, augmenter in image_augmenter.available_augmenters.items():
if not augmenter.use_augmenter:
continue
augmenter = augmenter.fit(X)
# check if augmenter in the component has correct name
self.assertEqual(augmenter.get_image_augmenter().name, name)
Expand Down