Geometric augmentations should happen before cropping in topdown pipelines

Currently, we are applying geometric augmentations (e.g., rotation) _after_ doing the instance crops:

https://github.com/talmolab/sleap-nn/blob/2f44d2db707496e7f2b9f587d69a791f89992bb3/sleap_nn/data/pipelines.py#L69-L92

This should happen _before_ cropping so we don't get black borders in the augmented images.

Intensity augmentation can happen after cropping, but we might as well do it all in one step (before).

	if use_augmentations and "intensity" in self.data_config.augmentation_config:
	datapipe = KorniaAugmenter(
	datapipe,
	**dict(self.data_config.augmentation_config.intensity),
	image_key="image",
	instance_key="instances",
	)

	datapipe = InstanceCentroidFinder(
	datapipe, anchor_ind=self.confmap_head.anchor_part
	)

	datapipe = InstanceCropper(
	datapipe,
	self.data_config.preprocessing.crop_hw,
	)

	if use_augmentations and "geometric" in self.data_config.augmentation_config:
	datapipe = KorniaAugmenter(
	datapipe,
	**dict(self.data_config.augmentation_config.geometric),
	image_key="instance_image",
	instance_key="instance",
	)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Geometric augmentations should happen before cropping in topdown pipelines #76

talmo
openedon Aug 14, 2024

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Geometric augmentations should happen before cropping in topdown pipelines #76

Description

talmoopenedon Aug 14, 2024

Metadata