Skip to content

Implement the ability to use layernorm with resnet models #7294

Closed
@MiguelMonteiro

Description

@MiguelMonteiro

Is your feature request related to a problem? Please describe.
Batch norm can be problematic with small batch sizes and certain applications due to computing running batch statistics. Layernorm does not have this problem but it is not available to use in with monai's resnet models:

from monai.networks.nets.resnet import ...

Describe the solution you'd like
Implement the ability to use layer norm with resnet models

Describe alternatives you've considered
I have tried using other models which support layer norm such as densenet but it seems to be broken also. See #7293

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions