Skip to content

The default setting for aggregation='mean' in Variable and Optimizer is incorrect. #20698

@Girls-WhatsApp-Groups-Links

Description

The default policy of using aggregation='mean' is incorrect and should be set to none. In distributed contexts, backends handle gradient reductions and variable updates, making the mean aggregation unnecessary. Using aggregation='mean' disrupts optimizer moment estimates. For reference, in keras==2.15.0, the default policy was effectively equivalent to aggregation='none

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions