Skip to content

Improve memory efficiency of the distributed wrappers #204

Closed
@KevinMusgrave

Description

@KevinMusgrave

For example the distributed loss wrapper computes a loss for the global batch, but keeps track of only the local gradients. It would be better to compute the loss relevant to only the local batch by making use of indices_tuple.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions