Open
Description
Background and Feature Description
Enable distributed and sharded training or document how this can be implemented by other means.
With the ever increasing size of models and datasets, DTensor seems to be the most flexible way to implement sharded training.
It would be great to know if you plan to implement/wrapt it.
API Definition and Usage
No response
Alternatives
Document how distributed, sharded training is possible
Risks
No response