Migrate to FSDP2?

[wconstab]

The PyTorch team is trying to migrate users from FSDP to FSDP2 (a.k.a. fully_shard). verl is a popular and growing source of new usage of FSDP1 and we want to ask if the verl team is open to migrating. We may be able to help with migration work and it would also be useful to know if there are specific features of FSDP1 you are relying on or if FSDP2 is already good enough. See FSDP1->FSDP2 for more info. Thanks!

cc @mori360 @weifengpy

[eric-haibin-lin]

yes we're quite open to migration. verl currently just use the common stuff, mixed precision, wrap policy, gradient accumulation.

[wconstab]

Thanks to @lxg2015 for initiating this work! #1026

[wconstab]

Happy to see that #1026 is merged! I wonder if folks have an idea about how/when to switch the default for VERL to use fsdp2? cc @PeterSH6 @eric-haibin-lin

[eric-haibin-lin]

Happy to see that #1026 is merged! I wonder if folks have an idea about how/when to switch the default for VERL to use fsdp2? cc @PeterSH6 @eric-haibin-lin

We can prioritize the fsdp2 implementation when doing #1371