Multi-GPU support is now experimental

[xuezhongcailian]

Feature

Does this repository currently support multi-GPU experiments

Motivation

Additional context

[fracape]

Hi, Not at the moment. For reference, there is a PR on that (#223) which requires some cleaning and a more robust / generic way to gather gpu info than using WORLD_SIZE directly.