RuntimeError: Default process group has not been initialized, please make sure to call init_process_group.

[cqray1990]

if args.local_rank == -1:
    device = torch.device("cuda")
else:
    torch.cuda.set_device(args.local_rank)
    device = torch.device("cuda", args.local_rank)
    deepspeed.init_distributed()
args.global_rank = torch.distributed.get_rank()

 此处代码是不是有问题？单独跑train.py 报错的，不用命令跑，不能单独调试代码，只能用distribute模式

[ZzYAmbition]

我用命令跑也是这个问题，请问解决了吗。