dev adamw graph config

[MARD1NO]

No description provided.

[Ldpe2G]

这里还有必要对 base_scale 做限制吗，我看#5821 这个pr里去掉了

[strint]

的确没有必要了

[strint]

同步一下那次讨论的结论

• loss scale是amp的功能之一，逻辑是防止loss过小，浮点截断后，导致回传梯度消失，所以先增大loss * scale，之后再grad/scale
• loss scale不属于optimizer的参数，所以optimizer中的scale在5821中都清理掉了
• optimizer调用modle_update_op的那个scale参数，是scale weight的一个接口，给内部调整weight用的通用接口

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 140.7ms (= 7033.6ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 128.3ms (= 6416.3ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.10 (= 140.7ms / 128.3ms)

PyTorch resnet50 time: 84.5ms (= 4224.9ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.3ms (= 3715.7ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.14 (= 84.5ms / 74.3ms)

PyTorch resnet50 time: 57.2ms (= 2860.7ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.8ms (= 2392.1ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.20 (= 57.2ms / 47.8ms)

PyTorch resnet50 time: 48.9ms (= 2445.2ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 46.1ms (= 2304.5ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.06 (= 48.9ms / 46.1ms)

PyTorch resnet50 time: 44.6ms (= 2231.0ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 40.3ms (= 2014.9ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 1.11 (= 44.6ms / 40.3ms)