Boxing expr

[lixinqi]

boxing的完整需求有很多：

1. 足够完备。
2. 提示用户成功的boxing链。
3. 提示用户失败的boxing检查。
4. 方便性能分析。
5. 可组合，可扩展。复杂boxing可以通过简单boxing组合出来，研发人员不应该手写复杂的boxing。

基于上述需求：设计了boxing表达式。拥有简单的文法：

boxing_expr : atomic_boxing_expr | dividor boxing_expr boxing_expr | boxing_expr '|' boxing_expr;
atomic_boxing_expr: (boxing_checker, boxing_function);

dividor: (placed_nd_sbp, placed_nd_sbp) -> placed_nd_sbp;
boxing_checker: (placed_nd_sbp, placed_nd_sbp) -> bool; 
boxing_function: (tensor, placed_nd_sbp, placed_nd_sbp) -> tensor;
placed_nd_sbp: (nd_sbp, placement);

实际上只有前两项是文法部分，后面都是对文法中概念的注解。

复合的结果其实是需要构造出复杂boxing_expr的checker函数和boxing_function函数。

任何复杂的boxing过程可以看成基本boxing函数上的两种复合：

1. 并联。代表多种boxing的选择。
2. 串联。代表多种boxing的接力。但是不能是简单的接力，需要用“分治”的思路来接力，即需要提供一个dividor。

dividor负责将一个复杂的boxing拆解成两个简单的boxing。dividor boxing_expr boxing_expr法则同时指定“分”和“治”的方式。后两个boxing_expr分别负责如何处理被切分后的两个boxing过程。

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 140.5ms (= 7026.1ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 128.8ms (= 6438.4ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.09 (= 140.5ms / 128.8ms)

PyTorch resnet50 time: 84.1ms (= 4204.8ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.9ms (= 3742.7ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.12 (= 84.1ms / 74.9ms)

PyTorch resnet50 time: 55.7ms (= 2783.0ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.8ms (= 2389.0ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.16 (= 55.7ms / 47.8ms)

PyTorch resnet50 time: 48.3ms (= 2414.0ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 40.0ms (= 2002.3ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.21 (= 48.3ms / 40.0ms)

PyTorch resnet50 time: 40.4ms (= 2020.7ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 44.1ms (= 2206.8ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 0.92 (= 40.4ms / 44.1ms)

[github-actions]

Speed stats:

GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 138.8ms (= 6940.4ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 128.6ms (= 6428.5ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.08 (= 138.8ms / 128.6ms)

PyTorch resnet50 time: 85.0ms (= 4250.4ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 74.9ms (= 3743.6ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.14 (= 85.0ms / 74.9ms)

PyTorch resnet50 time: 59.2ms (= 2960.3ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 47.9ms (= 2394.8ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.24 (= 59.2ms / 47.9ms)

PyTorch resnet50 time: 47.2ms (= 2361.3ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 40.7ms (= 2032.9ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 1.16 (= 47.2ms / 40.7ms)

PyTorch resnet50 time: 40.8ms (= 2039.0ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 44.1ms (= 2203.5ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 0.93 (= 40.8ms / 44.1ms)