Refactor boxing interpreter to boxing expr #6134

clackhan · 2021-09-01T23:46:21Z

删除旧版boxing interpreter实现，所有类型的boxing均通过boxing expr实现

…ata_transport_token_per_placement

…bugfix_data_transport_token_per_placement

…ithub.com/Oneflow-Inc/oneflow into bugfix_data_transport_token_per_placement Conflicts: oneflow/core/vm/oneflow_vm.cpp

…placement' into send_recv

…q_id

Signed-off-by: daquexian <daquexian566@gmail.com>

…q_id

* cuda base cpu mpi boxing * cpu_mpi * fix conflicts * add cpu mpi unittests * more checks and unittests * abstract_consistent_to_consistent_op_expr * fix compiler complaint * refactor consistent-to-consistent eager consisitent op interpreter * fix compiler complaint * refactor ConsistentToConsistentOpExpr * lazy interpreter (#5903) * fix bugs about consistent_id * more test_consistent_cast unittests * refactor functional::ToConsistent * refactor GetNdSbp * fix compiler complaints * Update eager_consistent_op_interpreter.cpp * Update eager_mirrored_op_interpreter.cpp * eager_boxing_1_to_n * add missing files * del useless file * minor fix * refine * refactor GetDevice4CurrentProcessCtx * refine * minor fix * Update naive_1ton_boxing_interpreter.cpp * eager_boxing_n_to_1 * add test case * refine * Update eager_boxing_interpreter_mgr.cpp * Update eager_boxing_interpreter_mgr.cpp * fix error * fix error * auto format by CI * fix error * refine * refine * make of_format * make of_format * Update nd_sbp.h * fix consistent id check error * refine * back up * refine * minor fix * refine * refine * refine * refine * minor fix * minor fix * refine * refine * Update nccl_boxing_function.cpp * back up * refine * minor fix * refine * fix consistent meta check bug * zoom kLimitParallelConfString * refine * add nccl functional api * Update naive_n_to_1_boxing.cpp * minor fix * refine * refine * naive_generic_boxing * refine * test case * back up * back up * fix nccl deadlock bug * add test case * add test case * add test_eager_boxing_with_overlapping_placement test case * refine * add test case * add test case * fix check bug and add test case * add test case * add boxing_expr_with_inclusive_placement boxing expr * refine * refine * refine * refine * minor fix Co-authored-by: Xinqi Li <lixinqi0703106@163.com> Co-authored-by: leaves-zwx <kunta0932@gmail.com> Co-authored-by: Li Xinqi <lixinqi2010@gmail.com> Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com> Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

…eter

…refactor_BoxingInterpreter_to_BoxingExpr Conflicts: oneflow/core/framework/op_interpreter/boxing/eager_boxing_interpreter_mgr.cpp oneflow/core/framework/op_interpreter/boxing/naive_b2p_boxing_interpreter.cpp

clackhan · 2021-09-01T23:55:07Z

oneflow/core/framework/op_interpreter/boxing/identity_boxing_interpreter.cpp

-  if (tensor_nd_sbp == out->nd_sbp()) { return tensor; }
+  // reset sbp if parallel_num == 1 and reset transport_token


由于会对输出设置transport_token，故即使输入输出的placement和sbp一样也不能直接返回tensor

lixinqi · 2021-09-02T01:17:58Z

oneflow/core/framework/op_interpreter/boxing/eager_boxing_interpreter.h

@@ -45,20 +45,6 @@ class EagerBoxingInterpreter {
                                           Symbol<ParallelDesc> out_parallel_desc) const = 0;
 };

-struct EagerBoxingCall {


这些先留着。因为decompose暂时需要他们。

…thub.com/Oneflow-Inc/oneflow into refactor_BoxingInterpreter_to_BoxingExpr

github-actions · 2021-09-02T02:50:46Z

Speed stats:

GPU Name: GeForce GTX 1080 

OneFlow resnet50 time: 128.1ms (= 6407.2ms / 50, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 139.9ms (= 6996.9ms / 50, input_shape=[16, 3, 224, 224])
Relative speed: 1.09 (= 139.9ms / 128.1ms)

OneFlow resnet50 time: 74.8ms (= 3740.0ms / 50, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 86.3ms (= 4315.8ms / 50, input_shape=[8, 3, 224, 224])
Relative speed: 1.15 (= 86.3ms / 74.8ms)

OneFlow resnet50 time: 48.8ms (= 2439.0ms / 50, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 58.8ms (= 2937.6ms / 50, input_shape=[4, 3, 224, 224])
Relative speed: 1.20 (= 58.8ms / 48.8ms)

OneFlow resnet50 time: 41.2ms (= 2057.5ms / 50, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 49.4ms (= 2468.0ms / 50, input_shape=[2, 3, 224, 224])
Relative speed: 1.20 (= 49.4ms / 41.2ms)

OneFlow resnet50 time: 36.6ms (= 1831.9ms / 50, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.6ms (= 1978.5ms / 50, input_shape=[1, 3, 224, 224])
Relative speed: 1.08 (= 39.6ms / 36.6ms)

OneFlow resnet50 time: 144.0ms (= 7200.5ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 160.8ms (= 8040.4ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
Relative speed: 1.12 (= 160.8ms / 144.0ms)

OneFlow resnet50 time: 93.1ms (= 4653.2ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 102.2ms (= 5112.4ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
Relative speed: 1.10 (= 102.2ms / 93.1ms)

OneFlow resnet50 time: 67.2ms (= 3358.5ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 80.7ms (= 4035.3ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
Relative speed: 1.20 (= 80.7ms / 67.2ms)

OneFlow resnet50 time: 70.4ms (= 3521.0ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 60.2ms (= 3009.6ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
Relative speed: 0.85 (= 60.2ms / 70.4ms)

OneFlow resnet50 time: 66.8ms (= 3341.3ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 58.6ms (= 2928.0ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
Relative speed: 0.88 (= 58.6ms / 66.8ms)

…thub.com/Oneflow-Inc/oneflow into refactor_BoxingInterpreter_to_BoxingExpr

github-actions · 2021-09-02T04:08:46Z

Speed stats:

GPU Name: GeForce GTX 1080 

OneFlow resnet50 time: 128.1ms (= 6405.0ms / 50, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 141.4ms (= 7069.0ms / 50, input_shape=[16, 3, 224, 224])
Relative speed: 1.10 (= 141.4ms / 128.1ms)

OneFlow resnet50 time: 74.7ms (= 3733.7ms / 50, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 83.9ms (= 4196.8ms / 50, input_shape=[8, 3, 224, 224])
Relative speed: 1.12 (= 83.9ms / 74.7ms)

OneFlow resnet50 time: 48.1ms (= 2406.9ms / 50, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 58.1ms (= 2904.0ms / 50, input_shape=[4, 3, 224, 224])
Relative speed: 1.21 (= 58.1ms / 48.1ms)

OneFlow resnet50 time: 41.0ms (= 2051.5ms / 50, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 45.1ms (= 2253.5ms / 50, input_shape=[2, 3, 224, 224])
Relative speed: 1.10 (= 45.1ms / 41.0ms)

OneFlow resnet50 time: 42.1ms (= 2103.3ms / 50, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 39.2ms (= 1958.5ms / 50, input_shape=[1, 3, 224, 224])
Relative speed: 0.93 (= 39.2ms / 42.1ms)

OneFlow resnet50 time: 144.7ms (= 7235.9ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 160.1ms (= 8004.4ms / 50, input_shape=[16, 3, 224, 224], ddp, world size=2)
Relative speed: 1.11 (= 160.1ms / 144.7ms)

OneFlow resnet50 time: 92.5ms (= 4623.1ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 104.2ms (= 5207.9ms / 50, input_shape=[8, 3, 224, 224], ddp, world size=2)
Relative speed: 1.13 (= 104.2ms / 92.5ms)

OneFlow resnet50 time: 66.3ms (= 3315.4ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 76.4ms (= 3820.8ms / 50, input_shape=[4, 3, 224, 224], ddp, world size=2)
Relative speed: 1.15 (= 76.4ms / 66.3ms)

OneFlow resnet50 time: 68.4ms (= 3421.9ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 65.7ms (= 3286.1ms / 50, input_shape=[2, 3, 224, 224], ddp, world size=2)
Relative speed: 0.96 (= 65.7ms / 68.4ms)

OneFlow resnet50 time: 60.8ms (= 3037.9ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 64.2ms (= 3209.4ms / 50, input_shape=[1, 3, 224, 224], ddp, world size=2)
Relative speed: 1.06 (= 64.2ms / 60.8ms)

lixinqi and others added 30 commits August 13, 2021 23:00

add send_op/recv_op

fef087f

bugfix: data_transport_token_per_placement

7b92fd4

refactor TransportToken::NewDataTransportToken

9bcda09

Merge branch 'master' into bugfix_data_transport_token_per_placement

472bae4

NewDataTransportToken(parallel_desc)

cd32787

Merge branch 'master' into bugfix_data_transport_token_per_placement

e7b86a2

Merge branch 'master' into send_recv

1831c84

fix bugs

3bb7877

Merge branch 'master' of github.com:Oneflow-Inc/oneflow into bugfix_d…

757d023

…ata_transport_token_per_placement

Merge branch 'master' into bugfix_data_transport_token_per_placement

5b52ec5

Merge branch 'master' of github.com:Oneflow-Inc/oneflow into bugfix_d…

765b62a

…ata_transport_token_per_placement

rename UniqueConsistentIdStorage to ConsistentIdStorage

500e384

Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …

d61789d

…bugfix_data_transport_token_per_placement

Merge branch 'bugfix_data_transport_token_per_placement' of https://g…

9b7e424

…ithub.com/Oneflow-Inc/oneflow into bugfix_data_transport_token_per_placement Conflicts: oneflow/core/vm/oneflow_vm.cpp

Merge remote-tracking branch 'origin/master' into send_recv

21359a8

Merge remote-tracking branch 'origin/master' into send_recv

a39ffc0

Merge remote-tracking branch 'origin/bugfix_data_transport_token_per_…

2b70389

…placement' into send_recv

Merge remote-tracking branch 'origin/master' into send_recv

b75072e

refactor TransportToken

2749d2c

add TransportToken::src_rank and TransportToken::dst_rank

65fb518

merge data_token_per_thread

970f379

harded coded thread_consistent_id

39a447e

StreamType::SupportingTransportInstructions()

650fb0c

Merge branch 'master' of github.com:Oneflow-Inc/oneflow into token_se…

3776ab5

…q_id

merge master

d73fb5e

add send and recv

dcc9cf1

Signed-off-by: daquexian <daquexian566@gmail.com>

Merge remote-tracking branch 'origin/token_seq_id' into send_recv

6ae2efe

Signed-off-by: daquexian <daquexian566@gmail.com>

Merge branch 'master' of github.com:Oneflow-Inc/oneflow into token_se…

125c150

…q_id

not thread_consistent_id in single-client mode

04405e8

Merge branch 'master' of github.com:Oneflow-Inc/oneflow into token_se…

42db5fb

…q_id

clackhan and others added 9 commits September 1, 2021 18:53

Merge branch 'master' into eager_boxing_one_to_one

04715f7

refine

fc8a589

Merge branch 'master' into eager_boxing_one_to_one

3f4c290

del if branch in EagerBoxingInterpreterManager::GetEagerBoxingInterpr…

1def85c

…eter

del useless code

42f6247

rename naive_1_to_n.cpp

84d6322

del useless code

cdff1aa

Merge branch 'master' of https://github.com/Oneflow-Inc/oneflow into …

4898a2a

…refactor_BoxingInterpreter_to_BoxingExpr Conflicts: oneflow/core/framework/op_interpreter/boxing/eager_boxing_interpreter_mgr.cpp oneflow/core/framework/op_interpreter/boxing/naive_b2p_boxing_interpreter.cpp

clackhan added enhancement automerge system labels Sep 1, 2021

clackhan requested a review from lixinqi September 1, 2021 23:46

Update identity_boxing_interpreter.cpp

2e70f20

clackhan commented Sep 1, 2021

View reviewed changes

lixinqi reviewed Sep 2, 2021

View reviewed changes

clackhan added 2 commits September 2, 2021 09:33

revert EagerBoxingCall

74fca72

Merge branch 'refactor_BoxingInterpreter_to_BoxingExpr' of https://gi…

ffdf3f6

…thub.com/Oneflow-Inc/oneflow into refactor_BoxingInterpreter_to_BoxingExpr

clackhan requested a review from oneflow-ci-bot September 2, 2021 01:59

oneflow-ci-bot removed their request for review September 2, 2021 02:54

lixinqi approved these changes Sep 2, 2021

View reviewed changes

clackhan and others added 3 commits September 2, 2021 10:58

refine RawMainBoxingExpr

d38929e

Merge branch 'master' into refactor_BoxingInterpreter_to_BoxingExpr

3ac9eb7

Merge branch 'refactor_BoxingInterpreter_to_BoxingExpr' of https://gi…

04a1f0f

…thub.com/Oneflow-Inc/oneflow into refactor_BoxingInterpreter_to_BoxingExpr

clackhan requested a review from oneflow-ci-bot September 2, 2021 03:11

oneflow-ci-bot removed their request for review September 2, 2021 04:11

oneflow-ci-bot merged commit 487dc00 into master Sep 2, 2021

oneflow-ci-bot deleted the refactor_BoxingInterpreter_to_BoxingExpr branch September 2, 2021 04:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor boxing interpreter to boxing expr #6134

Refactor boxing interpreter to boxing expr #6134

clackhan commented Sep 1, 2021

clackhan Sep 1, 2021

lixinqi Sep 2, 2021

clackhan Sep 2, 2021

github-actions bot commented Sep 2, 2021

github-actions bot commented Sep 2, 2021

		if (tensor_nd_sbp == out->nd_sbp()) { return tensor; }
		// reset sbp if parallel_num == 1 and reset transport_token

Refactor boxing interpreter to boxing expr #6134

Refactor boxing interpreter to boxing expr #6134

Conversation

clackhan commented Sep 1, 2021

clackhan Sep 1, 2021

Choose a reason for hiding this comment

lixinqi Sep 2, 2021

Choose a reason for hiding this comment

clackhan Sep 2, 2021

Choose a reason for hiding this comment

github-actions bot commented Sep 2, 2021

github-actions bot commented Sep 2, 2021