Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: LazyInterpret::ApplyImpl support SourceUserOpExpr and Copy #5711

Merged
merged 12 commits into from
Aug 4, 2021

Conversation

chengtbf
Copy link
Contributor

@chengtbf chengtbf commented Aug 3, 2021

  • LazyInterpret 支持处理 SourceOp,SourceOp 的 is_local 和 parallel_desc 由 OpExprInterpContext 获得
  • OpExpr 的 Apply 方法扩展支持参数 placement 和 device
  • LazyInterpret 支持处理 Copy, Copy 将不会在 InferCtx 里 AddOp,仅生成对应属性的 output Tensor,且该 tensor 的 name 与 input Tensor 的 lbn 一致
  • OFRecordReader op 支持配置 parallel_distribution 属性,用于指定 op 推导输出的 sbp; dataset 中的 OFRecordReader module 支持根据参数 placement 和 device 来分别调用不同的新提供的 OpExpr::apply 接口
  • 调试通过了 OFRecordReader nn.Graph 版本的 module 单测
  • 调试通过了 nn.Graph 内支持 tensor.to("cuda")
  • 重写了 CropMirrorNormalize Module 里的 op 构造逻辑,修复了之前 op 数据类型 和 mirror 强绑定的 BUG
  • 支持 CoinFlip Module;

TODO: 支持 COCO Reader (@leaves-zwx

@oneflow-ci-bot oneflow-ci-bot requested review from oneflow-ci-bot and removed request for oneflow-ci-bot August 4, 2021 10:16
@github-actions
Copy link
Contributor

github-actions bot commented Aug 4, 2021

Speed stats:
GPU Name: GeForce GTX 1080 

PyTorch resnet50 time: 138.5ms (= 6927.3ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 125.7ms (= 6286.3ms / 50, input_shape=[16, 3, 224, 224], backward is enabled)
Relative speed: 1.10 (= 138.5ms / 125.7ms)

PyTorch resnet50 time: 84.0ms (= 4197.8ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 72.8ms (= 3640.4ms / 50, input_shape=[8, 3, 224, 224], backward is enabled)
Relative speed: 1.15 (= 84.0ms / 72.8ms)

PyTorch resnet50 time: 59.3ms (= 2963.4ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 49.9ms (= 2493.5ms / 50, input_shape=[4, 3, 224, 224], backward is enabled)
Relative speed: 1.19 (= 59.3ms / 49.9ms)

PyTorch resnet50 time: 48.5ms (= 2427.1ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 52.0ms (= 2601.9ms / 50, input_shape=[2, 3, 224, 224], backward is enabled)
Relative speed: 0.93 (= 48.5ms / 52.0ms)

PyTorch resnet50 time: 40.9ms (= 2043.8ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
OneFlow resnet50 time: 44.4ms (= 2220.2ms / 50, input_shape=[1, 3, 224, 224], backward is enabled)
Relative speed: 0.92 (= 40.9ms / 44.4ms)

@oneflow-ci-bot oneflow-ci-bot merged commit d37b07a into master Aug 4, 2021
@oneflow-ci-bot oneflow-ci-bot deleted the dev_cc_source_op_expr branch August 4, 2021 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants