Dev support oneflow insight #10370

Flowingsun007 · 2023-12-06T14:53:42Z

本Pr实现了OneFlow Insight模块，相关issue：https://github.com/Oneflow-Inc/OneTeam/issues/2162

当我们需要profiling cuda kernel执行时间/瓶颈分析时，通常会基于nvidia提供的nsys指令，生成对应的profile文件（早期的.qdrep以及现在的.nsys-rep）并用Nvidia的GUI软件Nsight Systems来可视化分析、查看。

在nsys生成profile文件的同时，还会生成平台无关的数据信息，记录在.sqlite文件中，OneFlow Insight模块就可以通过解析.sqlite，来生成符合Google Chrome Trace Event格式的JSON文件，使得可以直接通过Chrome或者Edge浏览器，通过chrome://tracing/ 或 edge://tracing/来解析和渲染此JSON文件，从而进行可视化分析、查看，效果如下：

github-actions · 2023-12-07T03:57:17Z

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

github-actions · 2023-12-07T04:00:28Z

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

github-actions · 2023-12-07T05:29:32Z

View latest API docs preview at: https://oneflow-staging.oss-cn-beijing.aliyuncs.com/docs/Oneflow-Inc/oneflow/pr/10370/

github-actions · 2023-12-07T05:33:55Z

Speed stats:

GPU Name: NVIDIA GeForce RTX 3080 Ti 

❌ OneFlow resnet50 time: 43.3ms (= 4328.7ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 58.0ms (= 5803.8ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.34 (= 58.0ms / 43.3ms)

OneFlow resnet50 time: 26.0ms (= 2604.8ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 37.8ms (= 3779.0ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.45 (= 37.8ms / 26.0ms)

OneFlow resnet50 time: 19.0ms (= 3792.2ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 35.4ms (= 7074.9ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.87 (= 35.4ms / 19.0ms)

OneFlow resnet50 time: 17.4ms (= 3482.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 33.6ms (= 6718.9ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.93 (= 33.6ms / 17.4ms)

OneFlow resnet50 time: 17.2ms (= 3438.3ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 28.4ms (= 5670.4ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.65 (= 28.4ms / 17.2ms)

OneFlow swin dataloader time: 0.200s (= 39.968s / 200, num_workers=1)
PyTorch swin dataloader time: 0.129s (= 25.834s / 200, num_workers=1)
Relative speed: 0.646 (= 0.129s / 0.200s)

OneFlow swin dataloader time: 0.054s (= 10.896s / 200, num_workers=4)
PyTorch swin dataloader time: 0.033s (= 6.551s / 200, num_workers=4)
Relative speed: 0.601 (= 0.033s / 0.054s)

OneFlow swin dataloader time: 0.038s (= 7.528s / 200, num_workers=8)
PyTorch swin dataloader time: 0.021s (= 4.282s / 200, num_workers=8)
Relative speed: 0.569 (= 0.021s / 0.038s)

❌ OneFlow resnet50 time: 48.8ms (= 4878.4ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 66.1ms (= 6611.1ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.36 (= 66.1ms / 48.8ms)

OneFlow resnet50 time: 36.8ms (= 3675.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 47.8ms (= 4779.1ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.30 (= 47.8ms / 36.8ms)

OneFlow resnet50 time: 28.1ms (= 5621.1ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 39.3ms (= 7853.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.40 (= 39.3ms / 28.1ms)

OneFlow resnet50 time: 26.0ms (= 5197.0ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 40.2ms (= 8035.3ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.55 (= 40.2ms / 26.0ms)

OneFlow resnet50 time: 23.8ms (= 4758.9ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 38.1ms (= 7625.0ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.60 (= 38.1ms / 23.8ms)

github-actions · 2023-12-07T07:36:49Z

View latest API docs preview at: https://oneflow-staging.oss-cn-beijing.aliyuncs.com/docs/Oneflow-Inc/oneflow/pr/10370/

github-actions · 2023-12-07T08:05:51Z

Speed stats:

python/oneflow/utils/insight/requirements.txt

python/oneflow/utils/insight/sqlite_to_google_trace_event.py

github-actions · 2023-12-21T05:12:23Z

Code got formatted by CI. Please request CI again if you still want to have this PR merged. If the PR is from a forked repo, please download the patch files from the GitHub Actions web page and apply them locally.

github-actions · 2023-12-21T06:25:41Z

Speed stats:

GPU Name: NVIDIA GeForce RTX 3080 Ti 

❌ OneFlow resnet50 time: 43.3ms (= 4326.7ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 58.2ms (= 5816.9ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.34 (= 58.2ms / 43.3ms)

OneFlow resnet50 time: 26.5ms (= 2652.6ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 37.3ms (= 3734.5ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.41 (= 37.3ms / 26.5ms)

OneFlow resnet50 time: 17.7ms (= 3540.7ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 34.6ms (= 6913.1ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.95 (= 34.6ms / 17.7ms)

OneFlow resnet50 time: 16.4ms (= 3289.4ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 31.9ms (= 6377.7ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.94 (= 31.9ms / 16.4ms)

OneFlow resnet50 time: 15.8ms (= 3159.2ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 28.5ms (= 5707.9ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.81 (= 28.5ms / 15.8ms)

OneFlow swin dataloader time: 0.200s (= 39.980s / 200, num_workers=1)
PyTorch swin dataloader time: 0.128s (= 25.590s / 200, num_workers=1)
Relative speed: 0.640 (= 0.128s / 0.200s)

OneFlow swin dataloader time: 0.054s (= 10.820s / 200, num_workers=4)
PyTorch swin dataloader time: 0.033s (= 6.584s / 200, num_workers=4)
Relative speed: 0.609 (= 0.033s / 0.054s)

OneFlow swin dataloader time: 0.030s (= 5.985s / 200, num_workers=8)
PyTorch swin dataloader time: 0.017s (= 3.407s / 200, num_workers=8)
Relative speed: 0.569 (= 0.017s / 0.030s)

❌ OneFlow resnet50 time: 48.9ms (= 4889.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 64.1ms (= 6406.7ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.31 (= 64.1ms / 48.9ms)

OneFlow resnet50 time: 37.0ms (= 3703.1ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 46.5ms (= 4652.0ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.26 (= 46.5ms / 37.0ms)

OneFlow resnet50 time: 28.6ms (= 5717.8ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 40.3ms (= 8064.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.41 (= 40.3ms / 28.6ms)

OneFlow resnet50 time: 25.1ms (= 5029.4ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 38.7ms (= 7734.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.54 (= 38.7ms / 25.1ms)

OneFlow resnet50 time: 24.1ms (= 4825.3ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 37.6ms (= 7526.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.56 (= 37.6ms / 24.1ms)

…ort_oneflow_insight

github-actions · 2023-12-29T08:16:36Z

View latest API docs preview at: https://oneflow-staging.oss-cn-beijing.aliyuncs.com/docs/Oneflow-Inc/oneflow/pr/10370/

github-actions · 2023-12-29T08:56:18Z

Speed stats:

GPU Name: NVIDIA GeForce RTX 3080 Ti 

❌ OneFlow resnet50 time: 43.3ms (= 4332.7ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 57.5ms (= 5746.9ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.33 (= 57.5ms / 43.3ms)

OneFlow resnet50 time: 26.4ms (= 2637.3ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 37.7ms (= 3772.9ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.43 (= 37.7ms / 26.4ms)

OneFlow resnet50 time: 18.2ms (= 3644.2ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 35.2ms (= 7044.7ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.93 (= 35.2ms / 18.2ms)

OneFlow resnet50 time: 17.7ms (= 3540.9ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 33.6ms (= 6720.3ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.90 (= 33.6ms / 17.7ms)

OneFlow resnet50 time: 17.1ms (= 3420.4ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 31.4ms (= 6273.4ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.83 (= 31.4ms / 17.1ms)

OneFlow swin dataloader time: 0.200s (= 40.007s / 200, num_workers=1)
PyTorch swin dataloader time: 0.128s (= 25.501s / 200, num_workers=1)
Relative speed: 0.637 (= 0.128s / 0.200s)

OneFlow swin dataloader time: 0.057s (= 11.356s / 200, num_workers=4)
PyTorch swin dataloader time: 0.033s (= 6.500s / 200, num_workers=4)
Relative speed: 0.572 (= 0.033s / 0.057s)

OneFlow swin dataloader time: 0.031s (= 6.222s / 200, num_workers=8)
PyTorch swin dataloader time: 0.017s (= 3.362s / 200, num_workers=8)
Relative speed: 0.540 (= 0.017s / 0.031s)

❌ OneFlow resnet50 time: 49.3ms (= 4926.0ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 65.1ms (= 6514.6ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.32 (= 65.1ms / 49.3ms)

OneFlow resnet50 time: 36.5ms (= 3651.2ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 46.0ms (= 4601.5ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.26 (= 46.0ms / 36.5ms)

OneFlow resnet50 time: 28.2ms (= 5639.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 39.5ms (= 7893.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.40 (= 39.5ms / 28.2ms)

OneFlow resnet50 time: 25.0ms (= 4998.4ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 39.3ms (= 7858.5ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.57 (= 39.3ms / 25.0ms)

OneFlow resnet50 time: 24.1ms (= 4818.4ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 36.3ms (= 7257.2ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.51 (= 36.3ms / 24.1ms)

github-actions · 2023-12-29T09:40:41Z

Speed stats:

GPU Name: NVIDIA GeForce RTX 3080 Ti 

❌ OneFlow resnet50 time: 43.5ms (= 4352.2ms / 100, input_shape=[16, 3, 224, 224])
PyTorch resnet50 time: 57.7ms (= 5774.8ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.33 (= 57.7ms / 43.5ms)

OneFlow resnet50 time: 26.1ms (= 2610.7ms / 100, input_shape=[8, 3, 224, 224])
PyTorch resnet50 time: 37.2ms (= 3723.9ms / 100, input_shape=[8, 3, 224, 224])
✔️ Relative speed: 1.43 (= 37.2ms / 26.1ms)

OneFlow resnet50 time: 18.5ms (= 3690.9ms / 200, input_shape=[4, 3, 224, 224])
PyTorch resnet50 time: 35.5ms (= 7096.3ms / 200, input_shape=[4, 3, 224, 224])
✔️ Relative speed: 1.92 (= 35.5ms / 18.5ms)

OneFlow resnet50 time: 17.4ms (= 3485.2ms / 200, input_shape=[2, 3, 224, 224])
PyTorch resnet50 time: 31.4ms (= 6287.8ms / 200, input_shape=[2, 3, 224, 224])
✔️ Relative speed: 1.80 (= 31.4ms / 17.4ms)

OneFlow resnet50 time: 17.2ms (= 3436.3ms / 200, input_shape=[1, 3, 224, 224])
PyTorch resnet50 time: 29.0ms (= 5791.2ms / 200, input_shape=[1, 3, 224, 224])
✔️ Relative speed: 1.69 (= 29.0ms / 17.2ms)

OneFlow swin dataloader time: 0.199s (= 39.898s / 200, num_workers=1)
PyTorch swin dataloader time: 0.128s (= 25.656s / 200, num_workers=1)
Relative speed: 0.643 (= 0.128s / 0.199s)

OneFlow swin dataloader time: 0.055s (= 11.000s / 200, num_workers=4)
PyTorch swin dataloader time: 0.033s (= 6.584s / 200, num_workers=4)
Relative speed: 0.598 (= 0.033s / 0.055s)

OneFlow swin dataloader time: 0.029s (= 5.845s / 200, num_workers=8)
PyTorch swin dataloader time: 0.017s (= 3.337s / 200, num_workers=8)
Relative speed: 0.571 (= 0.017s / 0.029s)

❌ OneFlow resnet50 time: 49.4ms (= 4939.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 70.2ms (= 7016.3ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.42 (= 70.2ms / 49.4ms)

OneFlow resnet50 time: 36.9ms (= 3689.8ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 46.4ms (= 4638.0ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.26 (= 46.4ms / 36.9ms)

OneFlow resnet50 time: 28.0ms (= 5607.3ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 40.0ms (= 7996.6ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.43 (= 40.0ms / 28.0ms)

OneFlow resnet50 time: 25.0ms (= 4996.7ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 38.9ms (= 7782.2ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.56 (= 38.9ms / 25.0ms)

OneFlow resnet50 time: 24.2ms (= 4842.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 36.2ms (= 7236.6ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.49 (= 36.2ms / 24.2ms)

Flowingsun007 added 2 commits December 6, 2023 14:51

support oneflow insight

ca6c886

add requirements.txt

3ef8096

Flowingsun007 added enhancement feature eager labels Dec 6, 2023

add readme

afa097c

Flowingsun007 marked this pull request as ready for review December 7, 2023 03:56

auto format by CI

df8d242

Flowingsun007 requested review from mosout and levi131 December 7, 2023 03:58

auto format by CI

0d24701

Merge branch 'master' into dev_support_oneflow_insight

1a7185d

Flowingsun007 enabled auto-merge (squash) December 7, 2023 06:16

Flowingsun007 requested a review from oneflow-ci-bot December 7, 2023 06:17

mosout reviewed Dec 21, 2023

View reviewed changes

python/oneflow/utils/insight/requirements.txt Outdated Show resolved Hide resolved

python/oneflow/utils/insight/sqlite_to_google_trace_event.py Outdated Show resolved Hide resolved

python/oneflow/utils/insight/sqlite_to_google_trace_event.py Outdated Show resolved Hide resolved

Flowingsun007 and others added 4 commits December 21, 2023 04:53

Merge branch 'master' into dev_support_oneflow_insight

ac23a59

refine

98fd989

refine

706f8a4

auto format by CI

bd9f330

mosout added 3 commits December 27, 2023 12:27

Merge branch 'master' of github.com:Oneflow-Inc/oneflow into dev_supp…

6a8d508

…ort_oneflow_insight

refine doc

2bf8ab4

Merge branch 'master' of github.com:Oneflow-Inc/oneflow into dev_supp…

13698a7

…ort_oneflow_insight

mosout approved these changes Dec 29, 2023

View reviewed changes

Flowingsun007 merged commit 82c965b into master Dec 29, 2023

Flowingsun007 deleted the dev_support_oneflow_insight branch December 29, 2023 09:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dev support oneflow insight #10370

Dev support oneflow insight #10370

Flowingsun007 commented Dec 6, 2023 •

edited

Loading

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 21, 2023

github-actions bot commented Dec 21, 2023

github-actions bot commented Dec 29, 2023

github-actions bot commented Dec 29, 2023

github-actions bot commented Dec 29, 2023

Dev support oneflow insight #10370

Dev support oneflow insight #10370

Conversation

Flowingsun007 commented Dec 6, 2023 • edited Loading

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 7, 2023

github-actions bot commented Dec 21, 2023

github-actions bot commented Dec 21, 2023

github-actions bot commented Dec 29, 2023

github-actions bot commented Dec 29, 2023

github-actions bot commented Dec 29, 2023

Flowingsun007 commented Dec 6, 2023 •

edited

Loading