Skip to content

Commit f574307

Browse files
remlovehremloveh
committed
9.19
1 parent 67a590d commit f574307

File tree

3 files changed

+77
-10
lines changed

3 files changed

+77
-10
lines changed

‎docs/mrt/model.md

Lines changed: 75 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,14 @@
44

55
Some models have been compiled and the Accuracy and size of params is provided in the following chart for reference.
66

7-
| Model Name | Category | (Top1) Accuracy<br />(original / quantized) |
7+
| Model Name | Category | (Top1) Accuracy<br />(original / quantized / diff) |
88
| ------------------------- | -------------- | ------------------------------------------- |
9-
| ssd_512_mobilenet1.0_coco | detection | 21.50% / 15.60% |
10-
| ssd_512_resnet50_v1_voc | detection | 80.27% / 80.01% |
11-
| yolo3_darknet53_voc | detection | 81.37% / 82.08% |
12-
| shufflenet_v1 | classification | 63.48% / 60.45% |
13-
| mobilenet1_0 | classification | 70.77% / 66.11% |
14-
| mobilenetv2_1.0 | classification | 71.51% / 69.39% |
9+
| ssd_512_mobilenet1.0_coco | detection | 21.50% / 15.60% / 5.9% |
10+
| ssd_512_resnet50_v1_voc | detection | 80.27% / 80.01% / 0.26% |
11+
| yolo3_darknet53_voc | detection | 81.37% / 82.08% / -0.71% |
12+
| shufflenet_v1 | classification | 63.48% / 60.45% / 3.03% |
13+
| mobilenet1_0 | classification | 70.77% / 66.11% / 4.66% |
14+
| mobilenetv2_1.0 | classification | 71.51% / 69.39% / 2.12% |
1515

1616
| Model Name | Params Size | Path |
1717
| ------------------------- | ----------- | --------------------------------------- |
@@ -88,4 +88,71 @@ Some models have been tested and the precision before and after quantization is
8888
| yolo3_mobilenet1.0_voc | 29 | 75.98% | 71.53% | 4800 |
8989
| ssd_512_resnet50_v1_voc | 29 | 80.27% | 80.01% | 4800 |
9090
| ssd_512_mobilenet1.0_voc | 29 | 75.57% | 71.32% | 4800 |
91-
| mnist | 62 | top1=99.18%<br />top5=100% | 99.17%<br />top5=100% | 10000 |
91+
| mnist | 62 | top1=99.18%<br />top5=100% | top1=99.17%<br />top5=100% | 10000 |
92+
93+
94+
| model name | Iteration | evalfunc | quantize | time(ms) | cvm | total sample |
95+
| ------------------------ | --------- | ---------------------------- | ---------------------------- | --------- | ------------- | ------------ |
96+
| cifar_resnet20_v1 | 62 | top1=92.88%<br />top5=99.78% | top1=92.82%<br />top5=99.75% | 135 211 | 92.80% 12ms | 10080 |
97+
| cifar_resnet20_v2 | 62 | top1=92.68%<br />top5=99.77% | top1=92.39%<br />top5=99.77% | 122 209 | 92.39% 14ms | 10080 |
98+
| cifar_resnet56_v1 | 62 | top1=94.20%<br />top5=99.83% | top1=94.16%<br />top5=99.82% | 122 379 | 94.21% 26ms | 10080 |
99+
| cifar_resnet56_v2 | 62 | top1=94.64%<br />top5=99.89% | top1=94.48%<br />top5=99.87% | 90 300 | 94.48% 33ms | 10080 |
100+
| cifar_resnet110_v1 | 62 | top1=95.20%<br />top5=99.83% | top1=95.12%<br />top5=99.83% | 117 410 | 95.09% 45ms | 10080 |
101+
| cifar_resnet110_v2 | 62 | top1=95.54%<br />top5=99.84% | top1=95.31%<br />top5=99.82% | 109 707 | 95.26% 58ms | 10080 |
102+
| cifar_wideresnet16_10 | 62 | top1=96.71%<br />top5=99.91% | top1=96.59%<br />top5=99.91% | 851 887 | 96.58% 112ms | 10080 |
103+
| cifar_wideresnet28_10 | 62 | top1=97.12%<br />top5=99.92% | top1=96.99%<br />top5=99.89% | 876 1016 | 96.99% 204ms | 10080 |
104+
| cifar_wideresnet40_8 | 62 | top1=97.27%<br />top5=99.92% | top1=97.03%<br />top5=99.93% | 636 904 | 97.04% 218ms | 10080 |
105+
| alexnet | 312 | top1=55.89%<br />top5=78.75% | top1=51.48%<br />top5=77.39% | 25 114 | 54.98% 69ms | 50080 |
106+
| densenet121 | 999 | top1=74.94%<br />top5=92.17% | top1=72.67%<br />top5=91.04% | 20 848 | 73.09% 473ms | 16000 |
107+
| densenet161 | 312 | top1=77.62%<br />top5=93.82% | top1=77.27%<br />top5=93.60% | 425 3691 | 73.38% 898ms | 50080 |
108+
| densenet169 | 999 | top1=76.29%<br />top5=92.98% | top1=75.63%<br />top5=92.77% | 27 851 | 72.84% 591ms | 16000 |
109+
| densenet201 | 999 | top1=77.39%<br />top5=93.37% | top1=73.96%<br />top5=91.88% | 40 1152 | 73.96% 756ms | 16000 |
110+
| mobilenet0.5 | 312 | top1=65.22%<br />top5=86.35% | top1=59.70%<br />top5=82.46% | 34 218 | 59.62% 50ms | 50080 |
111+
| mobilenet0.75 | 312 | top1=70.27%<br />top5=89.49% | top1=66.25%<br />top5=87.06% | 48 261 | 64.38% 77ms | 50080 |
112+
| mobilenet1_0 | 312 | top1=73.29%<br />top5=91.30% | top1=64.64%<br />top5=88.34% | 58 330 | 65.20% 99ms | 50080 |
113+
| mobilenetv2_0.5 | 312 | top1=64.43%<br />top5=85.32% | top1=61.65%<br />top5=83.60% | 39 286 | 61.48% 66ms | 50080 |
114+
| mobilenetv2_0.75 | 312 | top1=69.38%<br />top5=88.50% | top1=65.50%<br />top5=86.05% | 53 331 | 66.16% 92ms | 50080 |
115+
| mobilenetv2_1.0 | 312 | top1=72.05%<br />top5=90.58% | top1=69.79%<br />top5=89.14% | 68 409 | 69.66% 117ms | 50080 |
116+
| resnet18_v1 | 312 | top1=70.96%<br />top5=89.93% | top1=70.11%<br />top5=89.61% | 54 276 | 69.95% 102ms | 50080 |
117+
| resnet18_v1b_0.89 | 312 | top1=67.20%<br />top5=87.45% | top1=63.78%<br />top5=85.62% | 33 193 | 63.63% 61ms | 50080 |
118+
| resnet18_v1b | 312 | top1=70.95%<br />top5=89.85% | top1=69.39%<br />top5=89.52% | 56 271 | 70.40% 104ms | 50080 |
119+
| resnet34_v1 | 312 | top1=74.39%<br />top5=91.88% | top1=73.75%<br />top5=91.71% | 87 486 | 73.94% 181ms | 50080 |
120+
| resnet34_v1b | 312 | top1=74.67%<br />top5=92.08% | top1=74.17%<br />top5=91.85% | 88 389 | 74.48% 181ms | 50080 |
121+
| resnet34_v2 | 312 | top1=74.43%<br />top5=92.11% | top1=73.46%<br />top5=91.55% | 92 559 | 73.88% 244ms | 50080 |
122+
| resnet50_v1 | 312 | top1=77.39%<br />top5=93.59% | top1=76.45%<br />top5=93.27% | 168 813 | 76.23% 250ms | 50080 |
123+
| resnet50_v1b | 312 | top1=77.69%<br />top5=93.83% | top1=76.55%<br />top5=93.35% | 161 867 | 76.94% 273ms | 50080 |
124+
| resnet50_v1c | 312 | top1=78.05%<br />top5=94.11% | top1=77.48%<br />top5=93.80% | 173 911 | 78.02% 292ms | 50080 |
125+
| resnet50_v1d_0.11 | 312 | top1=63.22%<br />top5=84.79% | top1=61.14%<br />top5=83.40% | 30 324 | 61.69% 47ms | 50080 |
126+
| resnet50_v1d_0.37 | 312 | top1=70.72%<br />top5=89.75% | top1=68.08%<br />top5=88.12% | 36 318 | 69.65% 73ms | 50080 |
127+
| resnet50_v1d_0.48 | 312 | top1=74.68%<br />top5=92.35% | top1=72.26%<br />top5=91.19% | 54 428 | 73.90% 108ms | 50080 |
128+
| resnet50_v1d_0.86 | 312 | top1=78.03%<br />top5=93.83% | top1=76.42%<br />top5=93.32% | 105 582 | 75.37% 176ms | 50080 |
129+
| resnet50_v1d | 312 | top1=79.19%<br />top5=94.59% | top1=78.26%<br />top5=94.22% | 170 986 | 77.25% 285ms | 50080 |
130+
| resnet50_v1s | 312 | top1=78.89%<br />top5=94.36% | top1=78.48%<br />top5=94.18% | 200 936 | 76.25% 352ms | 50080 |
131+
| resnet50_v2 | 312 | top1=77.15%<br />top5=93.44% | top1=74.15%<br />top5=91.74% | 171 1339 | 75.13% 415ms | 50080 |
132+
| resnet101_v1 | 312 | top1=78.36%<br />top5=94.01% | top1=77.62%<br />top5=93.64% | 277 1334 | 77.62% 430ms | 50080 |
133+
| resnet101_v1b | 312 | top1=79.23%<br />top5=94.62% | top1=77.68%<br />top5=94.00% | 263 1378 | 75.81% 444ms | 50080 |
134+
| resnet101_v1c | 312 | top1=79.62%<br />top5=94.77% | top1=78.14%<br />top5=94.25% | 273 1432 | 77.67% 479ms | 50080 |
135+
| resnet101_v1d_0.73 | 312 | top1=78.92%<br />top5=94.49% | top1=74.32%<br />top5=92.97% | 138 841 | 75.14% 295ms | 50080 |
136+
| resnet101_v1d_0.76 | 312 | top1=79.48%<br />top5=94.70% | top1=75.70%<br />top5=93.71% | 153 878 | 78.10% 310ms | 50080 |
137+
| resnet101_v1d | 312 | top1=80.55%<br />top5=95.13% | top1=77.49%<br />top5=94.44% | 272 1513 | 77.15% 489ms | 50080 |
138+
| resnet101_v1s | 312 | top1=80.34%<br />top5=95.25% | top1=79.75%<br />top5=95.06% | 302 1530 | 78.23% 519ms | 50080 |
139+
| resnet152_v1 | 312 | top1=79.25%<br />top5=94.65% | top1=78.56%<br />top5=94.38% | 417 1964 | 78.12% 641ms | 50080 |
140+
| resnet152_v1b | 312 | top1=79.70%<br />top5=94.75% | top1=78.77%<br />top5=94.36% | 373 2007 | 75.13% 650ms | 50080 |
141+
| resnet152_v1c | 312 | top1=80.05%<br />top5=94.96% | top1=79.22%<br />top5=94.66% | 381 2057 | 76.19% 714ms | 50080 |
142+
| resnet152_v1d | 312 | top1=80.63%<br />top5=95.36% | top1=78.76%<br />top5=94.84% | 383 2058 | 77.21% 675ms | 50080 |
143+
| resnet152_v1s | 312 | top1=81.13%<br />top5=95.55% | top1=80.81%<br />top5=95.38% | 410 2117 | 79.32% 716ms | 50080 |
144+
| resnet152_v2 | 312 | top1=79.26%<br />top5=94.66% | top1=78.56%<br />top5=94.38% | 397 2126 | 77.45% 1012ms | 50080 |
145+
| squeezenet1.0 | 312 | top1=57.19%<br />top5=80.04% | top1=54.91%<br />top5=78.64% | 70 331 | 55.38% 89ms | 50080 |
146+
| squeezenet1.1 | 312 | top1=56.96%<br />top5=79.77% | top1=52.62%<br />top5=78.04% | 43 259 | 55.73% 55ms | 50080 |
147+
| vgg11_bn | 3124 | top1=69.66%<br />top5=89.43% | top1=67.99%<br />top5=88.46% | 18 60 | 66.37% 299ms | 50000 |
148+
| vgg11 | 3124 | top1=68.10%<br />top5=88.25% | top1=66.37%<br />top5=87.69% | 16 60 | 66.13% 315ms | 50000 |
149+
| vgg13_bn | 3124 | top1=70.52%<br />top5=89.84% | top1=68.75%<br />top5=88.82% | 27 80 | 68.56% 433ms | 50000 |
150+
| vgg13 | 3124 | top1=68.94%<br />top5=88.88% | top1=68.47%<br />top5=88.66% | 24 82 | 66.39% 452ms | 50000 |
151+
| vgg16_bn | 3124 | top1=73.12%<br />top5=91.35% | top1=72.23%<br />top5=90.95% | 32 91 | 71.17% 536ms | 50000 |
152+
| vgg16 | 3124 | top1=73.22%<br />top5=91.31% | top1=71.50%<br />top5=91.20% | 28 83 | 72.99% 532ms | 50000 |
153+
| vgg19_bn | 781 | top1=74.35%<br />top5=91.86% | top1=73.68%<br />top5=91.61% | 143 290 | 73.02% 640ms | 50048 |
154+
| vgg19 | 781 | top1=74.13%<br />top5=91.77% | top1=73.28%<br />top5=91.52% | 130 326 | 73.78% 628ms | 50048 |
155+
| ssd_512_mobilenet1.0_voc | 308 | 75.51% | 71.26% | 179 425 | | 4944 |
156+
| ssd_512_resnet50_v1_voc | 76 | 80.30% | 80.05% | 480 2028 | | 4928 |
157+
| yolo3_darknet53_voc | 102 | 81.51% | 81.51% | 333 2537 | | 4944 |
158+
| yolo3_mobilenet1.0_voc | 76 | 76.03% | 71.56% | 265 1135 | | 4928 |

‎tests/mrt/model_zoo/cifar10/cifar_resnet110_v2.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
COMMON:
2-
MODEL_NAME: cifar_resnet20_v2
2+
MODEL_NAME: cifar_resnet110_v2
33
PREPARE:
44
INPUT_SHAPE: [-1,3,32,32]
55
CALIBRATE:

‎tests/mrt/model_zoo/imagenet/resnet152_v2.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
COMMON:
2-
MODEL_NAME: resnet152_v1
2+
MODEL_NAME: resnet152_v2
33
VERBOSITY: debug
44
RUN_EVALUATE: True
55
CALIBRATE:

0 commit comments

Comments
 (0)