05.srt

1
00:00:00,975 --> 00:00:03,639
字幕生成：慎独    校对：游吟外星人

2
00:00:04,929 --> 00:00:06,560
Hello，大家好

3
00:00:06,600 --> 00:00:09,680
我是五一终于有空过来更新的ZOMI了

4
00:00:09,720 --> 00:00:13,240
现在还是在AI芯片里面的GPU详解

5
00:00:13,280 --> 00:00:16,760
这里面的GPU主要是特指英伟达的GPU

6
00:00:16,975 --> 00:00:18,858
在上一节其实已经给大家汇报了

7
00:00:18,858 --> 00:00:22,138
从Fermi到Volta架构的具体的详细的内容

8
00:00:22,178 --> 00:00:24,578
今天来从Turing到Hopper架构

9
00:00:24,618 --> 00:00:26,978
去看看具体有什么不一样

10
00:00:27,178 --> 00:00:30,418
今天主要去分开几个内容给大家介绍的

11
00:00:30,538 --> 00:00:32,138
第一个就是从Turing的架构

12
00:00:32,178 --> 00:00:33,978
Ampere的架构到Hopper的架构

13
00:00:34,018 --> 00:00:35,858
主要讲解这三个架构

14
00:00:35,898 --> 00:00:40,587
这三个架构也是近5年来才慢慢的出现的

15
00:00:40,587 --> 00:00:43,858
Turing架构更多的是应用在游戏领域

16
00:00:43,858 --> 00:00:46,075
而Ampere架构对于AI训练

17
00:00:46,075 --> 00:00:48,618
或者AI的从业者来说是用的最多的

18
00:00:48,618 --> 00:00:51,178
这个系列里面很重要的有几个内容

19
00:00:51,178 --> 00:00:54,858
就是Tensor Core迎来了3.0就第三代了

20
00:00:54,898 --> 00:00:57,698
而NVLink也迎来了第三代了

21
00:00:57,738 --> 00:01:00,844
最后我至少我现在还拿不到货的

22
00:01:00,844 --> 00:01:02,170
就是Hopper的架构

23
00:01:02,170 --> 00:01:04,258
Hopper架构其实非常的惊艳

24
00:01:04,258 --> 00:01:07,378
除了使用了4nm的制程以外

25
00:01:07,418 --> 00:01:10,172
里面的晶体管对比起Ampere架构

26
00:01:10,172 --> 00:01:11,648
翻了接近三倍

27
00:01:11,756 --> 00:01:14,218
整个架构的设计也是非常惊艳

28
00:01:14,218 --> 00:01:16,898
后面会给大家去介绍展开的

29
00:01:17,098 --> 00:01:18,218
回到今天内容

30
00:01:18,218 --> 00:01:20,578
今天主要是回顾一下Volta架构

31
00:01:20,578 --> 00:01:23,978
Volta架构是第一代的Tensor Core

32
00:01:23,978 --> 00:01:25,508
通过第一代的Tensor Core

33
00:01:25,508 --> 00:01:27,258
更好的支持了AI的运算

34
00:01:27,258 --> 00:01:30,938
后面每一代架构都会对Tensor Core进行迭代

35
00:01:30,938 --> 00:01:31,935
在2018年

36
00:01:31,935 --> 00:01:35,658
Turing架构更多的是一个RT Core光线追踪的核心

37
00:01:35,658 --> 00:01:38,778
所以我说它主要是用在游戏领域

38
00:01:38,778 --> 00:01:40,858
后来有了Ampere架构和Hopper架构

39
00:01:40,858 --> 00:01:42,578
今天来详细的看看

40
00:01:42,618 --> 00:01:46,098
这4个架构或者后面三个架构有什么不一样

41
00:01:47,808 --> 00:01:48,702
英伟达的教主（皮衣刀客）

42
00:01:48,702 --> 00:01:51,458
黄仁勋就发布了Turing架构

43
00:01:51,618 --> 00:01:53,978
这一代架构确实也是非常的惊艳

44
00:01:53,978 --> 00:01:57,298
Turing架构里面有两个主要的更新

45
00:01:57,298 --> 00:01:59,498
第一个更新就是Tensor Core

46
00:01:59,498 --> 00:02:00,658
新一代的Tensor Core

47
00:02:00,658 --> 00:02:04,378
第二个更新就是引入了RT Core光线追踪

48
00:02:04,378 --> 00:02:06,938
而这里面这两代的架构很有意思的

49
00:02:06,938 --> 00:02:10,978
基本上都引入到了消费级的显卡

50
00:02:10,978 --> 00:02:13,855
也就是GTX系列里面

51
00:02:13,978 --> 00:02:15,938
第一个内容就是刚才说到的

52
00:02:15,938 --> 00:02:17,978
Turing架构引入了Tensor Core

53
00:02:18,258 --> 00:02:19,883
这个Tensor Core新增了

54
00:02:19,883 --> 00:02:22,578
INT8到INT4的支持

55
00:02:22,578 --> 00:02:26,578
更好的为深度学习的推理引擎进行加速

56
00:02:27,117 --> 00:02:29,418
第二个内容就是RT Core了

57
00:02:29,418 --> 00:02:33,138
RT就是Ray Traced光线追踪的核心

58
00:02:33,138 --> 00:02:36,768
主要是用来做一些三角形和光线的求交

59
00:02:37,033 --> 00:02:39,596
这个RT Core是在Block之外的

60
00:02:39,596 --> 00:02:43,058
相对于ALU，INT32，FP32这种计算来说

61
00:02:43,058 --> 00:02:45,298
两者之间是异步的关系

62
00:02:45,554 --> 00:02:47,098
现在简单的看看RT Core

63
00:02:47,098 --> 00:02:50,418
RT Core分为左边的框框和右边的框框

64
00:02:50,498 --> 00:02:53,258
左边的框框一部分是用来做碰撞的检测

65
00:02:53,258 --> 00:02:56,218
另外一部分就真正的去求交面的

66
00:02:56,498 --> 00:02:57,858
下面来看一下

67
00:02:57,858 --> 00:03:01,578
有RT Core和没有RT Core的一个不同点

68
00:03:01,778 --> 00:03:03,538
上面就是没有开RT Core的

69
00:03:03,538 --> 00:03:05,058
下面就是开了RT Core

70
00:03:05,058 --> 00:03:09,058
可以看到这里面的人物的光线是更进一步的

71
00:03:09,058 --> 00:03:10,809
就看出来更加逼真

72
00:03:10,809 --> 00:03:12,698
右边的没有RT Core的时候

73
00:03:12,698 --> 00:03:13,898
就没有做光线追踪

74
00:03:13,898 --> 00:03:16,804
这里有fire，有火焰的时候

75
00:03:16,804 --> 00:03:19,098
其实在车上是没有反映出来的

76
00:03:19,138 --> 00:03:20,704
而开了RT Core之后

77
00:03:20,704 --> 00:03:24,418
整个光线的反照是变得非常之有意思的

78
00:03:24,418 --> 00:03:25,858
非常之真实逼真

79
00:03:25,858 --> 00:03:30,298
而这里面连车头车灯这些反光全都映射出来

80
00:03:30,298 --> 00:03:33,589
这就是光线追踪非常有意思的一个话题点

81
00:03:33,589 --> 00:03:37,178
光线追踪也是英伟达最擅长的事情了

82
00:03:37,338 --> 00:03:38,507
下面看一看

83
00:03:38,699 --> 00:03:39,760
Turing架构出现在

84
00:03:39,760 --> 00:03:42,818
RTX2090，3090到40X系列

85
00:03:42,818 --> 00:03:45,298
就是40系列到现在能够卖的

86
00:03:45,298 --> 00:03:47,698
整体来说我个人是非常喜欢的

87
00:03:47,698 --> 00:03:51,138
因为真正的把AI变成一个消费级的显卡

88
00:03:51,138 --> 00:03:54,418
像我这种下班就没有更好的卡去用的时候

89
00:03:54,418 --> 00:03:56,618
我在家里还可以去玩一玩

90
00:03:57,495 --> 00:04:00,698
下面来到了Ampere架构（安倍架构）

91
00:04:00,698 --> 00:04:03,351
安倍架构也是2020年推出的

92
00:04:03,351 --> 00:04:04,738
三年前的一个架构

93
00:04:04,738 --> 00:04:07,938
其实现在在市场上还是抢断货的

94
00:04:08,443 --> 00:04:10,378
所以说安倍架构在AI领域

95
00:04:10,378 --> 00:04:12,578
还是非常经得住考验的

96
00:04:12,578 --> 00:04:15,923
现在看看安倍架构有哪些主要的特点

97
00:04:15,923 --> 00:04:16,757
首先第一个特点

98
00:04:16,757 --> 00:04:21,138
就是有超过540亿个晶体管所组成

99
00:04:21,138 --> 00:04:24,018
也就是当时候应该是2020年的时候

100
00:04:24,018 --> 00:04:27,498
世界上最大的7nm的处理器

101
00:04:27,498 --> 00:04:30,037
就是英伟达的A100

102
00:04:30,037 --> 00:04:30,893
而第二个（特点）

103
00:04:30,893 --> 00:04:33,498
就是新增了第3代的Tensor Core

104
00:04:33,498 --> 00:04:37,378
这里面包括新增了一个特别有意思的数据位

105
00:04:37,378 --> 00:04:40,778
就是TF32专门针对AI进行加速

106
00:04:40,778 --> 00:04:42,258
后面三个我觉得很重要

107
00:04:42,258 --> 00:04:43,658
简单的列一列

108
00:04:43,898 --> 00:04:45,553
这个就MIG Multi Instance GPU

109
00:04:45,553 --> 00:04:46,778
Multi-Instance GPU

110
00:04:46,778 --> 00:04:47,948
多实例的GPU

111
00:04:47,948 --> 00:04:49,736
将单个的A100 GPU

112
00:04:49,736 --> 00:04:51,698
划分成为多个独立的GPU

113
00:04:51,698 --> 00:04:54,058
为不同的用户提供不同的算力

114
00:04:54,058 --> 00:04:55,698
这个工作我觉得更多的

115
00:04:55,698 --> 00:04:59,978
是为云服务器厂商提供一种更好的算力切分的方案

116
00:05:00,242 --> 00:05:01,060
接着还引入了

117
00:05:01,060 --> 00:05:02,858
第3代的NV-Link和NV-Switch

118
00:05:02,858 --> 00:05:04,058
NV-Switch很有意思

119
00:05:04,058 --> 00:05:08,216
就把多台机器通过NV-Switch进行互联

120
00:05:08,216 --> 00:05:10,822
单卡之间也就单机多卡之间

121
00:05:10,822 --> 00:05:13,088
通过NV-Link进行互联

122
00:05:13,088 --> 00:05:15,938
这个顾名思义就是稀疏性的加速

123
00:05:15,938 --> 00:05:19,498
利用数学的稀疏性对AI的矩阵乘进行加速

124
00:05:20,138 --> 00:05:22,806
下面看看整体的Ampere架构

125
00:05:22,806 --> 00:05:25,148
右边就是Ampere架构的架构图

126
00:05:25,148 --> 00:05:26,818
A100是2020年的时候

127
00:05:26,818 --> 00:05:29,938
7nm里面最多晶体管的一款芯片

128
00:05:29,938 --> 00:05:33,698
里面就有6912个CUDA的内核

129
00:05:33,698 --> 00:05:35,778
和430个Tensor Core

130
00:05:35,778 --> 00:05:37,498
Tensor Core非常多

131
00:05:37,498 --> 00:05:40,658
所以A100是非常善于去处理深度学习的内容了

132
00:05:40,658 --> 00:05:44,898
整体的晶体管数已经到了540亿个晶体管

133
00:05:44,898 --> 00:05:48,938
108个SM，SM非常多

134
00:05:48,938 --> 00:05:51,098
而采用了第3代的NV-Link

135
00:05:51,098 --> 00:05:52,538
看看NV-Link在哪里

136
00:05:52,538 --> 00:05:54,578
对，NV-Link在下面

137
00:05:54,578 --> 00:05:56,378
采用了第3代的NV-Link

138
00:05:56,378 --> 00:06:00,378
GPU和服务器之间的双向带宽是4.8TB每秒

139
00:06:00,378 --> 00:06:04,282
而GPU跟GPU之间是600GB每秒

140
00:06:04,282 --> 00:06:06,458
至少我现在遇到很多友商

141
00:06:06,458 --> 00:06:08,498
他们在训练大模型的时候

142
00:06:08,498 --> 00:06:10,858
用的更多的都是Ampere架构

143
00:06:11,433 --> 00:06:12,825
Ampere架构里面很重要的

144
00:06:12,825 --> 00:06:14,898
就是Tensor Core的新一代

145
00:06:14,898 --> 00:06:16,818
引入了TF32 BF16

146
00:06:16,818 --> 00:06:19,298
还有FP64的支持

147
00:06:19,298 --> 00:06:24,098
在Tensor Core里面很重要的就是BF16和TF32

148
00:06:24,098 --> 00:06:25,218
这两个很有意思

149
00:06:25,218 --> 00:06:29,005
平时用的更多的是FP32和FP16

150
00:06:29,005 --> 00:06:30,658
在指数位有8个

151
00:06:30,658 --> 00:06:33,159
FP16在指数位有5个

152
00:06:33,159 --> 00:06:35,298
总体来说FP32的位宽

153
00:06:35,298 --> 00:06:38,690
也就是它的Range会比FP16更多

154
00:06:38,690 --> 00:06:41,018
后面的是小数位，小数位决定精度

155
00:06:41,018 --> 00:06:43,258
小数位在FP32有32个

156
00:06:43,258 --> 00:06:44,418
而整体来说

157
00:06:44,418 --> 00:06:45,898
FP16只有10个

158
00:06:45,898 --> 00:06:47,538
后来在训练AI的时候

159
00:06:47,538 --> 00:06:50,098
其实发现FP16很多时候是够用的

160
00:06:50,098 --> 00:06:51,658
但是会遇到部分情况下

161
00:06:51,658 --> 00:06:54,018
动态范围其实表示的不是很大

162
00:06:54,018 --> 00:06:56,498
于是英伟达就推出了TF32

163
00:06:56,498 --> 00:07:00,338
TF32就是指数位保持跟FP32相同

164
00:07:00,338 --> 00:07:03,138
而小数位也就是后面的小数位

165
00:07:03,138 --> 00:07:04,978
跟FP16相同

166
00:07:04,978 --> 00:07:07,258
后来又出现了BF16

167
00:07:07,258 --> 00:07:09,498
也就是从Ampere架构去引入的

168
00:07:09,498 --> 00:07:11,258
BF16用的指数位

169
00:07:11,258 --> 00:07:13,498
其实跟FP32

170
00:07:13,498 --> 00:07:15,178
还有TF32相同的

171
00:07:15,178 --> 00:07:18,218
但是小数位小了三位

172
00:07:18,218 --> 00:07:21,098
把三位让给了Range

173
00:07:21,098 --> 00:07:23,618
其实我听过坊间很多传言说

174
00:07:23,618 --> 00:07:27,023
FP16在在训练大模型的时候不够用

175
00:07:27,023 --> 00:07:28,662
更多的是用BF16

176
00:07:28,662 --> 00:07:31,058
其实我在训练大模型的时候

177
00:07:31,058 --> 00:07:33,498
用的很多FP16是够用的

178
00:07:33,498 --> 00:07:36,818
如果可以肯定是用TF32更好

179
00:07:36,818 --> 00:07:39,107
但是其实发现用FP16

180
00:07:39,107 --> 00:07:40,538
至少我现在训练的LLAMA

181
00:07:40,538 --> 00:07:41,618
还有GPT-3

182
00:07:41,618 --> 00:07:44,098
是没有遇到精度不收敛的问题

183
00:07:44,098 --> 00:07:46,778
或许我的大模型还没训练完

184
00:07:48,058 --> 00:07:49,818
下面看看Ampere架构的

185
00:07:49,818 --> 00:07:52,119
一个稀疏化的情况

186
00:07:52,119 --> 00:07:54,538
右边这个图就是Ampere架构的

187
00:07:54,538 --> 00:07:57,546
一个细粒度的稀疏化流程图

188
00:07:57,546 --> 00:08:00,338
Tensor Core除了执行乘加的操作以外

189
00:08:00,338 --> 00:08:03,898
它还可以支持稀疏的结构化的矩阵

190
00:08:04,413 --> 00:08:06,418
假设现在有一个稠密的矩阵

191
00:08:06,418 --> 00:08:08,818
稠密的矩阵是在训练的时候得到的

192
00:08:08,818 --> 00:08:10,338
但是真正在推理的时候

193
00:08:10,338 --> 00:08:12,338
做了一个简单的剪枝

194
00:08:12,338 --> 00:08:13,779
剪枝它是有比例的

195
00:08:13,779 --> 00:08:14,665
剪枝完之后

196
00:08:14,665 --> 00:08:16,098
我会做一个Fine-tunel的剪枝

197
00:08:16,098 --> 00:08:19,653
然后得到一个稀疏的矩阵或者稀疏的权重

198
00:08:19,653 --> 00:08:21,338
接着在英伟达架构里面

199
00:08:21,338 --> 00:08:23,778
就会对矩阵进行压缩

200
00:08:23,778 --> 00:08:25,458
变成一个稠密的矩阵

201
00:08:25,458 --> 00:08:27,538
稠密的矩阵有一个很有意思的点

202
00:08:27,538 --> 00:08:29,898
就是除了矩阵的数据之外

203
00:08:29,898 --> 00:08:31,698
它还有一个indices

204
00:08:31,948 --> 00:08:34,358
所以去把那些压缩过的数据

205
00:08:34,378 --> 00:08:36,173
进行检索记录

206
00:08:36,173 --> 00:08:38,118
最后进行一个矩阵乘

207
00:08:38,143 --> 00:08:39,138
这个是activation

208
00:08:39,138 --> 00:08:39,998
矩阵乘之后

209
00:08:40,018 --> 00:08:41,778
得到output的activation

210
00:08:41,778 --> 00:08:44,103
整体的逻辑那就是右边的这个图

211
00:08:44,323 --> 00:08:47,098
那现在来到了Ampere架构里面

212
00:08:47,098 --> 00:08:49,113
比较后期的一些内容了

213
00:08:49,168 --> 00:08:52,178
可以看到这里面是多实例分割

214
00:08:52,178 --> 00:08:53,778
也就是提到的MIG

215
00:08:53,778 --> 00:08:57,413
每个A100可以分为7个不同的GPU实例

216
00:08:57,463 --> 00:08:59,673
被不同的任务所执行

217
00:08:59,683 --> 00:09:02,618
所以可以看到有很多不同的user

218
00:09:02,618 --> 00:09:04,313
分为7个实例

219
00:09:04,383 --> 00:09:06,713
这些用户可以将这些虚拟化的GPU

220
00:09:06,738 --> 00:09:09,083
当成实际的GPU去执行

221
00:09:09,083 --> 00:09:10,983
所以ZOMI觉得A100架构

222
00:09:11,058 --> 00:09:13,418
就是为很多云计算的厂商

223
00:09:13,503 --> 00:09:17,178
提供了算力切分和多用户的租赁的任务

224
00:09:17,603 --> 00:09:19,178
这个也是Ampere架构

225
00:09:19,178 --> 00:09:20,698
被更多的人去用到（的原因）

226
00:09:20,698 --> 00:09:22,658
因为大家去租用服务器厂商（的服务）

227
00:09:22,658 --> 00:09:24,258
或者用云的时候

228
00:09:24,258 --> 00:09:26,378
用的很多的Ampere架构

229
00:09:26,378 --> 00:09:28,058
或者A100的服务器

230
00:09:28,833 --> 00:09:31,138
再往下看看就整个英伟达的

231
00:09:31,138 --> 00:09:33,778
A100的整体的硬件的规格

232
00:09:34,018 --> 00:09:35,698
放开硬件左边的硬件规格

233
00:09:35,698 --> 00:09:37,778
看看右边的图

234
00:09:37,783 --> 00:09:41,303
很有意思的就是现在它整块

235
00:09:41,448 --> 00:09:45,238
上面的这些，上面这一坨都是散热板

236
00:09:45,248 --> 00:09:48,448
而真正的A100是下面的贴片

237
00:09:48,498 --> 00:09:49,778
下面的芯片

238
00:09:49,778 --> 00:09:52,858
这里面不再是通过PCIE插进去

239
00:09:52,858 --> 00:09:56,018
而是直接焊在主板上面

240
00:09:56,018 --> 00:09:59,098
英伟达卖呢，是卖整一个节点

241
00:09:59,273 --> 00:10:03,258
整一个节点就有8个A100的芯片了

242
00:10:03,733 --> 00:10:05,038
在训练大模型的时候

243
00:10:05,058 --> 00:10:07,218
大家会感受的特别深刻

244
00:10:07,218 --> 00:10:10,393
我在同一个节点里面进行模型并行的时候

245
00:10:10,398 --> 00:10:11,678
是非常方便的

246
00:10:11,678 --> 00:10:14,253
但是跨节点跨机器之间训练大模型

247
00:10:14,273 --> 00:10:18,098
带宽就会成为整个网络制约的瓶颈

248
00:10:18,123 --> 00:10:20,823
或者整个大模型训练的瓶颈

249
00:10:21,058 --> 00:10:23,658
在这一个整机里面我发现有个很有意思的点

250
00:10:23,658 --> 00:10:25,728
就是整一个整机

251
00:10:25,798 --> 00:10:28,658
它的内存高达1TB

252
00:10:28,658 --> 00:10:31,673
或者2TB非常多

253
00:10:31,678 --> 00:10:35,298
我很多时候直接把数据全部都加载在CPU里面

254
00:10:35,298 --> 00:10:37,978
然后再不断的回传到GPU里面

255
00:10:37,978 --> 00:10:39,658
这样可以很好的去加速

256
00:10:39,658 --> 00:10:41,058
大模型的训练

257
00:10:43,113 --> 00:10:44,898
接下来就是英伟达

258
00:10:44,898 --> 00:10:47,518
2022年发布的Hopper架构

259
00:10:47,533 --> 00:10:49,978
Hopper它不是一个漏斗的意思

260
00:10:49,978 --> 00:10:52,053
因为嘛它英文叫Hopper

261
00:10:52,053 --> 00:10:54,903
实际上Hopper是耶鲁大学的第一位女博士

262
00:10:54,943 --> 00:10:57,738
一个杰出的计算机学家

263
00:10:57,768 --> 00:10:59,778
也是COBOL语言发明之母

264
00:10:59,778 --> 00:11:02,558
现在看看Hopper架构有什么不一样

265
00:11:02,623 --> 00:11:05,568
Hopper架构非常的惊艳聪明

266
00:11:05,578 --> 00:11:07,048
因为整个Hopper架构

267
00:11:07,058 --> 00:11:10,718
它除了提出了整个Hopper的GPU以外

268
00:11:10,773 --> 00:11:13,538
它还提出了一个Grace CPU

269
00:11:13,673 --> 00:11:15,088
因此整体它叫做

270
00:11:15,106 --> 00:11:17,006
Grace Hopper Superchip

271
00:11:17,058 --> 00:11:18,088
整个异构的架构

272
00:11:18,099 --> 00:11:19,899
将英伟达的Hopper的GPU的突破性

273
00:11:19,938 --> 00:11:23,418
跟英伟达的Grace CPU连在一起

274
00:11:23,423 --> 00:11:24,723
在整个Superchip里面 

275
00:11:24,738 --> 00:11:28,138
CPU跟GPU之间通过NVLink进行连接

276
00:11:28,138 --> 00:11:32,018
GPU跟GPU之间也是通过NVLink进行连接

277
00:11:32,018 --> 00:11:35,098
而跨机之间通过PCIe5进行连接

278
00:11:35,098 --> 00:11:37,178
可以看到CPU跟GPU之间

279
00:11:37,178 --> 00:11:39,298
以前是通过PCIe进行连接的

280
00:11:39,298 --> 00:11:42,218
现在直接通过NVLink进行传输

281
00:11:42,218 --> 00:11:45,258
数据的传输速率高达900GB每秒

282
00:11:45,258 --> 00:11:49,058
而GPU跟GPU之间传输速率也高达900GB每秒

283
00:11:49,558 --> 00:11:50,248
这个动作

284
00:11:50,258 --> 00:11:52,538
使得GPU跟CPU之间的数据传输

285
00:11:52,538 --> 00:11:55,338
时延和搬运不再是问题

286
00:11:55,338 --> 00:11:56,618
变成了一个C to C

287
00:11:56,618 --> 00:11:59,628
也就是chip to chip相互互联

288
00:11:59,653 --> 00:12:01,838
但是是不是所有用户都用得到？

289
00:12:01,898 --> 00:12:02,878
我觉得不一定

290
00:12:02,898 --> 00:12:06,618
因为训练大模型的用户确实没有那么多

291
00:12:06,618 --> 00:12:07,818
没有传说中的那么多

292
00:12:07,818 --> 00:12:11,493
更多的人是用大模型进行一个下游任务的微调

293
00:12:11,493 --> 00:12:13,568
然后适配到它具体的任务

294
00:12:13,658 --> 00:12:15,603
右边这个就是具体的

295
00:12:15,620 --> 00:12:16,970
Grace Hopper Superchip

296
00:12:16,978 --> 00:12:19,153
整体的3D的渲染图

297
00:12:19,283 --> 00:12:21,728
下面继续的展开整个Hopper架构

298
00:12:21,748 --> 00:12:23,263
有哪些不一样的点

299
00:12:23,303 --> 00:12:25,778
这里面ZOMI就总结了4个点

300
00:12:25,778 --> 00:12:26,738
首先第一句话

301
00:12:26,788 --> 00:12:30,198
它是真正异构的一个加速平台

302
00:12:30,198 --> 00:12:32,808
适用于高性能计算机和AI的工作负载

303
00:12:32,978 --> 00:12:36,298
我觉得我更关心的是整个AI的工作负载

304
00:12:36,298 --> 00:12:39,378
里面还非常重要的4个提出新的点

305
00:12:39,378 --> 00:12:41,178
就是Grace CPU

306
00:12:41,178 --> 00:12:43,633
英伟达本来基于2021年的时候

307
00:12:43,653 --> 00:12:45,563
想收购ARM的但是没有收购成功

308
00:12:45,618 --> 00:12:47,018
这是另外一个故事了

309
00:12:47,018 --> 00:12:50,578
但是英伟达基于ARM提出了自己的Grace CPU

310
00:12:50,658 --> 00:12:52,858
另外我觉得更关心的是Hopper GPU

311
00:12:52,858 --> 00:12:54,458
也就是今天的主角

312
00:12:54,458 --> 00:12:57,328
这里面就更新了第4代的Tensor Core

313
00:12:57,353 --> 00:13:00,013
里面就引出了Transformer Engine

314
00:13:00,018 --> 00:13:01,013
然后Transformer Engine

315
00:13:01,025 --> 00:13:03,850
专门针对大模型进行加速的

316
00:13:03,958 --> 00:13:05,658
另外在GPU里面的内存

317
00:13:05,674 --> 00:13:08,024
已经高达了300GB每秒的速度

318
00:13:08,083 --> 00:13:10,083
然后缓存也进一步的提升

319
00:13:10,108 --> 00:13:11,048
而整体来说

320
00:13:11,098 --> 00:13:13,458
它的互联是NVLink的C2C

321
00:13:13,458 --> 00:13:17,878
把CPU跟GPU之间通过NVLink进行连接

322
00:13:17,928 --> 00:13:19,818
现在来看看整体的Hopper架构

323
00:13:19,818 --> 00:13:23,218
Hopper架构这里面有8组GPC

324
00:13:23,218 --> 00:13:24,778
也就是图像处理簇

325
00:13:24,778 --> 00:13:28,138
然后一共有66组TCP

326
00:13:28,138 --> 00:13:31,058
TCP里面又有132组SM

327
00:13:31,058 --> 00:13:34,938
总共有16896个CUDA的核心

328
00:13:34,938 --> 00:13:36,538
528个Tensor Core

329
00:13:36,538 --> 00:13:38,038
之前在A100里面

330
00:13:38,058 --> 00:13:39,778
其实它的Tensor Core只有400多个

331
00:13:39,778 --> 00:13:41,713
现在已经多到500多个了

332
00:13:41,713 --> 00:13:45,248
在L2Cache里面出现了50MB的二级缓存

333
00:13:45,273 --> 00:13:47,453
整个显存是新一代的HBM3

334
00:13:47,503 --> 00:13:50,408
容量跟A100相同，保持在80G

335
00:13:50,418 --> 00:13:52,938
位宽是5120bit的带宽

336
00:13:52,938 --> 00:13:54,898
高达3TB每秒

337
00:13:54,898 --> 00:13:57,623
整体的性能非常的夸张

338
00:13:57,748 --> 00:13:59,428
用猛兽来形容Hopper架构

339
00:13:59,446 --> 00:14:01,446
其实一点都不夸张

340
00:14:01,538 --> 00:14:03,778
现在打开看看

341
00:14:03,778 --> 00:14:05,978
Hopper架构里面的SM

342
00:14:05,978 --> 00:14:07,618
SM里面其实

343
00:14:07,618 --> 00:14:08,618
总体来说

344
00:14:08,618 --> 00:14:10,138
看一下跟A100比

345
00:14:10,138 --> 00:14:12,253
后面会有A100的一些参数

346
00:14:12,268 --> 00:14:13,938
从右边那个图可以看到

347
00:14:13,998 --> 00:14:16,378
一个SM里面有4个Warp Scheduler

348
00:14:16,378 --> 00:14:18,858
1 2 3 4

349
00:14:18,858 --> 00:14:19,938
4个Warp Scheduler

350
00:14:19,938 --> 00:14:21,538
然后4个Dispatch Unit

351
00:14:21,538 --> 00:14:23,338
就是对指令进行分发

352
00:14:23,338 --> 00:14:26,513
在应该上上节课里面去给大家讲过

353
00:14:26,538 --> 00:14:28,698
Warp Scheduler和Dispatch Unit

354
00:14:28,698 --> 00:14:29,698
有什么不一样

355
00:14:29,698 --> 00:14:30,658
接着看一下

356
00:14:30,658 --> 00:14:32,298
里面的FP32的Core

357
00:14:32,298 --> 00:14:33,548
就这里面就变成

358
00:14:33,553 --> 00:14:36,578
INT32、FP32和FP64的Core

359
00:14:36,625 --> 00:14:40,275
基本上FP32、INT32都是翻了一倍

360
00:14:40,298 --> 00:14:43,138
而一个SM里面有4个Tensor Core

361
00:14:43,663 --> 00:14:47,018
比较有意思的一点就是下面这条绿色的

362
00:14:47,018 --> 00:14:49,233
相比A100，H100多了一个

363
00:14:49,287 --> 00:14:51,312
Tensor Memory Accelerator

364
00:14:51,328 --> 00:14:53,098
专门针对张量

365
00:14:53,138 --> 00:14:55,658
进行数据的传输的

366
00:14:55,658 --> 00:14:57,098
所以说以前张量

367
00:14:57,098 --> 00:14:59,488
都是放在L2或者L1的Cache

368
00:14:59,573 --> 00:15:01,288
会更多的，可能有些数据

369
00:15:01,298 --> 00:15:02,378
放在为Register File

370
00:15:02,378 --> 00:15:03,498
就寄存器

371
00:15:03,498 --> 00:15:06,533
现在有了张量Memory Accelerator

372
00:15:06,573 --> 00:15:07,778
更好的对大矩阵

373
00:15:07,778 --> 00:15:09,378
大模型进行加速的

374
00:15:10,018 --> 00:15:10,848
再往下看看

375
00:15:10,858 --> 00:15:12,918
这个就是英伟达官方的一个图

376
00:15:12,988 --> 00:15:15,578
可以简单的去买到一个A100的

377
00:15:15,578 --> 00:15:17,058
单款的异构的芯片

378
00:15:17,058 --> 00:15:18,378
把8块卡买回来

379
00:15:18,378 --> 00:15:19,978
同时插在插槽里面

380
00:15:19,978 --> 00:15:21,258
更多的刚才讲到的

381
00:15:21,258 --> 00:15:22,258
很多的Superchip

382
00:15:22,258 --> 00:15:23,188
或者NVLink

383
00:15:23,258 --> 00:15:24,498
是通过整机

384
00:15:24,498 --> 00:15:25,938
或者整个节点去卖的

385
00:15:25,938 --> 00:15:27,738
同时英伟达还提出了

386
00:15:27,738 --> 00:15:29,738
非常之变态的售卖方式

387
00:15:29,738 --> 00:15:31,458
就直接卖一台POD

388
00:15:33,433 --> 00:15:35,343
把网络都给你组好了

389
00:15:35,368 --> 00:15:36,898
整一台POD卖给你

390
00:15:36,898 --> 00:15:38,443
这是多么的变态

391
00:15:38,498 --> 00:15:40,338
这收钱收的真的是手软

392
00:15:40,338 --> 00:15:41,938
怪不得他的市值能这么高

393
00:15:41,938 --> 00:15:42,378
好了

394
00:15:42,378 --> 00:15:44,298
今天的内容就到这里为止

395
00:15:44,298 --> 00:15:45,578
简单的总结

396
00:15:45,578 --> 00:15:46,418
或者回顾一下

397
00:15:46,418 --> 00:15:48,178
GPU的架构的发展

398
00:15:48,178 --> 00:15:49,618
每一代GPU的架构

399
00:15:49,738 --> 00:15:51,858
都是以科学家进行命名的

400
00:15:51,858 --> 00:15:54,268
从2010年的Fermi架构开始

401
00:15:54,343 --> 00:15:56,938
整个完整的GPU的计算架构提出

402
00:15:56,938 --> 00:15:58,658
接着到了volta架构

403
00:15:58,658 --> 00:16:00,043
或者Pascal架构里面

404
00:16:00,098 --> 00:16:02,708
提出了第一代的NVLink双向的互联

405
00:16:02,748 --> 00:16:03,778
让GPU更好地

406
00:16:03,778 --> 00:16:06,138
在HPC场景里面去发挥它的作用

407
00:16:06,138 --> 00:16:07,978
接着应该是2017年的时候

408
00:16:07,978 --> 00:16:08,898
AI非常火

409
00:16:08,898 --> 00:16:11,098
于是顺势的推出了volta架构

410
00:16:11,178 --> 00:16:13,378
提出了第一代的Tensor Core

411
00:16:13,378 --> 00:16:16,218
而后来应该是在2018年的时候

412
00:16:16,218 --> 00:16:19,138
在消费级的显卡里面提出了RT Core

413
00:16:19,138 --> 00:16:21,093
实现了硬件的光线追踪

414
00:16:21,093 --> 00:16:22,354
非常的惊艳

415
00:16:22,354 --> 00:16:24,143
在AI云应用厂商里面

416
00:16:24,143 --> 00:16:25,858
Ampere架构是卖的最好的

417
00:16:25,858 --> 00:16:28,058
因为这里面除了Tensor Core、NVLink

418
00:16:28,058 --> 00:16:29,658
还有RT Core的更新之外

419
00:16:29,658 --> 00:16:31,538
还提出了MIG

420
00:16:31,673 --> 00:16:32,978
多用户的GPU实例

421
00:16:32,978 --> 00:16:34,898
最后到最近的应该是

422
00:16:34,898 --> 00:16:37,218
现在你还买不到货的Hopper架构

423
00:16:37,218 --> 00:16:38,859
也就是赫柏架构

424
00:16:38,859 --> 00:16:40,338
这里面就提出了CPU跟GPU

425
00:16:40,338 --> 00:16:42,258
异构的架构变成一个Superchip

426
00:16:42,258 --> 00:16:43,578
去对外去销售

427
00:16:43,578 --> 00:16:46,058
当然你也可以买一个简单的GPU

428
00:16:46,058 --> 00:16:47,418
或者简单的H100

429
00:16:47,418 --> 00:16:50,138
它的销售方式就变得非常多了

430
00:16:50,138 --> 00:16:52,738
或者它的解决方案就变得非常的多

431
00:16:52,738 --> 00:16:54,658
那今天的内容就到这里为止

432
00:16:54,658 --> 00:16:55,338
谢谢各位

433
00:16:55,338 --> 00:16:56,258
拜了个拜

434
00:16:56,940 --> 00:16:58,518
卷的不行了

435
00:16:58,518 --> 00:17:00,193
记得一键三连加关注

436
00:17:00,402 --> 00:17:01,721
所有的内容都会开源

437
00:17:01,721 --> 00:17:03,449
在下面这条链接里面

438
00:17:03,893 --> 00:17:04,853
拜了个拜