04.srt

1
00:00:02,823 --> 00:00:04,823
字幕生成：慎独     字母校对：lim

2
00:00:05,417 --> 00:00:06,953
哈喽大家好我是ZOMI

3
00:00:06,953 --> 00:00:08,537
现在已经是我看看啊

4
00:00:08,537 --> 00:00:10,379
到了晚上凌晨一点多了

5
00:00:10,379 --> 00:00:11,859
我刚到东莞

6
00:00:11,859 --> 00:00:14,995
这是我到东莞集结的第一天

7
00:00:14,995 --> 00:00:16,441
星期天的晚上

8
00:00:16,441 --> 00:00:19,669
那今天呢来到了一个AI芯片里面的

9
00:00:19,669 --> 00:00:20,425
AI计算体系

10
00:00:20,425 --> 00:00:22,963
主要讲到的是计算体系

11
00:00:22,963 --> 00:00:24,963
回头看看整体的目录

12
00:00:24,963 --> 00:00:26,328
在整体的目录里面呢

13
00:00:26,328 --> 00:00:28,328
现在来到了这个内容

14
00:00:28,328 --> 00:00:30,985
整个计算体系和矩阵的运算

15
00:00:30,985 --> 00:00:34,057
那计算体系呢我会给大家带来一些新的

16
00:00:34,057 --> 00:00:37,628
或者我自己的还有在网上去摘录的一些思考

17
00:00:37,828 --> 00:00:38,571
那最重要的

18
00:00:38,571 --> 00:00:40,699
在AI计算体系和矩阵运算里面

19
00:00:40,699 --> 00:00:42,057
分享四个内容

20
00:00:42,185 --> 00:00:46,185
第一个呢就是AI芯片的关键指标key metrics

21
00:00:46,185 --> 00:00:50,185
第二个呢就是比特的位数还有矩阵乘

22
00:00:50,185 --> 00:00:52,920
那这两个呢是更细的一些细节了

23
00:00:52,920 --> 00:00:57,016
最后看一下Domain Specific一些专用的硬件

24
00:00:57,016 --> 00:00:59,368
具体对于AI这个产业

25
00:00:59,368 --> 00:01:00,875
或者AI这个芯片呢

26
00:01:00,875 --> 00:01:02,875
有哪些牵引的方向

27
00:01:02,898 --> 00:01:04,681
和它的计算体系有什么不一样

28
00:01:04,681 --> 00:01:06,884
那今天主要是快速的去过一下

29
00:01:06,884 --> 00:01:09,526
AI芯片的关键指标这个内容

30
00:01:10,506 --> 00:01:12,746
好了来到正式的内容之前呢

31
00:01:12,746 --> 00:01:14,506
来看看整体的算力单位

32
00:01:14,506 --> 00:01:16,234
不管是哪款芯片刚发布

33
00:01:16,234 --> 00:01:19,856
包括里面的最近的苹果的M2这块SoC

34
00:01:19,856 --> 00:01:21,706
里面都会大量的提到

35
00:01:21,706 --> 00:01:24,666
我的峰值的算力到底是多少

36
00:01:24,666 --> 00:01:27,020
我到底有多少倍的性能的提升

37
00:01:27,020 --> 00:01:28,843
而这里面呢看看

38
00:01:28,843 --> 00:01:31,270
算力的单位呢主要有两个

39
00:01:31,270 --> 00:01:32,742
第一个呢是OPS

40
00:01:32,742 --> 00:01:35,675
注意哦OPS三个字母都是大写

41
00:01:35,937 --> 00:01:39,199
然后呢还有另外一个叫做MACs

42
00:01:39,199 --> 00:01:42,311
注意了这里面的s呢是小写

43
00:01:42,311 --> 00:01:43,620
而现在来看看

44
00:01:43,620 --> 00:01:47,735
OPS呢主要是指Operations Per Second

45
00:01:47,735 --> 00:01:51,735
每秒能够去运行的多少次计算

46
00:01:51,943 --> 00:01:55,488
而下面有另外一个单位加了/W

47
00:01:55,488 --> 00:01:58,332
是指每瓦特的运算的性能

48
00:01:58,332 --> 00:02:00,700
那通常在评判一款SoC的时候呢

49
00:02:00,700 --> 00:02:04,590
就会经常用TOPS/W这种

50
00:02:04,590 --> 00:02:06,590
去评价处理器的性能

51
00:02:07,096 --> 00:02:09,590
下面看一下MACs

52
00:02:09,590 --> 00:02:12,598
MACs其实后面会大量的去讲到的

53
00:02:12,598 --> 00:02:15,250
就是Multiply-Accumulate Operations

54
00:02:15,590 --> 00:02:16,965
乘积累加操作

55
00:02:16,965 --> 00:02:18,682
可以看到的层加累积操作主要是

56
00:02:18,682 --> 00:02:20,590
在前面的介绍过

57
00:02:20,590 --> 00:02:25,299
神经网络的最主要的运算就是乘积累加操作了

58
00:02:25,299 --> 00:02:27,743
而一个乘积累加操作呢

59
00:02:27,743 --> 00:02:30,590
就包括一个乘法和一个加法

60
00:02:30,846 --> 00:02:33,590
这里面呢就是就涉及到两个Flops了

61
00:02:33,590 --> 00:02:35,590
那接下来再看两个概念

62
00:02:35,590 --> 00:02:40,846
两个呢就是MAC还有FLOPs

63
00:02:40,846 --> 00:02:45,038
注意这里面呢MAC这个C后面没有s

64
00:02:45,038 --> 00:02:49,038
而FLOPs呢后面的s是小写

65
00:02:49,388 --> 00:02:53,388
这两个单位呢有点不一样跟模型相关

66
00:02:53,388 --> 00:02:56,388
像FLOPs呢主要是Full Point Operations

67
00:02:56,388 --> 00:02:59,042
具体的s呢是来自于这里面的s

68
00:02:59,042 --> 00:03:02,042
主要指的是就是浮点的运算的次数

69
00:03:02,042 --> 00:03:04,042
主要用来衡量网络模型啊

70
00:03:04,042 --> 00:03:06,715
AI模型算法的复杂度

71
00:03:06,715 --> 00:03:10,042
第二个呢是MAC这个是跟内存相关的

72
00:03:10,042 --> 00:03:12,522
叫做Memory Assessed Cost

73
00:03:12,522 --> 00:03:14,692
用来评价一些AI算法

74
00:03:14,692 --> 00:03:17,042
一些MobileNet在内存的占用量

75
00:03:17,056 --> 00:03:19,460
而OPS和MACs主要是用来

76
00:03:19,460 --> 00:03:20,884
评价硬件

77
00:03:20,884 --> 00:03:24,875
特别是AI芯片里面的一些主要的运算的性能

78
00:03:24,875 --> 00:03:29,400
而在后面讲到AI芯片的关键的Metrics

79
00:03:29,400 --> 00:03:31,400
就是接下来要分享的内容

80
00:03:31,400 --> 00:03:33,720
里面的内容就会大量的去用到

81
00:03:33,720 --> 00:03:35,420
上面介绍的一些

82
00:03:35,420 --> 00:03:37,400
主要的算力的指标

83
00:03:37,400 --> 00:03:40,400
后面呢就不会再详细地单独地展开

84
00:03:40,400 --> 00:03:43,535
而是统一通过简称来去给大家介绍

85
00:03:43,535 --> 00:03:46,152
所以它只是一个开篇的开胃菜

86
00:03:46,152 --> 00:03:48,400
大家知道有这么一回事就好了

87
00:03:50,400 --> 00:03:52,400
接下来呢来到一个正式的内容

88
00:03:52,400 --> 00:03:56,082
AI芯片的关键指标Key Metrics

89
00:03:56,082 --> 00:03:57,937
这里面总结了六个

90
00:03:57,937 --> 00:03:59,656
第一个就是精度

91
00:03:59,656 --> 00:04:02,400
精度不仅仅是指模型结果的精度

92
00:04:02,400 --> 00:04:04,400
还包括计算的精度

93
00:04:04,400 --> 00:04:07,400
例如我用FP32FP16来去计算

94
00:04:07,400 --> 00:04:09,400
还是用int8来去计算

95
00:04:09,400 --> 00:04:12,029
第二个呢就是吞吐量Throughput

96
00:04:12,029 --> 00:04:12,479
え？

97
00:04:12,479 --> 00:04:14,206
这里面单词拼错了

98
00:04:14,206 --> 00:04:14,656
没关系

99
00:04:14,656 --> 00:04:16,318
后面再改改

100
00:04:16,318 --> 00:04:19,553
而吞吐量主要是指AI芯片对于张量

101
00:04:19,553 --> 00:04:21,553
因为张量的数据量特别大

102
00:04:21,553 --> 00:04:23,993
它怎么去对这些数据进行抽取

103
00:04:23,993 --> 00:04:25,679
它的吞吐量到底有多少

104
00:04:25,679 --> 00:04:28,993
第二个呢主要是指实时的性能

105
00:04:28,993 --> 00:04:30,993
能不能够快速的去处理

106
00:04:30,993 --> 00:04:32,387
每秒30fps

107
00:04:32,387 --> 00:04:34,868
或者每秒20tokens的这样的数据

108
00:04:34,868 --> 00:04:35,456
那这两个呢

109
00:04:35,456 --> 00:04:37,120
其实都跟数据相关

110
00:04:37,120 --> 00:04:39,744
第三个呢就是时延Latency

111
00:04:39,744 --> 00:04:41,744
在这里面呢有一个很重要的词

112
00:04:41,744 --> 00:04:43,993
就是开发交互应用程序的

113
00:04:43,993 --> 00:04:46,340
一个时间叫做TTA

114
00:04:46,340 --> 00:04:48,856
后面呢还有几个指标

115
00:04:48,856 --> 00:04:51,096
再继续展开一下

116
00:04:51,596 --> 00:04:53,660
能耗这个指标非常关键

117
00:04:53,660 --> 00:04:57,060
IOT交互设备因为电池容量是有限的

118
00:04:57,060 --> 00:04:58,740
每秒去执行多少个FLOPs

119
00:04:58,740 --> 00:05:00,740
能耗消耗是多少

120
00:05:00,740 --> 00:05:02,740
那这个就刚才指到的

121
00:05:02,740 --> 00:05:06,740
OPS每瓦特的性能了

122
00:05:06,740 --> 00:05:10,365
第二点就是数据中心的液冷等大功耗的设备

123
00:05:10,365 --> 00:05:13,740
确实也是一个很重要的衡量指标

124
00:05:13,740 --> 00:05:15,740
第五个呢就是系统的价格

125
00:05:15,740 --> 00:05:17,320
叫做系统的价格

126
00:05:17,320 --> 00:05:20,006
是因为不仅包括AI芯片支撑的价格

127
00:05:20,006 --> 00:05:22,496
包括去光刻或者流片的价格

128
00:05:22,496 --> 00:05:26,031
还包括系统集成上下游全栈的成本

129
00:05:26,031 --> 00:05:27,375
因为有了芯片之后

130
00:05:27,375 --> 00:05:31,006
基于芯片要开发相关的大量的软件编译层

131
00:05:31,581 --> 00:05:33,581
还有在对上的SDK

132
00:05:33,581 --> 00:05:36,245
这些都是集成在系统的价格里面

133
00:05:36,245 --> 00:05:39,581
你买英伟达一款芯片你不仅是买他的芯片

134
00:05:39,581 --> 00:05:41,581
你买昇腾的一款产品

135
00:05:41,581 --> 00:05:43,195
也不仅是买他的产品

136
00:05:43,195 --> 00:05:45,234
而是他产品配套的服务

137
00:05:45,234 --> 00:05:46,581
配套的软硬件

138
00:05:46,581 --> 00:05:48,835
那最后一个呢就是应用性的

139
00:05:48,835 --> 00:05:50,984
应用性这个其实很难去衡量的

140
00:05:50,984 --> 00:05:53,395
他主要是看开发的效率

141
00:05:53,395 --> 00:05:54,581
还有开发的难度

142
00:05:54,581 --> 00:05:56,581
而应用性的好坏呢

143
00:05:56,581 --> 00:05:59,581
确实决定于整款的芯片

144
00:05:59,581 --> 00:06:01,581
用的人的多和少

145
00:06:01,581 --> 00:06:04,581
而且决定用户用的爽还是不爽

146
00:06:04,581 --> 00:06:07,581
下面AI加速器的关键设计点

147
00:06:07,581 --> 00:06:08,845
虽然它有很多指标

148
00:06:08,845 --> 00:06:11,581
但是还是有一些关键的一些点

149
00:06:11,581 --> 00:06:15,156
例如在跟客户交流或者做系统集成的时候

150
00:06:15,156 --> 00:06:16,138
会经常发现

151
00:06:16,138 --> 00:06:19,279
确实要尽可能的去提高吞吐量

152
00:06:19,279 --> 00:06:20,559
Increase整个Throughput

153
00:06:20,559 --> 00:06:23,311
还有降低数据处理的时延

154
00:06:23,581 --> 00:06:26,788
大家有没有发现不管是吞吐量还是时延

155
00:06:26,788 --> 00:06:30,209
更多是跟数据和计算相关的

156
00:06:30,209 --> 00:06:31,745
所以很多时候呢

157
00:06:31,745 --> 00:06:35,201
大部分都会跟数据和计算、内存

158
00:06:35,201 --> 00:06:37,377
三个东西打交道

159
00:06:37,377 --> 00:06:41,227
那第二点呢就是低时延和Batch Size

160
00:06:41,227 --> 00:06:42,507
之间的一个平衡

161
00:06:42,507 --> 00:06:43,851
叫做Tradeoff

162
00:06:43,851 --> 00:06:45,299
我想要时延更低

163
00:06:45,299 --> 00:06:46,707
我的Batch Size设的更少

164
00:06:46,707 --> 00:06:49,581
我的每一次处理的数据肯定要少

165
00:06:49,581 --> 00:06:51,331
每一次处理的数据大了

166
00:06:51,331 --> 00:06:52,867
我的时延就会上去了

167
00:06:52,867 --> 00:06:55,581
所以中间呢会取一个Tradeoff

168
00:06:55,581 --> 00:06:58,581
芯片到底能支持多大的Batch Size

169
00:06:58,581 --> 00:07:00,581
能支持多大的数据的吞吐量

170
00:07:00,581 --> 00:07:03,479
跟时延是息息相关的

171
00:07:04,581 --> 00:07:07,581
刚才提到一个很重要的指标

172
00:07:07,581 --> 00:07:11,240
是MACS在整个AI加速器

173
00:07:11,240 --> 00:07:13,356
或者AI芯片的关键的设计点呢

174
00:07:13,356 --> 00:07:14,700
在软件的角度来看

175
00:07:14,700 --> 00:07:16,581
或者在ZOMI的角度来看

176
00:07:16,581 --> 00:07:19,209
有两个我比较关心的点

177
00:07:19,209 --> 00:07:22,217
第一个呢就是去掉没有意义的MACs

178
00:07:22,217 --> 00:07:25,253
去掉没有意义的计算里面最重要的就是

179
00:07:25,253 --> 00:07:27,859
节省整体的时钟周期

180
00:07:27,859 --> 00:07:29,907
去SaveCycle

181
00:07:29,907 --> 00:07:32,125
另外呢他可能还有其他的作用

182
00:07:32,125 --> 00:07:34,581
第二个比较关键的点就是

183
00:07:34,581 --> 00:07:36,581
就是降低每一次MAC的

184
00:07:36,581 --> 00:07:38,581
降低每一次计算的时间

185
00:07:38,581 --> 00:07:40,082
例如可以增加

186
00:07:40,082 --> 00:07:41,959
时钟的周期的频率啦

187
00:07:41,959 --> 00:07:44,581
那这个时候可能功耗也会上去

188
00:07:44,581 --> 00:07:47,581
第二个呢就是减少指令的开销

189
00:07:47,581 --> 00:07:50,082
这一点不仅是对硬件

190
00:07:50,082 --> 00:07:52,504
还有ISA有关系 

191
00:07:52,581 --> 00:07:53,837
可能还会对编译器

192
00:07:53,837 --> 00:07:55,420
对并行的策略

193
00:07:55,420 --> 00:07:57,276
还有数据执行的策略

194
00:07:57,581 --> 00:07:59,581
有关系

195
00:07:59,817 --> 00:08:02,415
在AI加速器另外一个关键点呢

196
00:08:02,415 --> 00:08:03,977
叫做PE

197
00:08:03,977 --> 00:08:05,520
往下面这个图来看

198
00:08:05,520 --> 00:08:08,208
PE实际上呢就是执行单元

199
00:08:08,208 --> 00:08:11,208
每一款芯片呢都会有大量的PE阵列

200
00:08:11,208 --> 00:08:13,550
或者在SIMD架构里面呢

201
00:08:13,550 --> 00:08:14,808
有非常多的PE

202
00:08:14,808 --> 00:08:15,920
在SIMT里面呢

203
00:08:15,920 --> 00:08:17,920
你可以把大量的CUDA Core呢

204
00:08:17,920 --> 00:08:19,920
看成独立的每一个的PE

205
00:08:19,920 --> 00:08:22,789
那为了让芯片跑得更快

206
00:08:22,789 --> 00:08:25,221
首先第一个呢就是增加PE的数量

207
00:08:25,221 --> 00:08:27,397
PE的数量肯定越多越好

208
00:08:27,808 --> 00:08:30,808
例如使用台积电更高纳米的制程

209
00:08:30,808 --> 00:08:32,808
或者中芯国际更高纳米的制程呢

210
00:08:32,808 --> 00:08:35,286
就可以增加PE的面积的密度

211
00:08:35,286 --> 00:08:36,064
这个时候

212
00:08:36,064 --> 00:08:38,808
一款PE可以容纳更多的晶体管

213
00:08:38,808 --> 00:08:41,189
这个时候我执行的效率可能会更高

214
00:08:41,701 --> 00:08:43,320
或者我对应的位置呢

215
00:08:43,320 --> 00:08:44,320
可以塞更多的PE

216
00:08:44,320 --> 00:08:46,640
这也可以提升PE的核心的数量

217
00:08:46,640 --> 00:08:49,056
第二个呢就是增加PE的利用率

218
00:08:49,056 --> 00:08:50,320
因为可以看到呢

219
00:08:50,320 --> 00:08:51,688
在一款芯片里面呢

220
00:08:51,688 --> 00:08:53,320
有非常多的PE

221
00:08:53,320 --> 00:08:55,848
非常多的计算的核心

222
00:08:55,848 --> 00:08:56,552
那这个时候呢

223
00:08:56,552 --> 00:08:58,320
需要把不同的任务

224
00:08:58,320 --> 00:09:00,456
均衡地分配到不同的PE上面

225
00:09:00,456 --> 00:09:03,320
让PE呢满打满地去执行

226
00:09:03,320 --> 00:09:06,320
另外呢可能还会选择合适的内存带宽

227
00:09:06,320 --> 00:09:10,320
有效地去降低整个空闲的时钟周期

228
00:09:10,320 --> 00:09:11,937
就是简单的来说

229
00:09:11,937 --> 00:09:14,320
数据的流动的频率

230
00:09:14,320 --> 00:09:16,320
要跟PE的处理的频率

231
00:09:16,320 --> 00:09:17,761
要匹配得上

232
00:09:17,761 --> 00:09:21,320
才能够让PE发挥更大的效用

233
00:09:22,420 --> 00:09:23,191
诶 ZOMI老师

234
00:09:23,191 --> 00:09:24,420
等一下等一下

235
00:09:24,749 --> 00:09:26,749
你刚才讲了很多内容

236
00:09:26,749 --> 00:09:28,749
我有两个问题啊

237
00:09:28,749 --> 00:09:30,054
第一个你刚才讲的

238
00:09:30,054 --> 00:09:32,550
AI芯片的关键指标里面呢

239
00:09:32,550 --> 00:09:34,550
有吞吐量和时延

240
00:09:34,550 --> 00:09:37,817
那这个主要是由什么产生的呢

241
00:09:38,751 --> 00:09:39,967
小新同学你好

242
00:09:39,967 --> 00:09:41,823
AI芯片的关键的指标呢

243
00:09:41,823 --> 00:09:42,591
主要有六项

244
00:09:42,591 --> 00:09:44,831
这里面呢吞吐量和时延就占了两项

245
00:09:44,831 --> 00:09:47,148
这两个呢更多的是由计算

246
00:09:47,148 --> 00:09:48,834
还有内存导致的

247
00:09:48,834 --> 00:09:50,623
计算越快

248
00:09:50,623 --> 00:09:52,479
时延肯定就越少

249
00:09:52,479 --> 00:09:53,834
但是时延

250
00:09:53,834 --> 00:09:55,834
不仅仅是跟计算量相关

251
00:09:55,834 --> 00:09:57,834
还跟吞吐量相关

252
00:09:57,834 --> 00:10:00,218
内存、cache、吞吐量越大

253
00:10:00,218 --> 00:10:03,384
时延也会相对应的去减少

254
00:10:03,384 --> 00:10:04,088
那这个时候呢

255
00:10:04,088 --> 00:10:06,154
吞吐量和时延是息息相关的

256
00:10:06,300 --> 00:10:07,384
在下一个内容

257
00:10:07,384 --> 00:10:08,476
计算仿真的时候呢

258
00:10:08,476 --> 00:10:09,948
就会给大家去看看

259
00:10:09,948 --> 00:10:12,384
这里面具体有什么关系

260
00:10:12,384 --> 00:10:14,384
应该怎么去处理好

261
00:10:14,384 --> 00:10:17,034
芯片应该用哪个明确的指标

262
00:10:17,034 --> 00:10:19,384
或者仿真的指标去做牵引

263
00:10:19,384 --> 00:10:20,152
第二个问题

264
00:10:20,152 --> 00:10:21,384
你刚才提到

265
00:10:21,384 --> 00:10:24,384
AI芯片的关键的设计点呢

266
00:10:24,384 --> 00:10:26,987
是MAC和PE

267
00:10:26,987 --> 00:10:31,384
感觉这两个主要是针对提升单个核心的

268
00:10:31,384 --> 00:10:33,384
计算能力是吗

269
00:10:33,832 --> 00:10:35,384
说的非常对

270
00:10:35,384 --> 00:10:37,384
刚才讲了两个内容

271
00:10:37,384 --> 00:10:38,384
一个是关键指标

272
00:10:38,384 --> 00:10:40,384
一个是关键的设计点

273
00:10:40,384 --> 00:10:41,384
关键的指标呢

274
00:10:41,384 --> 00:10:44,384
是整体的去看AI芯片的

275
00:10:44,384 --> 00:10:45,384
而关键的设计点呢

276
00:10:45,384 --> 00:10:47,384
MACs和PE呢

277
00:10:47,384 --> 00:10:49,687
主要是针对单个核心的计算能力

278
00:10:49,687 --> 00:10:52,012
希望能够尽可能的去提升

279
00:10:52,012 --> 00:10:54,012
计算的峰值的算力

280
00:10:54,862 --> 00:10:55,862
谈到这一点呢

281
00:10:55,862 --> 00:10:57,609
就引入了下个内容

282
00:10:57,609 --> 00:10:59,862
计算性能的仿真

283
00:10:59,862 --> 00:11:02,446
下面呢来到了这节分享里面

284
00:11:02,446 --> 00:11:04,771
或者给大家汇报里面的内容里面

285
00:11:04,771 --> 00:11:06,862
最精华或者最重要的一部分

286
00:11:06,862 --> 00:11:07,838
如果大家听不明白

287
00:11:07,838 --> 00:11:09,029
或者我讲的不太好呢

288
00:11:09,029 --> 00:11:10,838
欢迎大家来去指正

289
00:11:10,838 --> 00:11:12,838
就是整个计算性能的仿真

290
00:11:12,838 --> 00:11:15,548
下面的这个图是非常的核心

291
00:11:15,548 --> 00:11:17,145
也是来源于这篇论文

292
00:11:17,145 --> 00:11:18,425
现在来看看

293
00:11:18,425 --> 00:11:21,588
纵坐标和横坐标所代表的意义

294
00:11:21,588 --> 00:11:23,034
那横坐标呢

295
00:11:23,034 --> 00:11:25,662
主要是指计算的密集度

296
00:11:25,662 --> 00:11:27,646
或者叫做计算的强度

297
00:11:27,646 --> 00:11:30,034
它的单位呢是ops/byte

298
00:11:30,034 --> 00:11:31,560
纵坐标呢

299
00:11:31,560 --> 00:11:33,534
就是性能performance

300
00:11:33,534 --> 00:11:36,534
它的单位呢是ops每second

301
00:11:36,534 --> 00:11:37,534
那这个时候呢

302
00:11:37,534 --> 00:11:40,534
中间有一条曲折线

303
00:11:40,534 --> 00:11:41,534
那这条曲折线呢

304
00:11:41,534 --> 00:11:44,534
代表的就是整个AI芯片的

305
00:11:44,534 --> 00:11:46,534
峰值算力

306
00:11:46,534 --> 00:11:47,982
峰值的性能

307
00:11:47,982 --> 00:11:49,220
而这个峰值性能呢

308
00:11:49,220 --> 00:11:51,332
更多的是指理论的峰值性能

309
00:11:51,332 --> 00:11:53,534
而不是实际的峰值性能

310
00:11:53,534 --> 00:11:54,873
实际的峰值性能呢

311
00:11:54,873 --> 00:11:56,985
会在后面去讲到

312
00:11:56,985 --> 00:11:58,985
现在有一个前提的假设

313
00:11:58,985 --> 00:12:01,985
假设现在的所有的PE

314
00:12:01,985 --> 00:12:03,489
就是所有的计算单元呢

315
00:12:03,489 --> 00:12:04,985
计算性能都是相同的

316
00:12:04,985 --> 00:12:08,360
PE核心执行单元的数量越多

317
00:12:08,360 --> 00:12:09,360
理论上呢

318
00:12:09,360 --> 00:12:12,504
峰值性能肯定是越高的

319
00:12:12,504 --> 00:12:15,360
但是呢这是一条直线

320
00:12:15,360 --> 00:12:17,024
计算性能

321
00:12:17,024 --> 00:12:19,328
是跟数据相关的

322
00:12:19,360 --> 00:12:21,360
只要它PE有数据

323
00:12:21,360 --> 00:12:22,360
它能够执行起来

324
00:12:22,360 --> 00:12:23,616
那这个时候呢

325
00:12:23,616 --> 00:12:27,360
就对这个图进行填充

326
00:12:27,360 --> 00:12:29,088
分成红色跟绿色

327
00:12:29,088 --> 00:12:30,773
红色这个部分呢

328
00:12:30,773 --> 00:12:33,360
就是带宽的上限

329
00:12:33,360 --> 00:12:37,059
带宽会制约整个运算的峰值性能

330
00:12:37,059 --> 00:12:39,043
假设带宽传输的数据非常慢

331
00:12:39,043 --> 00:12:40,043
那这个时候

332
00:12:40,043 --> 00:12:42,043
不管PE数量有多少

333
00:12:42,043 --> 00:12:44,043
它都不可能完全上去的

334
00:12:44,043 --> 00:12:45,685
因为PE的利用率非常高

335
00:12:45,685 --> 00:12:47,043
刚才也讲到了

336
00:12:47,043 --> 00:12:49,043
假设带宽

337
00:12:49,043 --> 00:12:50,681
已经不再是瓶颈了

338
00:12:50,681 --> 00:12:52,288
它能够快速的把数据

339
00:12:52,288 --> 00:12:53,427
传到给PE

340
00:12:53,427 --> 00:12:54,579
那这个时候呢

341
00:12:54,579 --> 00:12:56,043
PE的核心数量了

342
00:12:56,043 --> 00:12:58,043
就是右边的这个绿色

343
00:12:58,043 --> 00:12:59,454
compute bounded

344
00:12:59,454 --> 00:13:00,709
性能的峰值呢

345
00:13:00,709 --> 00:13:02,043
是跟带宽

346
00:13:02,043 --> 00:13:04,043
还有PE计算的能力

347
00:13:04,043 --> 00:13:05,043
是相关的

348
00:13:05,043 --> 00:13:06,043
这个时候

349
00:13:06,043 --> 00:13:09,043
必须要寻求一个最平衡的点

350
00:13:09,043 --> 00:13:10,043
就是这个点

351
00:13:10,043 --> 00:13:11,043
能够让

352
00:13:11,043 --> 00:13:13,043
可能在一定数量的PE之下

353
00:13:13,043 --> 00:13:14,043
一定的带宽之下

354
00:13:14,043 --> 00:13:17,043
能够让性能发挥到最好

355
00:13:17,043 --> 00:13:18,043
而不是说

356
00:13:18,043 --> 00:13:20,043
我的带宽设计的非常小

357
00:13:20,043 --> 00:13:21,043
但是我的PE

358
00:13:21,043 --> 00:13:23,043
我的核心执行数量非常多

359
00:13:23,043 --> 00:13:24,043
那这个时候呢

360
00:13:24,043 --> 00:13:26,043
是没有办法充分发挥

361
00:13:26,043 --> 00:13:28,043
计算的性能的

362
00:13:28,043 --> 00:13:29,043
于是呢

363
00:13:29,043 --> 00:13:30,043
就着这个理论

364
00:13:30,043 --> 00:13:32,306
继续往下看一看

365
00:13:32,700 --> 00:13:33,852
3和4呢

366
00:13:33,852 --> 00:13:35,043
讲的是一个相同的概念

367
00:13:35,043 --> 00:13:36,043
PE的数量

368
00:13:36,043 --> 00:13:39,043
对整个峰值的算力的影响

369
00:13:39,443 --> 00:13:41,265
step5红色这条线呢

370
00:13:41,265 --> 00:13:42,043
就很有意思了

371
00:13:42,043 --> 00:13:44,340
就是PE执行单元的数量

372
00:13:44,340 --> 00:13:45,517
小于cache

373
00:13:45,517 --> 00:13:47,951
小于存储单元的时候呢

374
00:13:47,951 --> 00:13:49,493
就会严重的约束了

375
00:13:49,493 --> 00:13:51,818
整个峰值的算力

376
00:13:52,243 --> 00:13:53,818
接下来看一下

377
00:13:53,818 --> 00:13:56,010
再往下step6的时候

378
00:13:56,010 --> 00:13:57,418
就是另外一种情况了

379
00:13:57,418 --> 00:13:58,506
这里的step呢

380
00:13:58,506 --> 00:13:59,696
不是指每一步

381
00:13:59,696 --> 00:14:01,424
而是指具体的case

382
00:14:01,424 --> 00:14:02,448
具体的情况

383
00:14:02,448 --> 00:14:03,818
那现在呢

384
00:14:03,818 --> 00:14:05,520
更低的算力的利用率

385
00:14:05,520 --> 00:14:07,026
或者更低的PE的利用率

386
00:14:07,026 --> 00:14:10,136
是由于带宽不足所影响的

387
00:14:10,136 --> 00:14:11,498
在step7里面

388
00:14:11,498 --> 00:14:12,650
性能就更差了

389
00:14:12,650 --> 00:14:14,793
现在PE的利用这么低

390
00:14:14,793 --> 00:14:17,818
同样是因为不足的内存带宽

391
00:14:17,818 --> 00:14:19,818
还有不足的存取空间

392
00:14:19,818 --> 00:14:21,207
或者叫做cache

393
00:14:21,207 --> 00:14:22,103
那这个时候呢

394
00:14:22,103 --> 00:14:23,651
就会严重的制约到

395
00:14:23,651 --> 00:14:26,084
整体的计算的性能

396
00:14:26,084 --> 00:14:28,084
而最好的计算性能呢

397
00:14:28,084 --> 00:14:30,084
就是绿色的这条线

398
00:14:30,084 --> 00:14:31,703
在整个计算体系呢

399
00:14:31,703 --> 00:14:33,084
需要找到带宽

400
00:14:33,084 --> 00:14:36,084
还有PE之间的一个平衡点

401
00:14:36,084 --> 00:14:41,084
从而发挥整个AI芯片的最优的性能值

402
00:14:41,509 --> 00:14:42,948
讲完这么多内容呢

403
00:14:42,948 --> 00:14:44,084
回头看看

404
00:14:44,084 --> 00:14:46,527
刚才给大家去分享的key metrics

405
00:14:46,527 --> 00:14:48,340
就是关键的实际指标

406
00:14:48,340 --> 00:14:50,579
跟计算体系的一个思考

407
00:14:50,579 --> 00:14:52,279
也就是作为一个总结了

408
00:14:52,279 --> 00:14:53,254
那首先第一个呢

409
00:14:53,254 --> 00:14:55,254
就是精度AI芯片希望能够处理

410
00:14:55,254 --> 00:14:57,254
各种各样没有规则的数据

411
00:14:57,254 --> 00:14:59,254
还有应对很多复杂的网络模型

412
00:14:59,254 --> 00:15:01,098
这种模式呢

413
00:15:01,098 --> 00:15:02,839
在英伟达的H100

414
00:15:02,839 --> 00:15:04,759
还有华为的NPU里面呢

415
00:15:04,759 --> 00:15:06,254
都是有不同异构的IP

416
00:15:06,254 --> 00:15:07,009
那第二种呢

417
00:15:07,009 --> 00:15:09,254
就是能够应对复杂的网络

418
00:15:09,254 --> 00:15:10,907
这个时候就要求计算单元

419
00:15:10,907 --> 00:15:12,763
有充足的冗余性

420
00:15:12,763 --> 00:15:14,690
所谓的计算单元有充足的冗余性呢 

421
00:15:14,690 --> 00:15:16,354
就是不仅仅能够处理

422
00:15:16,354 --> 00:15:17,840
cube矩阵的运算

423
00:15:17,840 --> 00:15:19,184
还要处理vector

424
00:15:19,184 --> 00:15:20,160
还有scalar

425
00:15:20,160 --> 00:15:22,336
标量和向量的数据

426
00:15:22,336 --> 00:15:23,040
那第二点呢

427
00:15:23,040 --> 00:15:24,840
就是刚才大量的提到的

428
00:15:24,840 --> 00:15:26,840
还有吞吐量和时延

429
00:15:26,840 --> 00:15:27,840
吞吐量呢

430
00:15:27,840 --> 00:15:28,840
除了看峰值的算力

431
00:15:28,840 --> 00:15:29,840
很重要的就要看

432
00:15:29,840 --> 00:15:31,840
PE的平均的利用率

433
00:15:31,840 --> 00:15:32,840
PE的利用率越多

434
00:15:32,840 --> 00:15:34,840
性能肯定是越好的

435
00:15:34,840 --> 00:15:36,390
另外还要看一下

436
00:15:36,390 --> 00:15:38,374
sota网络模型的整体的时间

437
00:15:38,374 --> 00:15:40,840
这个时候就很有意思的会去提到 

438
00:15:40,840 --> 00:15:42,840
一个很重要的内容

439
00:15:42,840 --> 00:15:43,840
就是mlperf

440
00:15:43,840 --> 00:15:45,840
它是英伟达跟谷歌

441
00:15:45,840 --> 00:15:46,840
去主导的一个

442
00:15:46,840 --> 00:15:48,840
AI的benchmark

443
00:15:48,840 --> 00:15:50,840
去提升相同的网络模型

444
00:15:50,840 --> 00:15:51,840
不同的AI框架

445
00:15:51,840 --> 00:15:53,840
不同的AI的芯片之下

446
00:15:53,840 --> 00:15:55,360
它的整体的性能

447
00:15:55,360 --> 00:15:56,384
而食盐呢

448
00:15:56,384 --> 00:16:00,288
更多的是希望能够时延越少越好嘛 

449
00:16:00,288 --> 00:16:01,840
很重要的两个优化点

450
00:16:01,840 --> 00:16:03,282
就是优化带宽呢

451
00:16:03,282 --> 00:16:05,840
优化多级的缓存

452
00:16:05,840 --> 00:16:07,504
另外三个内容

453
00:16:07,504 --> 00:16:09,360
能耗系统的价格

454
00:16:09,360 --> 00:16:10,640
还有易用性

455
00:16:10,640 --> 00:16:11,344
那能耗呢

456
00:16:11,344 --> 00:16:12,112
刚才讲了

457
00:16:12,112 --> 00:16:13,840
它跟部署的场景

458
00:16:13,840 --> 00:16:15,096
是相关的

459
00:16:15,096 --> 00:16:16,312
手机

460
00:16:16,312 --> 00:16:17,720
肯定能耗越少越好

461
00:16:17,720 --> 00:16:18,424
但是呢

462
00:16:18,424 --> 00:16:19,448
数据中心

463
00:16:19,448 --> 00:16:20,645
可以有大量的供电

464
00:16:20,645 --> 00:16:21,413
有液冷

465
00:16:21,413 --> 00:16:24,059
可以有另外的应用场景去驱动 

466
00:16:24,059 --> 00:16:24,955
例如训练场景呢

467
00:16:24,955 --> 00:16:26,840
大部分都在训练中心

468
00:16:26,840 --> 00:16:28,281
在AI计算中心

469
00:16:28,281 --> 00:16:28,985
另外呢

470
00:16:28,985 --> 00:16:30,393
很重要的一点

471
00:16:30,393 --> 00:16:32,633
就是减少内存的读写啊

472
00:16:32,633 --> 00:16:33,465
等各种方法

473
00:16:33,465 --> 00:16:35,015
去降低能耗

474
00:16:35,015 --> 00:16:36,662
缓存cache呢

475
00:16:36,662 --> 00:16:37,840
是对价格

476
00:16:37,840 --> 00:16:38,840
非常的敏感

477
00:16:38,840 --> 00:16:39,840
另外几个呢

478
00:16:39,840 --> 00:16:41,352
就是电路设计了

479
00:16:41,352 --> 00:16:42,312
PE的数量

480
00:16:42,312 --> 00:16:43,336
芯片的大小

481
00:16:43,336 --> 00:16:44,840
还有拉米的制程数

482
00:16:44,840 --> 00:16:46,305
对价格

483
00:16:46,305 --> 00:16:47,840
也是非常之敏感的

484
00:16:47,840 --> 00:16:48,840
另外呢

485
00:16:48,840 --> 00:16:49,840
整体的应用性

486
00:16:49,840 --> 00:16:51,840
主要是跟软件栈

487
00:16:51,840 --> 00:16:52,840
相关的

488
00:16:52,840 --> 00:16:54,600
软件栈做得越好

489
00:16:54,600 --> 00:16:56,840
应用性肯定会越好

490
00:16:56,840 --> 00:16:57,840
而这个时候呢

491
00:16:57,840 --> 00:16:59,484
不仅仅只强调性能

492
00:16:59,484 --> 00:17:00,840
而且还要看看

493
00:17:00,840 --> 00:17:02,344
应用性

494
00:17:02,840 --> 00:17:03,608
卷的不行了

495
00:17:03,608 --> 00:17:04,608
卷的不行了

496
00:17:04,608 --> 00:17:06,225
记得一键三连加关注哦 

497
00:17:06,225 --> 00:17:07,225
所有的内容

498
00:17:07,225 --> 00:17:08,225
都会开源在

499
00:17:08,225 --> 00:17:09,410
下面这条链接里面

500
00:17:09,735 --> 00:17:11,147
摆了个拜