14 - 4 - Principal Component Analysis Algorithm (15 min).srt

1
00:00:00,340 --> 00:00:01,410
In this video I'd like
在这个视频中，我将(字幕翻译：中国海洋大学，孙中卫)

2
00:00:01,550 --> 00:00:03,020
to tell you about the principle
介绍

3
00:00:03,340 --> 00:00:04,570
components analysis algorithm.
主成分分析算法

4
00:00:05,600 --> 00:00:06,560
And by the end of this
在视频的最后，

5
00:00:06,710 --> 00:00:09,200
video you know to implement PCA for yourself.
你应该能独自使用主成分分析算法。

6
00:00:10,170 --> 00:00:12,540
And use it reduce the dimension of your data.
同时，能够使用PCA对自己的数据进行降维。

7
00:00:13,100 --> 00:00:14,690
Before applying PCA, there is
在使用PCA之前，

8
00:00:14,800 --> 00:00:17,760
a data pre-processing step which you should always do.
将首先进行数据预处理。

9
00:00:18,510 --> 00:00:20,220
Given the trading sets of the
给定一个交易例子的集合，

10
00:00:20,520 --> 00:00:22,290
examples is important to
重要的步骤是

11
00:00:22,600 --> 00:00:24,070
always perform mean normalization,
要总是执行均值标准化。

12
00:00:25,330 --> 00:00:26,140
and then depending on your data,
依据你的数据，

13
00:00:26,840 --> 00:00:28,540
maybe perform feature scaling as well.
大概也需要进行特征的缩放。

14
00:00:29,620 --> 00:00:30,950
this is very similar to the
这是很相似的，

15
00:00:31,650 --> 00:00:33,250
mean normalization and feature scaling
在均值标准化过程与特征缩放的过程之间

16
00:00:34,080 --> 00:00:36,580
process that we have for supervised learning.
在我们已学习的监督学习中。

17
00:00:36,910 --> 00:00:38,240
In fact it's exactly the
实际上，确实是

18
00:00:38,390 --> 00:00:40,160
same procedure except that we're
相同过程，除了我们现在

19
00:00:40,310 --> 00:00:41,790
doing it now to our unlabeled
对未标记数据做的，

20
00:00:42,930 --> 00:00:43,670
data, X1 through Xm.
X1到Xm。

21
00:00:44,180 --> 00:00:45,530
So for mean normalization we
对于均值标准化，

22
00:00:45,720 --> 00:00:47,080
first compute the mean of
我们首先计算

23
00:00:47,390 --> 00:00:49,070
each feature and then
每个特征的均值，

24
00:00:49,340 --> 00:00:50,900
we replace each feature, X,
我们取代每个特征X用

25
00:00:51,150 --> 00:00:52,680
with X minus its mean,
X减去它的均值。

26
00:00:52,810 --> 00:00:54,120
and so this makes each feature
这将使每个特征

27
00:00:54,520 --> 00:00:57,450
now have exactly zero mean
现在有个恰当的零均值。

28
00:00:58,690 --> 00:01:00,690
The different features have very different scales.
不同的特征有非常不同的缩放。

29
00:01:01,540 --> 00:01:03,050
So for example, if x1
例如，x1

30
00:01:03,080 --> 00:01:04,060
is the size of a house, and
是房子的尺寸，

31
00:01:04,100 --> 00:01:05,390
x2 is the number of bedrooms, to
x2是卧室的数量，

32
00:01:05,580 --> 00:01:07,370
use our earlier example, we
对于我们先前的例子，

33
00:01:07,480 --> 00:01:08,680
then also scale each feature
我们也可以缩放每个特征

34
00:01:09,130 --> 00:01:10,540
to have a comparable range of values.
获取一个相对的价值范围。

35
00:01:10,980 --> 00:01:12,490
And so, similar to what
相似于

36
00:01:12,680 --> 00:01:13,860
we had with supervised learning,
我们之前的监督学习，

37
00:01:14,060 --> 00:01:16,200
we would take x, i substitute
我们提取xi

38
00:01:16,680 --> 00:01:17,620
j, that's the j feature
的第j个特征。

39
00:01:23,250 --> 00:01:25,530
and so we would
所以我们能够

40
00:01:25,890 --> 00:01:27,610
subtract of the mean,
减去均值。

41
00:01:28,370 --> 00:01:29,520
now that's what we have on top, and then divide by sj.
想在对于我们上面做的，除以sj.

42
00:01:29,610 --> 00:01:30,020
Here, sj is some measure of the beta values of feature j.  So, it could be the max minus
在这里，sj是特征j的beta值的一些测量值。所以，它是最大减去

43
00:01:30,080 --> 00:01:31,310
min value, or more commonly,
最小值，或更普通的。

44
00:01:31,890 --> 00:01:33,540
it is the standard deviation of
sj是一个特征j的偏差标准。

45
00:01:33,640 --> 00:01:35,520
feature j. Having done

46
00:01:36,230 --> 00:01:39,480
this sort of data pre-processing, here's what the PCA algorithm does.
做了数据预处理流程后，这是PCA算法做的。

47
00:01:40,620 --> 00:01:41,630
We saw from the previous video
我们能从先前的视频中看到

48
00:01:41,960 --> 00:01:43,050
that what PCA does is, it
PCA所做的，

49
00:01:43,170 --> 00:01:44,520
tries to find a lower
它尝试着找到一个

50
00:01:44,790 --> 00:01:46,080
dimensional sub-space onto which to
低维子空间，

51
00:01:46,170 --> 00:01:47,500
project the data, so as
进行数据处理，

52
00:01:47,650 --> 00:01:49,780
to minimize the squared projection
只为了最小化平方投影

53
00:01:50,540 --> 00:01:51,660
errors, sum of the
误差，

54
00:01:51,740 --> 00:01:53,080
squared projection errors, as the
最小化平方和投影误差，

55
00:01:53,420 --> 00:01:54,800
square of the length of
随着这些蓝线所表示的

56
00:01:54,870 --> 00:01:56,790
those blue lines that and so
的平方，

57
00:01:57,110 --> 00:01:58,510
what we wanted to do specifically
我们想做的是

58
00:01:59,210 --> 00:02:02,730
is find a vector, u1, which
找到一个向量，u1,

59
00:02:03,280 --> 00:02:04,750
specifies that direction or
指定这个方向或者

60
00:02:05,040 --> 00:02:06,630
in the 2D case we want
在2维中，我们

61
00:02:06,880 --> 00:02:08,760
to find two vectors, u1 and
想找到2个向量，u1和

62
00:02:10,640 --> 00:02:12,980
u2, to define this surface
u2,来定义这个表面，

63
00:02:13,590 --> 00:02:14,610
onto which to project the data.
用于投射数据。

64
00:02:16,620 --> 00:02:17,920
So, just as a
正如

65
00:02:18,040 --> 00:02:19,160
quick reminder of what reducing
一个快速的提醒对于减少

66
00:02:19,730 --> 00:02:20,820
the dimension of the data means,
数据均值的维数。

67
00:02:21,490 --> 00:02:22,430
for this example on the
对左边的例子，

68
00:02:22,470 --> 00:02:23,560
left we were given
我们给予

69
00:02:23,680 --> 00:02:26,010
the examples xI, which are in r2.
西，在r2中。

70
00:02:26,300 --> 00:02:28,390
And what we
我们

71
00:02:28,660 --> 00:02:29,500
like to do is find
想找到

72
00:02:29,970 --> 00:02:32,400
a set of numbers zI in
一个数据集合zi在r

73
00:02:33,000 --> 00:02:34,950
r push to represent our data.
中代表我们的数据。

74
00:02:36,000 --> 00:02:37,820
So that's what from reduction from 2D to 1D means.
所以我们的均值从2维降到1维。

75
00:02:39,020 --> 00:02:41,450
So specifically by projecting
所以特别提醒

76
00:02:42,710 --> 00:02:44,080
data onto this red line there.
把数据投射到红线上。

77
00:02:44,800 --> 00:02:46,320
We need only one number to
我们仅需一个数字

78
00:02:46,450 --> 00:02:48,340
specify the position of the points on the line.
来表示在线上的点的位置。

79
00:02:48,590 --> 00:02:49,380
So i'm going to call that number
所以我们叫这个数字为

80
00:02:50,700 --> 00:02:51,830
z or z1.
z或z1.

81
00:02:52,020 --> 00:02:54,850
Z here  [xx] real number, so that's like a one dimensional vector.
Z是真实数字，以便像一个1维向量。

82
00:02:55,380 --> 00:02:56,650
So z1 just refers to
所以z1仅涉及

83
00:02:56,690 --> 00:02:58,080
the first component of this,
一个主成分，

84
00:02:58,280 --> 00:03:00,430
you know, one by one matrix, or this one dimensional vector.
它是一个1：1矩阵，或是一个一维向量。

85
00:03:01,670 --> 00:03:03,170
And so we need only
所以我们仅需要

86
00:03:03,490 --> 00:03:05,590
one number to specify the position of a point.
一个数字来指明一个点的位置。

87
00:03:06,330 --> 00:03:07,940
So if this example
所以如果这个例子

88
00:03:08,460 --> 00:03:09,510
here was my example
是我的一个例子X1,

89
00:03:10,610 --> 00:03:13,160
X1, then maybe that gets mapped here.
大概得到相应的映射。

90
00:03:13,900 --> 00:03:15,450
And if this example was X2
如果这个例子是x2

91
00:03:15,680 --> 00:03:17,250
maybe that example gets mapped
大概例子也得到映射。

92
00:03:17,530 --> 00:03:18,790
And so this point
所以对应的点

93
00:03:19,060 --> 00:03:20,400
here will be Z1
将是z1

94
00:03:20,840 --> 00:03:21,920
and this point here will be
这个点对应的将是

95
00:03:22,080 --> 00:03:24,240
Z2, and similarly we
z2,相似的

96
00:03:24,620 --> 00:03:26,410
would have those other points
我们将有其他的点对于这些，

97
00:03:26,840 --> 00:03:30,230
for These, maybe X3,
大概是x3,

98
00:03:30,510 --> 00:03:32,550
X4, X5 get mapped to Z1, Z2, Z3.
x4,x5,得到映射z1,z2,和z3.

99
00:03:34,360 --> 00:03:35,940
So What PCA has
所以PCA要做的是

100
00:03:36,050 --> 00:03:36,830
to do is we need to
我们需要

101
00:03:36,930 --> 00:03:38,920
come up with a way to compute two things.
想出一个方法计算两个事情。

102
00:03:39,310 --> 00:03:40,710
One is to compute these vectors,
一个是计算这些向量

103
00:03:41,830 --> 00:03:44,970
u1, and in this case u1 and u2.
例如u1,和在哪个事件中的u1和u2.

104
00:03:45,230 --> 00:03:46,880
And the other is
另一个是

105
00:03:47,130 --> 00:03:48,140
how do we compute these numbers,
如何来计算这些数字

106
00:03:49,360 --> 00:03:51,200
Z. So on the
Z.所以

107
00:03:51,430 --> 00:03:53,910
example on the left we're reducing the data from 2D to 1D.
在左边的例子汇总我们降维数据从2维到1维。

108
00:03:55,290 --> 00:03:56,100
In the example on the right,
在右边的例子中，

109
00:03:56,510 --> 00:03:58,100
we would be reducing data from
我们能降维数据从

110
00:03:58,450 --> 00:04:00,600
3 dimensional as in
三维

111
00:04:00,710 --> 00:04:04,840
r3, to zi, which is now two dimensional.
到zi,它是一个二维数据。

112
00:04:05,390 --> 00:04:07,790
So these z vectors would now be two dimensional.
所以这些z向量现在将是二维的。

113
00:04:08,450 --> 00:04:09,590
So it would be z1
所以它将是z1

114
00:04:10,150 --> 00:04:11,410
z2 like so, and so
z2这样的，所以

115
00:04:11,640 --> 00:04:12,940
we need to give away to compute
我们需要有方法计算

116
00:04:13,670 --> 00:04:15,410
these new representations, the z1
这些新的模型代表，

117
00:04:15,570 --> 00:04:17,350
and z2 of the data as well.
这些z1和z2的数据也一样。

118
00:04:18,280 --> 00:04:20,350
So how do you compute all of these quantities?
所以你如何来计算所有的参数那？

119
00:04:20,520 --> 00:04:21,520
It turns out that a mathematical
已经证明一个数学的

120
00:04:22,490 --> 00:04:23,660
derivation, also the mathematical
推导，也就是数学的

121
00:04:24,300 --> 00:04:26,020
proof, for what is
证明对于

122
00:04:26,090 --> 00:04:27,970
the right value U1, U2, Z1,
U1，U2,Z1,Z2等是正确的。

123
00:04:28,290 --> 00:04:29,480
Z2, and so on.

124
00:04:29,690 --> 00:04:31,230
That mathematical proof is very
这个数学证明是非常

125
00:04:31,480 --> 00:04:32,890
complicated and beyond the
复杂的，超出了

126
00:04:32,950 --> 00:04:34,620
scope of the course.
课程的范围。

127
00:04:35,280 --> 00:04:37,290
But once you've done  [xx] it
但是你已经做的，

128
00:04:37,590 --> 00:04:38,590
turns out that the procedure
证明这个过程

129
00:04:39,350 --> 00:04:40,570
to actually find the value
实际能够找到

130
00:04:41,200 --> 00:04:42,210
of u1 that you want
你想要的u1值，

131
00:04:42,950 --> 00:04:43,950
is not that hard, even though
这是不困难的，尽管

132
00:04:44,180 --> 00:04:45,640
so that the mathematical proof that
这个数学证明

133
00:04:45,840 --> 00:04:46,940
this value is the correct
这个值是正确的。

134
00:04:47,260 --> 00:04:48,450
value is someone more
这个值有些人想更多

135
00:04:48,700 --> 00:04:49,960
involved and more than i want to get into.
的涉及，超出想得到的i.

136
00:04:50,880 --> 00:04:52,070
But let me just describe the
但让我描述

137
00:04:52,480 --> 00:04:53,830
specific procedure that you
具体的过程，

138
00:04:53,960 --> 00:04:55,250
have to implement in order
你想执行为了

139
00:04:55,440 --> 00:04:56,450
to compute all of these
计算所有的

140
00:04:56,570 --> 00:04:57,840
things, the vectors, u1, u2,
参数，向量u1,u2

141
00:04:58,910 --> 00:05:00,980
the vector z.  Here's the procedure.
向量z,这是一个过程。

142
00:05:02,070 --> 00:05:02,970
Let's say we want to reduce
我们想把

143
00:05:03,170 --> 00:05:04,220
the data to n dimensions
n维的数据降到

144
00:05:04,840 --> 00:05:05,760
to k dimension What we're
k维，

145
00:05:06,760 --> 00:05:07,640
going to do is first
我们首先要做的

146
00:05:07,900 --> 00:05:09,400
compute something called the
是计算

147
00:05:09,830 --> 00:05:11,140
covariance matrix, and the covariance
协方差，这个协方差

148
00:05:11,700 --> 00:05:13,620
matrix is commonly denoted by
通常用

149
00:05:13,820 --> 00:05:15,050
this Greek alphabet which is
希腊字母表中

150
00:05:15,190 --> 00:05:16,880
the capital Greek alphabet sigma.
的 sigma表示。

151
00:05:18,000 --> 00:05:19,210
It's a bit unfortunate that the
不幸的是

152
00:05:19,310 --> 00:05:21,080
Greek alphabet sigma looks exactly
这个希腊字母sigma看起来

153
00:05:21,760 --> 00:05:22,710
like the summation symbols.
像一个求和标记。

154
00:05:23,210 --> 00:05:24,620
So this is the
所以这里的

155
00:05:24,700 --> 00:05:26,220
Greek alphabet Sigma is used
希腊字母Sigma被用来

156
00:05:26,420 --> 00:05:29,540
to denote a matrix and this here is a summation symbol.
标记一个矩阵，在那里是一个求和标记。

157
00:05:30,510 --> 00:05:32,330
So hopefully in these slides
所以希望在幻灯片

158
00:05:32,680 --> 00:05:34,190
there won't be ambiguity about which
不要有模糊对于

159
00:05:34,410 --> 00:05:36,340
is Sigma Matrix, the
Sigma 矩阵，

160
00:05:36,520 --> 00:05:37,850
matrix, which is a
另一个是

161
00:05:38,090 --> 00:05:39,620
summation symbol, and hopefully
求和标记，希望

162
00:05:39,940 --> 00:05:41,460
it will be clear from context when
它能被区分从内容上，

163
00:05:41,820 --> 00:05:43,510
I'm using each one.
当我们使用每一个的时候。

164
00:05:43,740 --> 00:05:44,790
How do you compute this matrix
如何计算这个矩阵

165
00:05:45,530 --> 00:05:46,550
let's say we want to
比如说我们想

166
00:05:47,135 --> 00:05:47,640
store it in an octave variable
存储它到一个octave变量中，

167
00:05:48,120 --> 00:05:49,970
variable called sigma.
变量叫sigma。

168
00:05:50,840 --> 00:05:51,890
What we need to do is
我们需要做什么

169
00:05:52,030 --> 00:05:53,660
compute something called the
来计算被叫做

170
00:05:54,130 --> 00:05:56,190
eigenvectors of the matrix sigma.
矩阵sigma的特征向量。

171
00:05:57,560 --> 00:05:58,450
And an octave, the way you
对于一个octave，你的方法

172
00:05:58,590 --> 00:05:59,820
do that is you use this
是用这个

173
00:05:59,970 --> 00:06:01,020
command, u s v equals
命令

174
00:06:01,350 --> 00:06:02,600
s v d of sigma.
或公式来计算。

175
00:06:03,650 --> 00:06:06,090
SVD, by the way, stands for singular value decomposition.
SVD代表奇异值分解，

176
00:06:08,520 --> 00:06:10,590
This is a Much
这是一个更

177
00:06:10,790 --> 00:06:12,660
more advanced single value composition.
加高级的单值组成。

178
00:06:14,450 --> 00:06:15,560
It is much more advanced linear
它也是更加高级的线性

179
00:06:15,800 --> 00:06:16,950
algebra than you actually need
代数比你实际需要

180
00:06:16,950 --> 00:06:18,770
to know but now It turns out
知道的但结果是

181
00:06:18,950 --> 00:06:20,250
that when sigma is equal
sigma等同

182
00:06:20,480 --> 00:06:21,800
to matrix there is
矩阵，有几个

183
00:06:21,880 --> 00:06:23,420
a few ways to compute these are
方法可以在高维向量中

184
00:06:23,610 --> 00:06:25,810
high in vectors and If you
进行计算，如果你

185
00:06:25,960 --> 00:06:27,350
are an expert in linear algebra
是一个专家在线性代数上，

186
00:06:27,700 --> 00:06:28,610
and if you've heard of high in
如果你已经听过高维的

187
00:06:28,860 --> 00:06:30,170
vectors before you may know
向量在你知道

188
00:06:30,350 --> 00:06:31,660
that there is another octet function
有另一个叫I的octet函数，

189
00:06:31,990 --> 00:06:33,420
called I, which can
它能够

190
00:06:33,520 --> 00:06:35,030
also be used to compute the same thing.
被用来计算相同的事。

191
00:06:35,950 --> 00:06:36,980
and It turns out that the
已经证明这个

192
00:06:37,370 --> 00:06:39,180
SVD function and the
SVD函数和

193
00:06:39,290 --> 00:06:40,310
I function it will give
这个I函数将给

194
00:06:40,370 --> 00:06:42,170
you the same vectors, although SVD
你相同的向量，但SVD

195
00:06:42,840 --> 00:06:44,210
is a little more numerically stable.
将更加具有数据稳定性。

196
00:06:44,540 --> 00:06:45,890
So I tend to use SVD, although
所以我趋向于用SVD，尽管

197
00:06:46,140 --> 00:06:47,040
I have a few friends that use
我有几个朋友用

198
00:06:47,280 --> 00:06:48,720
the I function to do
I函数也能很好的做这些，

199
00:06:48,920 --> 00:06:50,050
this as well but when you
很好的做这些，当你

200
00:06:50,130 --> 00:06:51,270
apply this to a covariance matrix
用I函数求一个协方差矩阵

201
00:06:51,750 --> 00:06:52,960
sigma it gives you the same thing.
sigma，给你的结果是相同。

202
00:06:53,870 --> 00:06:55,070
This is because the covariance matrix
这是因为协方差矩阵

203
00:06:55,500 --> 00:06:57,250
always satisfies a mathematical
总是满足一个数学

204
00:06:57,940 --> 00:07:00,560
Property called symmetric positive definite
属性叫正定矩阵。

205
00:07:01,360 --> 00:07:02,140
You really don't need to know
你不需要知道

206
00:07:02,280 --> 00:07:03,890
what that means, but the SVD
他的含义，但这个SVD

207
00:07:05,340 --> 00:07:07,090
and I-functions are different functions but
和I函数是不同的函数
208
00:07:07,400 --> 00:07:08,670
when they are applied to a
当他们被运用到

208
00:07:08,780 --> 00:07:10,410
covariance matrix which can
协方差矩阵时这被

209
00:07:10,550 --> 00:07:12,080
be proved to always satisfy this
证明总是满足

210
00:07:13,190 --> 00:07:15,220
mathematical property; they'll always give you the same thing.
这个数学属性；这两个函数给予相同的结果。

211
00:07:16,580 --> 00:07:19,180
Okay, that was probably much more linear algebra than you needed to know.
好的，这大概将是更高深的线性代数比我们知道的。

212
00:07:19,260 --> 00:07:22,380
In case none of that made sense, don't worry about it.
如果没有明白的话，不用担心。

213
00:07:22,560 --> 00:07:23,490
All you need to know is that
你需要知道的

214
00:07:24,130 --> 00:07:27,840
this system command you
这个系统命令，你能够

215
00:07:28,140 --> 00:07:29,690
should implement in Octave.
用它在Octave。

216
00:07:30,080 --> 00:07:30,550
And if you're implementing this in a
你也可以用它在

217
00:07:30,710 --> 00:07:32,120
different language than Octave or MATLAB,
不同语言如Octave或MATLAB。

218
00:07:32,710 --> 00:07:33,790
what you should do is find
我们应该做的是找到

219
00:07:34,190 --> 00:07:35,860
the numerical linear algebra library
数值线性代数库

220
00:07:36,730 --> 00:07:37,960
that can compute the SVD
能够计算 SVD

221
00:07:38,230 --> 00:07:40,460
or singular value decomposition, and
或奇异值分解，

222
00:07:40,970 --> 00:07:42,680
there are many such libraries for
有许多这样的库

223
00:07:43,570 --> 00:07:45,060
probably all of the major programming languages.
对大多数的主要编程语言。

224
00:07:45,300 --> 00:07:46,920
People can use that to
人们能够用这些来

225
00:07:47,050 --> 00:07:48,920
compute the matrices u,
计算矩阵u，

226
00:07:49,200 --> 00:07:52,770
s, and d of the covariance matrix sigma.
s,和协方差矩阵sigma d.

227
00:07:53,340 --> 00:07:54,490
So just to fill
仅为了满足

228
00:07:54,620 --> 00:07:56,090
in some more details, this covariance
一些细节，这个协方差

229
00:07:56,660 --> 00:07:58,080
matrix sigma will be
矩阵sigma将

230
00:07:58,250 --> 00:08:01,480
an n by n matrix.
是 N—N的矩阵。

231
00:08:02,250 --> 00:08:03,240
And one way to see that
一个明白的方法是

232
00:08:03,510 --> 00:08:04,220
is if you look at the definition
你能够看到定义

233
00:08:05,250 --> 00:08:06,280
this is an n by 1
一个N-1的

234
00:08:06,660 --> 00:08:08,680
vector and this
向量，

235
00:08:08,920 --> 00:08:10,830
here I transpose is
另一个是

236
00:08:11,010 --> 00:08:13,260
1 by N so the
1-N的，所以

237
00:08:13,380 --> 00:08:14,480
product of these two things
这两个的结果

238
00:08:15,150 --> 00:08:15,800
is going to be an N
是一个N-N

239
00:08:16,570 --> 00:08:17,530
by N matrix.
的矩阵。

240
00:08:19,100 --> 00:08:22,130
1xN transfers, 1xN, so
1-N 转移，1-N，所以

241
00:08:22,280 --> 00:08:22,840
there's an NxN matrix and when
一个N-N的矩阵，当

242
00:08:22,910 --> 00:08:23,710
we add up all of these you still
我们合计这些后，你还是

243
00:08:23,840 --> 00:08:26,140
have an NxN matrix.
有一个N-N的矩阵。

244
00:08:27,600 --> 00:08:29,920
And what the SVD outputs three
SVD输出的是三

245
00:08:30,500 --> 00:08:32,580
matrices, u, s, and
个矩阵，u，s,和

246
00:08:32,830 --> 00:08:35,070
v.  The thing you really need out of the SVD is the u matrix.
v. 你真的需要的SVD输出是u矩阵。

247
00:08:36,230 --> 00:08:40,160
The u matrix will also be a NxN matrix.
这个 u矩阵也将是一个N-N的矩阵。

248
00:08:41,510 --> 00:08:42,280
And if we look at the
我们看到的

249
00:08:42,350 --> 00:08:43,260
columns of the U
u矩阵的列

250
00:08:43,480 --> 00:08:45,330
matrix it turns
被证明：

251
00:08:45,630 --> 00:08:47,210
out that the columns
u矩阵

252
00:08:48,570 --> 00:08:50,180
of the U matrix will be
的列将是

253
00:08:50,350 --> 00:08:53,860
exactly those vectors, u1,
这些向量， u1,

254
00:08:54,260 --> 00:08:56,290
u2 and so on.
u2等。

255
00:08:57,640 --> 00:08:59,330
So u, will be matrix.
所以u ,将是矩阵。

256
00:09:00,910 --> 00:09:01,830
And if we want to reduce
如果我们想减

257
00:09:02,230 --> 00:09:03,200
the data from n dimensions
少数据的维数从n维

258
00:09:03,800 --> 00:09:05,380
down to k dimensions, then what
到k维。

259
00:09:05,490 --> 00:09:07,950
we need to do is take the first k vectors.
我们需要做的是提取前k个向量。

260
00:09:09,800 --> 00:09:12,670
that gives us u1 up
这将给我们u1

261
00:09:12,860 --> 00:09:14,700
to uK which gives
到uK的向量，给了我们

262
00:09:14,780 --> 00:09:16,930
us the K direction onto which
K个方向，我们想把数据

263
00:09:17,200 --> 00:09:19,770
we want to project the data.
投射的方向。

264
00:09:20,090 --> 00:09:21,640
the rest of the procedure from
这个剩余的过程是

265
00:09:22,410 --> 00:09:24,170
this SVD numerical linear
这个SVD数值线性

266
00:09:24,490 --> 00:09:25,580
algebra routine we get this
代数的程序，我们得到

267
00:09:25,840 --> 00:09:27,140
matrix u.  We'll call
矩阵u. 我们称

268
00:09:27,530 --> 00:09:29,080
these columns u1-uN.
这些列为u1-uN。

269
00:09:30,580 --> 00:09:31,620
So, just to wrap up the
所以，为了结束这个

270
00:09:31,830 --> 00:09:32,520
description of the rest of
剩余过程

271
00:09:32,540 --> 00:09:34,550
the procedure, from the SVD
的描述，来自于SVD

272
00:09:35,320 --> 00:09:36,940
numerical linear algebra routine we
数值线性代数程序，我们

273
00:09:37,240 --> 00:09:38,650
get these matrices u, s,
得到这些矩阵u，s,

274
00:09:38,830 --> 00:09:41,320
and d.  we're going
和d.我们将

275
00:09:41,900 --> 00:09:44,460
to use the first K columns
用这个矩阵的前

276
00:09:45,050 --> 00:09:46,310
of this matrix to get u1-uK.
K个列获取u1-uK的向量。

277
00:09:48,710 --> 00:09:49,460
Now the other thing we need
现在我们需要做的其它事情

278
00:09:49,700 --> 00:09:53,730
to is take my original
是获取我们的原始

279
00:09:54,110 --> 00:09:55,430
data set, X which is
数据集，X这是一个

280
00:09:55,630 --> 00:09:57,080
an RN And find a
RN值域。找到

281
00:09:57,250 --> 00:09:59,210
lower dimensional representation Z, which
一个低维的代表Z,他是

282
00:09:59,420 --> 00:10:01,280
is a R K for this data.
R K 对于这些数据。

283
00:10:01,570 --> 00:10:02,800
So the way we're
所以，我们

284
00:10:02,900 --> 00:10:03,930
going to do that is
将要做的是

285
00:10:04,180 --> 00:10:06,690
take the first K Columns of the U matrix.
获取u矩阵的前K列。

286
00:10:08,330 --> 00:10:09,790
Construct this matrix.
构建这个矩阵。

287
00:10:11,060 --> 00:10:13,040
Stack up U1, U2 and
累计U1,U2等

288
00:10:14,170 --> 00:10:16,690
so on up to U K in columns.
直到UK在列上。

289
00:10:17,350 --> 00:10:19,120
It's really basically taking, you know,
你知道我们本质上是

290
00:10:19,280 --> 00:10:20,350
this part of the matrix, the
获取矩阵的这部分，

291
00:10:20,530 --> 00:10:22,260
first K columns of this matrix.
这个矩阵的前K列。

292
00:10:23,420 --> 00:10:25,370
And so this is
所以这将获取

293
00:10:25,600 --> 00:10:26,920
going to be an N
一个N-K

294
00:10:27,200 --> 00:10:28,580
by K matrix.
的矩阵。

295
00:10:29,500 --> 00:10:30,690
I'm going to give this matrix a name.
我将给予这个矩阵一个名字，

296
00:10:30,880 --> 00:10:32,200
I'm going to call this matrix
我将叫这个矩阵为

297
00:10:32,930 --> 00:10:35,760
U, subscript "reduce," sort
U，下标"reduce"，这是一类，

298
00:10:36,090 --> 00:10:38,620
of a reduced version of the U matrix maybe.
U矩阵被降维的版本。

299
00:10:39,140 --> 00:10:41,250
I'm going to use it to reduce the dimension of my data.
我将用它对我的数据降维。

300
00:10:43,040 --> 00:10:43,950
And the way I'm going to compute Z is going
我将计算Z，将让

301
00:10:44,250 --> 00:10:45,960
to let Z be equal to this
Z等于这个

302
00:10:46,220 --> 00:10:49,570
U reduce matrix transpose times
U的降维矩阵转置相乘

303
00:10:50,010 --> 00:10:52,030
X. Or alternatively, you know,
X.或者，你知道

304
00:10:52,510 --> 00:10:53,860
to write down what this transpose means.
写下这个转置的意思。

305
00:10:54,630 --> 00:10:55,910
When I take this transpose of
当我获取这个

306
00:10:56,010 --> 00:10:57,920
this U matrix, what I'm
u矩阵的转置时，我将

307
00:10:58,010 --> 00:11:00,680
going to end up with is these vectors now in rows.
在行上结束这些向量。

308
00:11:00,950 --> 00:11:04,540
I have U1 transpose down to UK transpose.
我有U1转置到UK转置。

309
00:11:07,060 --> 00:11:08,860
Then take that times X,
然后乘以X,

310
00:11:09,700 --> 00:11:10,740
and that's how I get
这将让我

311
00:11:10,920 --> 00:11:12,670
my vector Z. Just to
得到向量Z.为了

312
00:11:12,740 --> 00:11:14,280
make sure that these dimensions make sense,
确定这些维数有意义，

313
00:11:15,370 --> 00:11:16,380
this matrix here is going
这个矩阵将是

314
00:11:16,560 --> 00:11:17,450
to be k by n
K-N的，

315
00:11:18,270 --> 00:11:19,350
and x here is going
x是

316
00:11:19,420 --> 00:11:20,530
to be n by 1
N-1,

317
00:11:20,750 --> 00:11:21,810
and so the product
所以这个结果

318
00:11:22,320 --> 00:11:24,330
here will be k by 1.
将是K-1的。

319
00:11:24,820 --> 00:11:27,920
And so z is k
所以z是k

320
00:11:28,790 --> 00:11:29,810
dimensional, is a k
维的，是一个k

321
00:11:30,010 --> 00:11:31,230
dimensional vector, which is exactly
维向量，这是

322
00:11:32,000 --> 00:11:33,180
what we wanted.
我们想要的。

323
00:11:33,550 --> 00:11:34,640
And of course these x's here right, can
当然，这个x是正确的，

324
00:11:34,890 --> 00:11:36,010
be Examples in our
将是例子在我们的

325
00:11:36,100 --> 00:11:36,970
training set can be examples
训练集，也是例子

326
00:11:37,540 --> 00:11:38,750
in our cross validation set, can be
在我们的交叉验证集中，

327
00:11:38,980 --> 00:11:40,330
examples in our test set, and
也是例子在我们的测试集中，

328
00:11:40,500 --> 00:11:41,590
for example if you know,
例如，如果你知道，

329
00:11:41,930 --> 00:11:43,830
I wanted to take training example i,
我想得到训练集i,

330
00:11:44,260 --> 00:11:45,910
I can write this as xi
我能够写这些作为xi,

331
00:11:47,270 --> 00:11:48,430
XI and that's what will
XI 和，将给我们

332
00:11:48,510 --> 00:11:50,080
give me ZI over there.
ZI是什么。

333
00:11:50,940 --> 00:11:53,140
So, to summarize, here's the
所以，总而言之，这是

334
00:11:53,460 --> 00:11:54,820
PCA algorithm on one slide.
幻灯片上的PCA算法。

335
00:11:56,250 --> 00:11:58,200
After mean normalization, to ensure
进行均值归一化后，为确保

336
00:11:58,420 --> 00:11:59,230
that every feature is zero mean
每一个特征都是零均值的，

337
00:11:59,610 --> 00:12:01,420
and optional feature scaling which You
任选特征缩放，你

338
00:12:02,280 --> 00:12:03,780
really should do feature scaling if
能够做特征的缩放，如果

339
00:12:03,890 --> 00:12:05,820
your features take on very different ranges of values.
你的特征能够呈现不同范围的值。

340
00:12:06,620 --> 00:12:08,610
After this pre-processing we compute
预处理完成之后，我们计算

341
00:12:09,130 --> 00:12:12,010
the carrier matrix Sigma like
这个载体矩阵Sigma像

342
00:12:12,240 --> 00:12:14,070
so by the
这个方法，

343
00:12:14,210 --> 00:12:16,340
way if your data is
如果你的数据

344
00:12:16,610 --> 00:12:17,780
given as a matrix
是被给予作为一个矩阵，

345
00:12:18,030 --> 00:12:18,960
like hits if you have your
你有的

346
00:12:19,230 --> 00:12:22,580
data Given in rows like this.
数据被给与在行中像这样。

347
00:12:22,780 --> 00:12:24,370
If you have a matrix X
如果你有一个矩阵X，

348
00:12:24,960 --> 00:12:26,190
which is your time trading sets
这是你的时间交易集

349
00:12:27,030 --> 00:12:28,830
written in rows where x1
被写在行上是x1转置

350
00:12:29,210 --> 00:12:30,400
transpose down to xm transpose,
到xm的转置。

351
00:12:31,530 --> 00:12:32,700
this covariance matrix sigma actually has
这个协方差矩阵sigma 实际上有

352
00:12:33,020 --> 00:12:36,040
a nice vectorizing implementation.
一个非常好的向量化实现。

353
00:12:37,390 --> 00:12:38,980
You can implement in octave,
你能够执行在octave。

354
00:12:39,440 --> 00:12:41,130
you can even run sigma equals 1
你能够甚至运行sigma等于

355
00:12:41,670 --> 00:12:45,250
over m, times x,
1/m乘以x的转置,

356
00:12:45,550 --> 00:12:46,440
which is this matrix up here,
这是这个矩阵放在哪。

357
00:12:47,250 --> 00:12:50,770
transpose times x and
乘以x,

358
00:12:50,980 --> 00:12:53,320
this simple expression, that's
这个简单的表达，这个

359
00:12:53,570 --> 00:12:55,070
the vectorize implementation of how
向量化的实现，是如何计算

360
00:12:55,220 --> 00:12:56,910
to compute the matrix sigma.
矩阵sigma。

361
00:12:58,020 --> 00:12:59,020
I'm not going to prove that today.
今天，我将不证明它。

362
00:12:59,160 --> 00:13:00,600
This is the correct vectorization whether you
正如你所知道的，这是一个正确的向量化，

363
00:13:00,740 --> 00:13:02,460
want, you can either numerically test
你或者单独的数值测试

364
00:13:02,870 --> 00:13:03,900
this on yourself by trying out an
它通过尝试octave。

365
00:13:03,980 --> 00:13:05,100
octave and making sure that
或者确信

366
00:13:05,840 --> 00:13:06,890
both this and this implementations
在它和它的实施中

367
00:13:06,920 --> 00:13:10,050
give the same answers or you Can try to prove it yourself mathematically.
有相同的答案，或者你能够通过数学证明他。

368
00:13:11,250 --> 00:13:12,330
Either way but this is the
无论怎么样，这是

369
00:13:12,430 --> 00:13:14,580
correct vectorizing implementation, without compusing next
正确的向量化引用，不需要要质疑。

370
00:13:16,480 --> 00:13:17,570
we can apply the SVD
我们能够运用SVD

371
00:13:17,920 --> 00:13:19,050
routine to get u, s,
程序得到u，s,

372
00:13:19,250 --> 00:13:20,840
and d. And then we
和d.我们

373
00:13:21,100 --> 00:13:22,720
grab the first k
抓取

374
00:13:23,050 --> 00:13:24,450
columns of the u
矩阵u的前k列，

375
00:13:24,660 --> 00:13:26,550
matrix you reduce and
你所要降维到的，

376
00:13:26,650 --> 00:13:28,540
finally this defines how
最终这个定义

377
00:13:28,740 --> 00:13:29,980
we go from a feature
我们如何得到从一个特征

378
00:13:30,290 --> 00:13:31,600
vector x to this
向量x到这

379
00:13:31,850 --> 00:13:34,340
reduce dimension representation z. And
个降维的代表z.

380
00:13:34,540 --> 00:13:35,760
similar to k Means
相似的k的意思，

381
00:13:36,090 --> 00:13:37,860
if you're apply PCA, they way
如果你运用PCA，

382
00:13:38,030 --> 00:13:40,300
you'd apply this is with vectors X and RN.
你也运用它到向量X和RN。

383
00:13:41,100 --> 00:13:43,990
So, this is not done with X-0 1.
所以，这里xo不能为1.

384
00:13:44,200 --> 00:13:46,080
So that was
这就是

385
00:13:46,990 --> 00:13:48,680
the PCA algorithm.
PCA算法。

386
00:13:50,120 --> 00:13:51,380
One thing I didn't do is
有件事情是我不能

387
00:13:51,590 --> 00:13:53,190
give a mathematical proof that
给予一个数学的证明，

388
00:13:53,520 --> 00:13:54,600
this There it actually give
这里它实际上

389
00:13:54,970 --> 00:13:56,560
the projection of the data onto
给予数据的投射

390
00:13:57,230 --> 00:13:58,730
the K dimensional subspace
到k维子空间

391
00:13:58,870 --> 00:14:00,620
 that actually
这实际

392
00:14:02,170 --> 00:14:04,800
minimizes the square projection error Proof
是最小化投射平方的误差证明

393
00:14:05,110 --> 00:14:07,170
of that is beyond the scope of this course.
超出了我们课程的学习范围。

394
00:14:07,700 --> 00:14:09,110
Fortunately the PCA algorithm
幸运的是这个PCA算法

395
00:14:09,470 --> 00:14:10,940
can be implemented in not
将不会被执行

396
00:14:11,320 --> 00:14:12,510
too many lines of code.
用太多的代码。

397
00:14:13,190 --> 00:14:14,510
and if you implement this in
如果你执行它在

398
00:14:14,640 --> 00:14:16,120
octave or algorithm, you
太多或者算法中，你

399
00:14:16,520 --> 00:14:17,590
actually get a very effective
实际能够得到一个非常有效

400
00:14:18,110 --> 00:14:19,710
dimensionality reduction algorithm.
的降维算法。

401
00:14:22,430 --> 00:14:23,850
So, that was the PCA algorithm.
所以，这就是PCA算法。

402
00:14:25,010 --> 00:14:26,290
One thing I didn't do was
一个事情是我不能

403
00:14:26,840 --> 00:14:28,420
give a mathematical proof that
给予一个数学上的证明，

404
00:14:29,170 --> 00:14:30,360
the U1 and U2 and so
U1和U2等，

405
00:14:30,720 --> 00:14:31,630
on and the Z and so
和Z等。

406
00:14:31,770 --> 00:14:32,830
on you get out of this
你得出的这个过程

407
00:14:32,980 --> 00:14:34,330
procedure is really the
是真实

408
00:14:34,680 --> 00:14:35,870
choices that would minimize
的选着是最小化

409
00:14:36,500 --> 00:14:37,800
these squared projection error.
平方投影误差。

410
00:14:38,140 --> 00:14:39,350
Right, remember we said What
记住我说的，

411
00:14:39,610 --> 00:14:40,660
PCA tries to do is try
PCA所做的是尝试找到

412
00:14:40,960 --> 00:14:42,170
to find a surface or line
一个面或线

413
00:14:42,570 --> 00:14:43,690
onto which to project the data
把数据投影到这个面或线上，

414
00:14:44,280 --> 00:14:46,340
so as to minimize to square projection error.
以便于最小化平方投影误差。

415
00:14:46,700 --> 00:14:48,610
So I didn't prove that this
我将不去证明它，

416
00:14:49,140 --> 00:14:50,680
that, and the mathematical proof
这个数学的证明

417
00:14:50,970 --> 00:14:52,520
of that is beyond the scope of this course.
超出了我们课程的范围。

418
00:14:53,170 --> 00:14:55,550
But fortunately the PCA algorithm can
但幸运的是这个PCA算法能够

419
00:14:55,730 --> 00:14:58,890
be implemented in not too many lines of octave code.
被执行不需要太多行octave代码。

420
00:14:59,350 --> 00:15:00,740
And if you implement this,
如果你执行它 ，

421
00:15:01,430 --> 00:15:02,560
this is actually what will
这实际上是我们将做的

422
00:15:02,770 --> 00:15:03,730
work, or this will work well,
或这将做的很好，

423
00:15:04,710 --> 00:15:05,940
and if you implement this algorithm,
如果你执行这个算法。

424
00:15:06,500 --> 00:15:09,220
you get a very effective dimensionality reduction algorithm.
你将得到一个非常有效的降维算法。

425
00:15:09,780 --> 00:15:10,650
That does do the right thing
这个做的一个正确的事情是

426
00:15:11,050 --> 00:15:13,460
of minimizing this square projection error.
最小化平方投影误差。