Chapter 5: Unsupervised Learning
Acknowledgment: Most of the knowledge comes from Yuan Yang's course "Machine Learning".
Principle component analysis(PCA)
The direction keeping more variance is more important.
variance is defined this way:
\[E_{\mathbf{x}_i} \left[ \langle \mathbf{v}, \mathbf{x}_i \rangle^2 \right]= \frac{1}{n} \sum_{i=1}^{n} (\mathbf{v}^{\top} \mathbf{x}_i)^2 = \frac{1}{n} \mathbf{v}^{\top}\mathbf{X}\mathbf{X}^{\top} \mathbf{v} \]Here $ \mathbf{X} = [\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_n] \in \mathbb{R}^{d \times n} $.
我们的目标就是
\[\max_{\mathbf{v}} \mathbf{v}^{\top} \mathbf{XX}^{\top} \mathbf{v} \quad \text{s.t. } \mathbf{v}^{\top} \mathbf{v} = 1 \]那只需要求出\(\mathbf{XX}^{\top}\)最大的那个特征值对应的特征向量就可以了。
Relationship to SVD
对于任意的 \(X \in \mathbb{R}^{
标签:bar,frac,log,text,sum,学习,监督,机器,pi From: https://www.cnblogs.com/yzc5827/p/17979929