Diagonalization
$1\le \dim(E_{\lambda})\le $ multiplicity of \(\lambda\).
Assume \(T\) splits, it is diagonalizable iff $\dim(E_{\lambda})= $ multiplicity of \(\lambda\) for all \(\lambda\).
Cayley-Hamilton Theorem
Lemma 1. Let \(W\) be a \(T\)-invariant subspace. Then the characteristic polynomial of \(T_W\) divides the characteristic polynomial of \(T\).
Lemma 2. For each \(v\in V\), let \(k\) be the maximum integer that \(\beta=\{v,T(v),...,T^{k-1}(v)\}\) is linear independent. Then \(\text{span}(\{v,T(v),...,T^{m-1}(v)\})\) is an invariant subspace. Suppose \(a_0v+a_1T(v)+...+a_{k-1}T^{k-1}(v)+T^k(v)=0\), then the characteristic polynomial of \(T_W\) is $$f(t)=(-1)^{k} (a_0+a_1 t+...+a_{k-1}t^{k-1} +t^k ).$$
Proof. Write down \([T_W]_{\beta}\) and calculate its characteristic polynomial.
For any \(v\in V\), \(f(T)(v)=0.\) Also we know \(f\) divides the characteristic polynomial of \(T\). Thus \(T=T_0\).
Inner Product
Schur's Lemma. \(T\) is linear and its characteristic polynomial splits. Then \([T]_{\beta}\) is upper triangular for some orthonormal basis \(\beta\).
Proof. Let \(T^*(z)=\lambda z\). Show \(W^{\perp}\) is \(T\)-invariant for \(W=\text{span}(\{z\})\). By induction, there exists \(\gamma\) s.t. \([T_{W^{\perp}}]_{\gamma}\) is upper triangular. Thus \([T]_{\beta}\) is upper triangular for \(\beta=\gamma\cup\{z\}\).
Theorem 6.15 \(T\) is normal then \(T-c I\) is normal and \(T(z)=\lambda z\Rightarrow T^*(z)=\overline{\lambda}z\).
Theorem 6.16/6.17 \(T\) has orthogonormal basis consisting of eigenvectors iff:
- For complex case, \(T\) is normal. (\(TT^*=T^*T\).)
- For real case, \(T\) is self-adjoint. (\(T=T^*\).)
Unitary: \(TT^*=T^*T=I\) or \(\|T(x)\|=\|x\|\). (orthgonal for real case)
- equivalent to has orthonormal basis consisting of eigenvectors with eigenvalues of absolute value \(1\). (need self-adjoint for real case.)
- is the change-of-base matrix of some orthonormal basis.
\(T\) is a projection iff \(T=T^2\). A projection \(T\) is orthogonal iff \(R(T)^{\perp}=N(T)\). \(T\) is an orthogonal projection iff \(T^2=T=T^*\).
Spectral Theorem. \(T\) is normal (\(F=\mathbb{C}\)) or self-adjoint (\(F=\mathbb{R}\)). Let \(T_i\) be the orthogonal projection on \(E_{\lambda_i}\), then $$T=\lambda_1T_1+...+\lambda_kT_k.$$
Additionally, \(V=E_{\lambda_1}\oplus E_{\lambda_2}\oplus ...\oplus E_{\lambda_k},I=T_1+...+T_k,T_iT_j=\delta_{ij}T_i.\)
Corollary. \(T^*\) and all \(T_i\) are a polynomial of \(T\).
Bilinear form
\[H(x,y)=[\phi_{\beta}(x)]^t\psi_{\beta}(H)[\phi_{\beta}(y)]. \]Theorem 6.33 \(\psi_{\gamma}(H)=Q^T\psi_{\beta}(H)Q\), where \(Q=[I]_{\gamma}^{\beta}\).
Theorem 6.35 Let \(V\) be a vector space over a field not of characteristic two. Then every symmetric bilinear form is diagonalizable.
Theorem 6.36 Let \(V\) be a vector space over \(\mathbb{R}\). Then \(\psi_{\beta}(H)\) is diagonal for some orthonormal basis \(\beta\).
How to diagonalize a symmetric matrix?
Quadratic form
Corollary. There exists orthonormal basis \(\beta=\{v_1,...,v_k\}\) and sclars \(\lambda_1,...,\lambda_k\) s.t. for any \(x=a_1v_1+...+a_kv_k\), $$K(x,x)=\lambda_1 a_1^2 +...+\lambda_k a_k^2.$$
Singular value decomposition
- Compute semi-definite matrix \(AA^*\).
- Get the singular values, which are the square roots of the eigenvalues of \(AA^*\), forms a diagonal matrix \(\Sigma\).
- Get an unitary matrix \(V\) whose columns form an orthonormal basis consisting of eigenvectors of \(AA^*\).
- Compute \(U=\Sigma V\).
- Normalize and complete \(U\) to make \(U\) becomes an unitary matrix.
- \(A=U\Sigma V^T.\)
How to compute Jordan canonical form and corresponding basis?
- Find a basis \(\beta_1\) for \(N(A-\lambda I)\).
- Find a basis \(\beta_2\) for \(\{x|(A-\lambda I)x\in \beta_1\}\).
- ....
- \(\beta=\beta_1\cup\beta_2\cup...\) is what we need, the order is up to down, left to right.