EM Algorithm
目录Jensen's inequality
convex function: \(f''(x) \ge 0\) or \(H \ge 0\) (Hessian matrix when x is a vector)
\[E[f(x)] \ge f(EX) \]EM Algorithm
EM can be proved that it make the likelihood function increase monotonically.
maximize the lower-bound on the likelihood \(\ell\), \(\log\) is a concave function. the process is
\[\theta_{t-1} \rightarrow^{assign} Q_{z,t-1} \rightarrow^{\arg\max} \theta_{t} \rightarrow Q_{z,t} \]define:
\[J(Q,\theta)=\sum_{i} \sum_{z^{(i)}}Q_i(z^{(i)})\log \frac{p(x^{(i)},z^{(i)};\theta)}{Q_i(z^{(i)})} \]the EM algorithm can also be defined as coordinate ascent on \(J\).
标签：EM,function,Algorithm,ge,算法,theta,rightarrow From： https://www.cnblogs.com/BUAA-Stargazer/p/16613979.html