【高斯分布】01-极大似然估计

高斯分布

输入数据:$X=(x_1,x_2,...,x_n)^T=\begin{pmatrix} x_1^T\\ x_2^T\\ ... \\ x_n^T\\ \end{pmatrix}$X=(x1​,x2​,...,xn​)T=⎝⎜⎜⎛​x1T​x2T​...xnT​​⎠⎟⎟⎞​
$x_i\in R^p,x_i\ \sim^{iid}\ N(μ,Σ),θ=(μ,Σ)$xi​∈Rp,xi​ ∼iid N(μ,Σ),θ=(μ,Σ)

iid指独立同分布
回顾：
1.数学期望是对随机变量中心位置的一种度量，是每次实验中可能的结果乘以其结果的总和
$E(x)=xf(x)$E(x)=xf(x)
方差就是这种风险的度量，即随机变量的变异性
$E(x)=(x-μ)f(x)$E(x)=(x−μ)f(x)
2.独立：一个事件的发生不依赖于另外一个事件，两个事件同时发生的概率为P(AB) = P(A)·P(B)
独立同分布：各事件相互独立，但满足同一个概率分布

$MLE:θ_{MLE}=argmax_{(θ)}P(X|θ)$MLE:θMLE​=argmax(θ)​P(X∣θ)
$令p=1,θ=(μ,\sigma^2)$令p=1,θ=(μ,σ2)
$\begin{array}{lcr} P(x) &=& \frac{1}{\sqrt{2\pi}\sigma}exp(-\frac{(x-μ)^2}{2\sigma^2})\\ P(x) &=& \frac{1}{\sqrt{2\pi}\frac{p}{2}|\sum|^{\frac{1}{2}}}exp(-\frac{1}{2}(x-μ)^T\sum^{-1}(x-μ)))\\\\ logP(x|θ) &=& log\prod_{i=1}^{N}p(x_i|θ)\\ &=& \sum_{i=1}^{N}logp(x_i|θ)\\ &=& \sum_{i=1}^{N}log\frac{1}{\sqrt{2\pi}\sigma}exp(-\frac{(x-μ)^2}{2\sigma^2})\\ &=& \sum_{i=1}^{N}[log\frac{1}{\sqrt{2\pi}}+log\frac{1}{\sigma}-\frac{(x_i-μ)^2}{2\sigma^2}]\\\\ μ_{MLE} &=& argmax_{μ} logP(x|θ)\\ &=& argmax\sum_{i=1}^{N}-\frac{x_i-μ}{2\sigma^2}\\ &=& argmax_{u}min\sum_{i=1}{N}-\frac{x_i-μ}{2\sigma^2}\\\\ \frac{\partial}{\partialμ}\sum(x_i-μ)^2 &=& \sum_{i=1}^{N}2(x_i-μ)(-1)=0\\ \sum_{i=1}^{N}(x_i-μ) &=& 0 \end{array}$P(x)P(x)logP(x∣θ)μMLE​∂μ∂​∑(xi​−μ)2∑i=1N​(xi​−μ)​===========​2π​σ1​exp(−2σ2(x−μ)2​)2π​2p​∣∑∣21​1​exp(−21​(x−μ)T∑−1(x−μ)))log∏i=1N​p(xi​∣θ)∑i=1N​logp(xi​∣θ)∑i=1N​log2π​σ1​exp(−2σ2(x−μ)2​)∑i=1N​[log2π​1​+logσ1​−2σ2(xi​−μ)2​]argmaxμ​logP(x∣θ)argmax∑i=1N​−2σ2xi​−μ​argmaxu​min∑i=1​N−2σ2xi​−μ​∑i=1N​2(xi​−μ)(−1)=00​

$\begin{array}{lcr} μ_{MLE} &=& \frac{1}{N}\sum_{i=1}^{N}x_i(无偏估计)\\ E[μ_{MLE}] &=& \frac{1}{N}\sum_{i=1}^{N}E[x_i]\\ &=& \frac{1}{N}\sum_{i=1}^{N}μ\\ &=& \frac{1}{N}N\ μ\\ &=& μ \end{array}$μMLE​E[μMLE​]​=====​N1​∑i=1N​xi​(无偏估计)N1​∑i=1N​E[xi​]N1​∑i=1N​μN1​N μμ​

$\begin{array}{lcr} \sigma_{MLE}^{2} &=& argmax_{\sigma}\ P(x|\sigma)\\ &=& argmax\sum(-log\ \sigma\ - \frac{1}{2\sigma^2})\\\\ \frac{\partial\wp}{\partial\sigma} &=& \sum_{i=1}^{N}[- \frac{1}{\sigma}+\frac{1}{2}(x_i-μ)^2(+2)\sigma^{-3}] &=& 0 \\ \sum_{i=1}^{N}[-\frac{1}{\sigma}+(x_i-μ)^2\sigma^{-3}] &=& 0\\ -\sum_{i=1}^{N}\sigma^2 \ + \ \sum_{i=1}^{N}(x_i-μ)^2 &=& 0\\ \sum_{i=1}^{N}\sigma^2 &=& \sum_{i=1}^{N}(x_i-μ)^2\\ \sigma_{MLE}^{2} &=& \frac{1}{N}\sum_{i=1}{N}(x_i-μ)^2(有偏估计)\\\\ E[\sigma_{MLE}^{2}] &=& \frac{N-1}{N}\sigma^2 \\ \hat\sigma &=& \frac{1}{N-1}\sum_{i=1}^{N}(x_i-μ_{MLE})(无偏估计) \end{array}$σMLE2​∂σ∂℘​∑i=1N​[−σ1​+(xi​−μ)2σ−3]−∑i=1N​σ2 + ∑i=1N​(xi​−μ)2∑i=1N​σ2σMLE2​E[σMLE2​]σ^​=========​argmaxσ​ P(x∣σ)argmax∑(−log σ −2σ21​)∑i=1N​[−σ1​+21​(xi​−μ)2(+2)σ−3]00∑i=1N​(xi​−μ)2N1​∑i=1​N(xi​−μ)2(有偏估计)NN−1​σ2N−11​∑i=1N​(xi​−μMLE​)(无偏估计)​=0

参考资料

1.板书 高斯分布1-极大似然估计