矩阵求导
在看多元线性回归的闭式解的时候遇到矩阵求导问题，总体来讲矩阵求导与函数求导有极大的相似性，查看wiki后记录下矩阵求导的一些性质，方面日后查看。
用到比较多的公式如下(分母布局)：
\[\frac{\partial \boldsymbol A\boldsymbol x}{\partial \boldsymbol x}=A^T
\]
\[\frac{\partial \boldsymbol x^T\boldsymbol A}{\partial \boldsymbol x}=A
\]
\[\frac{\partial \boldsymbol u \cdot \boldsymbol v}{\partial \boldsymbol x}=\frac{\partial \boldsymbol u^T\boldsymbol v}{\partial \boldsymbol x}=\frac{\partial \boldsymbol u}{\partial \boldsymbol x}\boldsymbol v+\frac{\partial \boldsymbol v}{\partial \boldsymbol x}\boldsymbol u
\]
证明几个公式：
1.\(\cfrac{\partial \boldsymbol x^T\boldsymbol A \boldsymbol x}{\partial \boldsymbol x}=(\boldsymbol A+\boldsymbol A^T)\boldsymbol x\)
\[\begin{align}
\frac{\partial \boldsymbol x^T\boldsymbol A \boldsymbol x}{\partial \boldsymbol x}=&\frac{\partial (\boldsymbol A^T \boldsymbol x)^T \boldsymbol x}{\partial \boldsymbol x}\
\\=&\frac{\partial \boldsymbol A^T \boldsymbol x}{\partial \boldsymbol x}\boldsymbol x+\frac{\partial \boldsymbol x}{\partial \boldsymbol x}\boldsymbol A^T \boldsymbol x\\
=&\boldsymbol A\boldsymbol x+\boldsymbol A^T\boldsymbol x\\
=&(\boldsymbol A+\boldsymbol A^T)\boldsymbol x
\end{align}
\]
2.$ \cfrac{\partial E_{\hat{\boldsymbol w}}}{\partial \hat{\boldsymbol w}}=2\mathbf{X}^T(\mathbf{X}\hat{\boldsymbol w}-\boldsymbol{y})$
\[\begin{align}
\cfrac{\partial E_{\hat{\boldsymbol w}}}{\partial \hat{\boldsymbol w}}=& \cfrac{\partial \boldsymbol{y}^T\boldsymbol{y}}{\partial \hat{\boldsymbol w}}-\cfrac{\partial \boldsymbol{y}^T\mathbf{X}\hat{\boldsymbol w}}{\partial \hat{\boldsymbol w}}-\cfrac{\partial \hat{\boldsymbol w}^T\mathbf{X}^T\boldsymbol{y}}{\partial \hat{\boldsymbol w}}+\cfrac{\partial \hat{\boldsymbol w}^T\mathbf{X}^T\mathbf{X}\hat{\boldsymbol w}}{\partial \hat{\boldsymbol w}}\\
=&0-(\boldsymbol{y}^T\mathbf{X})^T-\mathbf{X}^T\boldsymbol{y}+(\mathbf{X}^T\mathbf{X}+\mathbf{X}^T\mathbf{X})\hat{\boldsymbol w}\\
=&2\mathbf{X}^T(\mathbf{X}\hat{\boldsymbol w}-\boldsymbol{y})
\end{align}
\]
参考文献：
https://en.wikipedia.org/wiki/Matrix_calculus#Scalar-by-vector_identities