文章目录
一、基本概念1.1 协方差矩阵 及推导1.2 Hessian矩阵1.3 Hessian矩阵 示例1.3 正定矩阵定义及性质1.4 正定矩阵 示例一、基本概念
1.1 协方差矩阵 及推导
在统计学中用标准差描述样本数据的 “散布度” 公式中之所以除以 n-1 而不是 n,
是因为这样使我们以较少的样本集更好的逼近总体标准差。即统计学上所谓的 “无偏估计”。
关于协方差与散度：/wsp_1138886114/article/details/80967843
方差：var(X)=∑i=1n(Xi−Xˉ)(Xi−Xˉ)n−1var(X) = \frac{\sum_{i=1}^n(X_i-\bar{X})(X_i-\bar{X})}{n-1}var(X)=n−1∑i=1n(Xi−Xˉ)(Xi−Xˉ)
各个维度偏离其均值的程度，协方差：cov(X,Y)=∑i=1n(Xi−Xˉ)(Yi−Yˉ)n−1\text{cov}(X,Y) = \frac{\sum_{i=1}^n(X_i-\bar{X})(Y_i-\bar{Y})}{n-1}cov(X,Y)=n−1∑i=1n(Xi−Xˉ)(Yi−Yˉ)
协方差矩阵的计算：
cov(z)=(123434122314)jcov(z) = \begin{pmatrix} 1 & 2 &3 &4 \\ 3&4 &1 & 2\\ 2& 3& 1& 4 \end{pmatrix}jcov(z)=⎝⎛132243311424⎠⎞j
1.2 Hessian矩阵
Hessian矩阵定义：
若一元函数f(x)f(x)f(x) 在x=x(0)x = x^{(0)}x=x(0) 点的某个领域内具有任意阶导数，则 f(x)f(x)f(x) 在x(0)x^{(0)}x(0) 点的泰勒展开式为：
f(x)=f(x(0))+f′(x(0))Δx+12f′′(x(0))(Δx2)+⋯(1)f(x) = f(x^{(0)}) + f'(x^{(0)})\Delta x + \frac{1}{2} f''(x^{(0)})(\Delta x^2)+\cdots \tag{1}f(x)=f(x(0))+f′(x(0))Δx+21f′′(x(0))(Δx2)+⋯(1)
其中：Δx=x−x(0),Δx2=(x−x(0))2\Delta x = x-x^{(0)},\Delta x^2 = (x-x^{(0)})^2Δx=x−x(0),Δx2=(x−x(0))2
二元函数f(x1,x2)f(x_1,x_2)f(x1,x2)在X(0)(x1(0),x2(0))X^{(0)}(x^{(0)}_1,x^{(0)}_2)X(0)(x1(0),x2(0))点处的泰勒展开式为：
12[∂2f∂2x12∣x(0)Δx12+2∂2f∂x1∂x2∣x(0)Δx1Δx2+∂2f∂2x22∣x(0)Δx22]+⋯(2)\frac{1}{2}\left [ \frac{\partial^2f}{\partial^2x_1^2}|_{x^{(0)}} \Delta x_1^2 + 2\frac{\partial^2f}{\partial x_1\partial x_2}|_{x^{(0)}}\Delta x_1\Delta x_2+\frac{\partial^2f}{\partial^2x_2^2}|_{x^{(0)}} \Delta x_2^2\right ]+\cdots \tag{2}21[∂2x12∂2f∣x(0)Δx12+2∂x1∂x2∂2f∣x(0)Δx1Δx2+∂2x22∂2f∣x(0)Δx22]+⋯(2)
其中：Δx1=x1−x1(0),Δx2=x2−x2(0)\Delta x_1 = x_1-x^{(0)}_1,\Delta x_2 = x_2-x_2^{(0)}Δx1=x1−x1(0),Δx2=x2−x2(0)
将上述(2)展开式写成矩阵形式，则有：
f(X)=f(X(0))+(∂f∂x1,∂f∂x2)x(0)(Δx1Δx2)+12(Δx1,Δx2){∂2f∂x12∂2f∂x1∂x2∂2f∂x2∂x1∂2f∂x22}∣x(0)(Δx1Δx2)+⋯(3)f(X) = f(X^{(0)})+\left ( \frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2} \right )_{x^{(0)}}\begin{pmatrix} \Delta x_1\\ \Delta x_2 \end{pmatrix}+\frac{1}{2}(\Delta x_1,\Delta x_2)\begin{Bmatrix} \frac{\partial^2f}{\partial x_1^2} & \frac{\partial^2f}{\partial x_1 \partial x_2}\\ \frac{\partial^2f}{\partial x_2 \partial x_1}& \frac{\partial^2f}{\partial x_2^2} \end{Bmatrix}|_{x^{(0)}} \begin{pmatrix} \Delta x_1\\ \Delta x_2 \end{pmatrix} +\cdots \tag{3}f(X)=f(X(0))+(∂x1∂f,∂x2∂f)x(0)(Δx1Δx2)+21(Δx1,Δx2){∂x12∂2f∂x2∂x1∂2f∂x1∂x2∂2f∂x22∂2f}∣x(0)(Δx1Δx2)+⋯(3)
即为：
f(X)=f(X(0))+∇f(X(0))T+12ΔxTG(X(0))ΔX+⋯(4)f(X) = f(X^{(0)})+\nabla f(X^{(0)})^T + \frac{1}{2} \Delta x^T G(X^{(0)}) \Delta X +\cdots \tag{4}f(X)=f(X(0))+∇f(X(0))T+21ΔxTG(X(0))ΔX+⋯(4)
其中：
G(X(0))={∂2f∂x12∂2f∂x1∂x2∂2f∂x2∂x1∂2f∂x22}∣x(0),ΔX=(Δx1Δx2)G(X^{(0)}) = \begin{Bmatrix} \frac{\partial^2f}{\partial x_1^2} & \frac{\partial^2f}{\partial x_1 \partial x_2}\\ \frac{\partial^2f}{\partial x_2 \partial x_1}& \frac{\partial^2f}{\partial x_2^2} \end{Bmatrix}|_{x^{(0)}}, ~~\Delta X = \begin{pmatrix} \Delta x_1\\ \Delta x_2 \end{pmatrix}G(X(0))={∂x12∂2f∂x2∂x1∂2f∂x1∂x2∂2f∂x22∂2f}∣x(0),ΔX=(Δx1Δx2)
G(X(0))G(X^{(0)})G(X(0))是 f(x1,x2)f(x_1,x_2)f(x1,x2) 在 X(0)X^{(0)}X(0) 点处的Hessian矩阵。它是由函数 f(x1,x2)f(x_1,x_2)f(x1,x2) 在 X(0)X^{(0)}X(0)点处的二阶偏导数所组成的方阵。我们一般将其表示为:
H(f)=[∂2f∂x12∂2f∂x1∂x2⋯∂2f∂x1∂xn∂2f∂x2∂x1∂2f∂x22⋯∂2f∂x2∂xn⋮⋮⋱⋮∂2f∂xn∂x1∂2f∂xn∂x2⋯∂2f∂xn2]H(f) = \begin{bmatrix} \frac{\partial^2f}{\partial x_1^2} & \frac{\partial^2f}{\partial x_1 \partial x_2} & \cdots & \frac{\partial^2f}{\partial x_1 \partial x_n} \\ \frac{\partial^2f}{\partial x_2 \partial x_1} & \frac{\partial^2f}{\partial x_2^2} & \cdots & \frac{\partial^2f}{\partial x_2 \partial x_n}\\ \vdots & \vdots & \ddots &\vdots \\ \frac{\partial^2f}{\partial x_n \partial x_1} & \frac{\partial^2f}{\partial x_n \partial x_2} & \cdots & \frac{\partial^2f}{\partial x_n^2} \end{bmatrix}H(f)=⎣⎢⎢⎢⎢⎢⎡∂x12∂2f∂x2∂x1∂2f⋮∂xn∂x1∂2f∂x1∂x2∂2f∂x22∂2f⋮∂xn∂x2∂2f⋯⋯⋱⋯∂x1∂xn∂2f∂x2∂xn∂2f⋮∂xn2∂2f⎦⎥⎥⎥⎥⎥⎤
简写成：QHessian=[IxxIxyIyxIyy]\mathbf{Q_{Hessian}} = \begin{bmatrix} I_{xx} & I_{xy}\\ I_{yx} & I_{yy} \end{bmatrix}QHessian=[IxxIyxIxyIyy]
1.3 Hessian矩阵 示例
1.3 正定矩阵定义及性质
在线性代数中，正定矩阵（positive definite matrix）简称正定阵。
定义：A是n阶方阵，如果对于任何非零向量x都有xTAx>0x^TAx>0xTAx>0就称A正定矩阵。
