Symmetric Rank-One Method

The Quasi-Newton Method method with a symmetric condition

Algorithm

Define $\Delta {\mathbf{f}}_n:={\mathbf{f}}_n-{\mathbf{f}}_{n-1}$ $\Delta {\mathbf{x}}_n:={\mathbf{x}}_n-{\mathbf{x}}_{n-1}$ $J_n\simeq {\nabla }^{2}f(x)$ $H_n=J_n^{-1}\simeq {\nabla }^{2}f(x{)}^{-1}$

Then, $\Delta {\mathbf{f}}_n={\mathbf{J}}_{\mathbf{n}}\Delta {\mathbf{x}}_n$ $\Delta {\mathbf{x}}_n={\mathbf{H}}_{\mathbf{n}}\Delta {\mathbf{f}}_n$

To assure the symmetry of the approximation of hessian matrix, let the perturbation matrix be $\mathbf{u}{\mathbf{u}}^{⊺}$ $H_n=H_{n-1}+\mathbf{u}{\mathbf{u}}^{⊺}$

Then, $\Delta {\mathbf{x}}_n=H_n\Delta {\mathbf{f}}_n\Rightarrow \Delta {\mathbf{x}}_n=H_{n-1}\Delta {\mathbf{f}}_n+\mathbf{u}{\mathbf{u}}^{⊺}\Delta {\mathbf{f}}_n\Rightarrow \mathbf{u}{\mathbf{u}}^{⊺}\Delta {\mathbf{f}}_n=\Delta {\mathbf{x}}_n-H_{n-1}\Delta {\mathbf{f}}_n$

Now we can express $\mathbf{u}=\frac{(\Delta {\mathbf{x}}_n-H_{n-1}\Delta {\mathbf{f}}_n)}{{\mathbf{u}}^{⊺}\Delta {\mathbf{f}}_n}$ and $\mathbf{u}{\mathbf{u}}^{⊺}=\frac{(\Delta {\mathbf{x}}_n-H_{n-1}\Delta {\mathbf{f}}_n)(\Delta {\mathbf{x}}_n-H_{n-1}\Delta {\mathbf{f}}_n{)}^{⊺}}{({\mathbf{u}}^{⊺}\Delta {\mathbf{f}}_n{)}^2}$

Since $\mathbf{u}{\mathbf{u}}^{⊺}$ satisfying $\mathbf{u}{\mathbf{u}}^{⊺}\Delta {\mathbf{f}}_n=\Delta {\mathbf{x}}_n-H_{n-1}\Delta {\mathbf{f}}_n$ is unique(because $\mathbf{u}({\mathbf{u}}^{⊺}\Delta {\mathbf{f}}_n)=⟨{\mathbf{u}}^{⊺},\Delta {\mathbf{f}}_n⟩\mathbf{u}$ is the scaling of $\mathbf{u}$), we can get $({\mathbf{u}}^{⊺}\Delta {\mathbf{f}}_n{)}^2$ by multiplying $\Delta {\mathbf{f}}_n^{⊺}$ to the both side $({\mathbf{u}}^{⊺}\Delta {\mathbf{f}}_n{)}^2=\Delta {\mathbf{f}}_n^{⊺}\mathbf{u}{\mathbf{u}}^{⊺}\Delta {\mathbf{f}}_n=\Delta {\mathbf{f}}_n^{⊺}(\Delta {\mathbf{x}}_n-H_{n-1}\Delta {\mathbf{f}}_n)$

by substituting $({\mathbf{u}}^{⊺}\Delta {\mathbf{f}}_n{)}^2$ to original expression, $\mathbf{u}{\mathbf{u}}^{⊺}=\frac{(\Delta {\mathbf{x}}_n-H_{n-1}\Delta {\mathbf{f}}_n)(\Delta {\mathbf{x}}_n-H_{n-1}\Delta {\mathbf{f}}_n{)}^{⊺}}{\Delta {\mathbf{f}}_n^{⊺}(\Delta {\mathbf{x}}_n-H_{n-1}\Delta {\mathbf{f}}_n)}$

Therefore, $H_n=H_{n-1}+\frac{(\Delta {\mathbf{x}}_n-H_{n-1}\Delta {\mathbf{f}}_n)(\Delta {\mathbf{x}}_n-H_{n-1}\Delta {\mathbf{f}}_n{)}^{⊺}}{(\Delta {\mathbf{x}}_n-H_{n-1}\Delta {\mathbf{f}}_n{)}^{⊺}\Delta {\mathbf{f}}_n}$ Since the denominator is a scalar and ${\mathbf{x}}_{n+1}={\mathbf{x}}_n-H_n\mathbf{f}({\mathbf{x}}_n)$