Quadratic Form

Definition

$q(\mathbf{x})={\mathbf{x}}^{⊺}\mathbf{Ax}$ where $A$ is a Symmetric Matrix

A mapping $Q:V\to K$ where $V$ is a Module on Commutative Ring $K$ that has the following properties. $\forall k,l\in K,\forall u,v,w\in V$

• $Q(kv)=k^{2}Q(v)$
• $Q(u+v+w)=Q(u+v)+Q(v+w)+Q(u+w)-Q(u)-Q(v)-Q(w)$
• $Q(ku+lv)=k^{2}Q(u)+l^{2}Q(v)+klQ(u+v)-klQ(u)-klQ(v)$

Matrix Expressions

$a_1{x}_1^2+a_2{x}_2^2+2a_3{x}_1{x}_2\iff \left[\begin{array}{cc}{x}_1& x_2\end{array}\right]\left[\begin{array}{cc}{a}_1& a_3\\ a_3& a_2\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]$

$a_1{x}_1^2+a_2{x}_2^2+a_3{x}_3^2+2a_4{x}_1{x}_2+2a_5{x}_1{x}_3+2a_6{x}_2{x}_3\iff \left[\begin{array}{ccc}{x}_1& x_2& x_3\end{array}\right]\left[\begin{array}{ccc}{a}_1& a_4& a_5\\ a_4& a_2& a_6\\ a_5& a_6& a_3\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right]$

Facts

Let $\mathbf{x}$ be a Random Vector and $A$ be a symmetric matrix of constants. If $E(\mathbf{x})=\mathbf{\mu }$ and $\mathrm{Var}(\mathbf{x})=\Sigma$, then the expectation of the quadratic form $Q:={\mathbf{x}}^{⊺}A\mathbf{x}$ is $E(Q)={\mathbf{\mu }}^{⊺}A\mathbf{\mu }+\mathrm{tr}(A\Sigma )$

Let $\mathbf{x}=(X_1,\dots ,X_n{)}^{⊺}$ be a Random Vector and $A$ be a symmetric matrix of constants. If $E(\mathbf{x})=\mathbf{\theta }=({\theta }_1,\dots ,{\theta }_n{)}^{⊺}$ and $\mathrm{Var}(X_i)={\mu }_2$, $E[(X_i-{\theta }_i{)}^3]={\mu }_3$, and $E[(X_i-{\theta }_i{)}^4]={\mu }_4$, where $i=1,2,\dots ,n$, then the variance of the quadratic form ${\mathbf{x}}^{⊺}A\mathbf{x}$ is $\mathrm{Var}({\mathbf{x}}^{⊺}A\mathbf{x})=({\mu }_4-3{\mu }_2^2){\mathbf{a}}^{⊺}\mathbf{a}+2{\mu }_2^2\mathrm{tr}(A^2)+4{\mu }_2{\mathbf{\theta }}^{⊺}{A}^2\mathbf{\theta }+4{\mu }_3{\mathbf{\theta }}^{⊺}A\mathbf{a}$ where $\mathbf{a}$ is the column vector of diagonal elements of $A$.

If $X_i\sim N({\theta }_i,{\mu }_2),i=1,\dots ,n$ and $X_i$‘s are independent, then $\mathrm{Var}({\mathbf{x}}^{⊺}A\mathbf{x})=2{\mu }_2^2\mathrm{tr}(A^2)+4{\mu }_2{\mathbf{\theta }}^{⊺}{A}^2\mathbf{\theta }$ If $X_i\sim N(0,1),i=1,\dots ,n$ and $X_i$‘s are independent, then $\mathrm{Var}({\mathbf{x}}^{⊺}A\mathbf{x})=2\mathrm{tr}(A^2)$

Let ${\mathbf{X}}^{⊺}\sim N(0,{\sigma }^2{I}_n)$, $Q:={\mathbf{X}}^{⊺}A\mathbf{X}/{\sigma }^2$, where $A$ is a Symmetric Matrix and $\mathrm{rank}(A)=r\le n$, then the MGF of $Q$ is $M(t)={\prod }_{i=1}^r(1-2t{\lambda }_i{)}^{-1/2}=|I-2tA{|}^{-1/2}$ where ${\lambda }_i$‘s are non-zero eigenvalue of $A$

Let $\mathbf{X}\sim N_n(\mathbf{\mu },\Sigma )$, where $\Sigma$ is Positive-Definite Matrix, then $Q=(\mathbf{X}-\mathbf{\mu }{)}^{⊺}{\Sigma }^{-1}(\mathbf{X}-\mathbf{\mu })\sim {\chi }^2(n)$

Let $\mathbf{X}\sim N(0,{\sigma }^2{I}_n)$, $Q={\mathbf{X}}^{⊺}A\mathbf{X}/{\sigma }^2$, where $A$ is a Symmetric Matrix and $\mathrm{rank}(A)=r\le n$, then $Q\sim {\chi }^2(r)\iff A=A^k$ where $k\in \mathbb{N}$

Let $\mathbf{X}\sim N(0,{\sigma }^2{I}_n)$, $Q_1={\mathbf{X}}^{⊺}A\mathbf{X}/{\sigma }^2,Q_2={\mathbf{X}}^{⊺}B\mathbf{X}/{\sigma }^2$, where $A,B$ are symmetric matrices, then $Q_1,Q_2$ are independent if and only if $AB=O$

Let $Q=Q_1+Q_2+\cdots +Q_{k-1}+Q_k$, where $Q,Q_1,Q_2,\dots ,Q_k$ are quadratic forms in Random Sample from $N(0,{\sigma }^2)$ If $Q/{\sigma }^2\sim {\chi }^2(r),Q_1/{\sigma }^2\sim {\chi }^2(r_1),Q_2/{\sigma }^2\sim {\chi }^2(r_2),\dots ,Q_{k-1}/{\sigma }^2\sim {\chi }^2(r_{k-1})$ and $Q_k$ is non-negative, then

• $Q_1,Q_2,\dots ,Q_k$ are independent
• $Q_k/{\sigma }^2\sim {\chi }^2(r-r_1-r_2-\cdots -r_{k-1})$

Let $\mathbf{X}=(X_1,X_2,\dots ,X_n)\sim N(0,{\sigma }^2{I}_n)$, $\sum _{i=1}^n{X}_i^2=Q_1+Q_2+\cdots +Q_k$, where $Q_j={\mathbf{X}}^{⊺}{A}_j\mathbf{X}$, where $\mathrm{rank}(A_j)=r_j\le n$, then $\forall j=\{1,2,\dots ,k\},\frac{Q_j}{{\sigma }^2}\sim {\chi }^2(r_j)\iff {\sum }_{j=1}^k{r}_j=n$

${\mathbf{x}}^{⊺}A\mathbf{x}-2{\mathbf{x}}^{⊺}\mathbf{b}=(\mathbf{x}-A^{-1}\mathbf{b}{)}^{⊺}A(\mathbf{x}-A^{-1}\mathbf{b})-{\mathbf{b}}^{⊺}{A}^{-1}\mathbf{b}$