Two-way ANOVA

Definition

Two-way ANOVA Without Replications

	$1$	$2$	$\dots$	$b$
$1$	$X_{11}$	$X_{12}$	$\dots$	$X_{1b}$
$2$	$X_{21}$	$X_{22}$	$\dots$	$X_{2b}$
$⋮$	$⋮$	$⋮$	$\ddots$	$⋮$
$a$	$X_{a1}$	$X_{a2}$	$\dots$	$X_{ab}$

Let $X_{ij}$‘s be Random Sample from $N({\mu }_{ij},{\sigma }^2)$. The ${\mu }_{ij}=\mu +{\alpha }_i+{\beta }_j$ can be decomposed as sum of means (global mean $+$ $i$-th level effect of factor $A$ $+$ $j$-th level effect of factor $B$). In this setup, we assume that $\sum _{i=1}^a{\alpha }_i=\sum _{j=1}^b{\beta }_j=0$

We want to test $H_0:{\beta }_1,{\beta }_2=\cdots ={\beta }_b$. $F=\frac{\sum _{i=1}^a\sum _{j=1}^b({\overline{X}}_{.j}-{\overline{X}}_{..}{)}^2/(b-1)}{\sum _{i=1}^a\sum _{j=1}^b(X_{ij}-{\overline{X}}_{i.}-{\overline{X}}_{.j}+{\overline{X}}_{..}{)}^2/(a-1)(b-1)}\sim F((b-1),(a-1)(b-1))$ where $\sum _{i=1}^a\sum _{j=1}^b({\overline{X}}_{.j}-{\overline{X}}_{..}{)}^2\sim {\chi }^2(b-1)$, and $\sum _{i=1}^a\sum _{j=1}^b(X_{ij}-{\overline{X}}_{i.}-{\overline{X}}_{.j}+{\overline{X}}_{..}{)}^2\sim {\chi }^2((a-1)(b-1))$

The Likelihood Ratio Test rejects $H_0$ if $F>F_{\alpha }((b-1),(a-1)(b-1))$

Under $H_1$, the $F$ follows Noncentral F-Distribution $F^{\prime }=\frac{a\sum _{j=1}^b({\overline{X}}_{.j}-{\overline{X}}_{..}{)}^2/(b-1)}{\sum _{i=1}^a\sum _{j=1}^b(X_{ij}-{\overline{X}}_{i.}-{\overline{X}}_{.j}+{\overline{X}}_{..}{)}^2/(a-1)(b-1)}\sim F\left((b-1),(a-1)(b-1),a{\sum }_{j=1}^b{\beta }_j^2/{\sigma }^2\right)$ where $\frac{1}{{\sigma }^2}a\sum _{j=1}^b({\overline{X}}_{.j}-{\overline{X}}_{..}{)}^2\sim {\chi }^2(b-1,a\sum _{j=1}^b{\beta }_j^2/{\sigma }^2)$

Therefore, the power of the test is $P(F^{\prime }>F_{\alpha })$

Two-way ANOVA With Equal Replications

	$1$	$2$	$\dots$	$b$
$1$	$X_{111},\dots ,X_{11n}$	$X_{121},\dots ,X_{12n}$	$\dots$	$X_{1b1},\dots ,X_{1bn}$
$2$	$X_{211},\dots ,X_{21n}$	$X_{221},\dots ,X_{22n}$	$\dots$	$X_{2b1},\dots ,X_{1bn}$
$⋮$	$⋮$	$⋮$	$\ddots$	$⋮$
$a$	$X_{a11},\dots ,X_{a1n}$	$X_{a21},\dots ,X_{a2n}$	$\dots$	$X_{ab1},\dots ,X_{abn}$

Let $X_{ijk}$‘s be Random Sample from $N({\mu }_{ij},{\sigma }^2)$. The ${\mu }_{ijk}=\mu +{\alpha }_i+{\beta }_j+{\gamma }_{ij}$ can be decomposed as sum of means (global mean + $i$-th level effect of factor $A$ + $j$-th level effect of factor $B$ + interaction effect of $i$-th level of factor $A$ and the $j$-th level of factor $B$ ). In this setup, we assume that $\sum _{i=1}^a{\alpha }_i=\sum _{j=1}^b{\beta }_j=\sum _{i=1}^a{\gamma }_{ij}=\sum _{j=1}^b{\gamma }_{ij}=0$

We want to test $H_0:{\gamma }_{ij}=0,\forall i,j$. $F=\frac{\sum _{i=1}^a\sum _{j=1}^b\sum _{k=1}^n({\overline{X}}_{ij.}-{\overline{X}}_{i..}-{\overline{X}}_{.j.}+{\overline{X}}_{...}{)}^2/(a-1)(b-1)}{\sum _{i=1}^a\sum _{j=1}^b\sum _{k=1}^n(X_{ijk}-{\overline{X}}_{ij.}{)}^2/(N-ab)}\sim F((a-1)(b-1),N-ab)$ where $N=\sum _{i=1}^a\sum _{j=1}^{b}n=abn$, $\sum _{i=1}^a\sum _{j=1}^b\sum _{k=1}^n({\overline{X}}_{ij.}-{\overline{X}}_{i..}-{\overline{X}}_{.j.}+{\overline{X}}_{...}{)}^2\sim {\chi }^2((a-1)(b-1))$, and $\sum _{i=1}^a\sum _{j=1}^b\sum _{k=1}^n(X_{ijk}-{\overline{X}}_{ij.}{)}^2\sim {\chi }^2((N-ab))$

The Likelihood Ratio Test rejects $H_0$ if $F>F_{\alpha }((a-1)(b-1),(N-ab))$

Under $H_1$, the $F$ follows Noncentral F-Distribution $F^{\prime }=\frac{n\sum _{i=1}^a\sum _{j=1}^b({\overline{X}}_{ij.}-{\overline{X}}_{i..}-{\overline{X}}_{.j.}+{\overline{X}}_{...}{)}^2/(a-1)(b-1)}{\sum _{i=1}^a\sum _{j=1}^b\sum _{k=1}^n(X_{ijk}-{\overline{X}}_{ij.}{)}^2/(N-ab)}\sim F\left((a-1)(b-1),(N-ab),n\sum _{i=1}^a\sum _{j=1}^b{\gamma }_{ij}^2/{\sigma }^2\right)$ where $n\sum _{i=1}^a\sum _{j=1}^b({\overline{X}}_{ij.}-{\overline{X}}_{i..}-{\overline{X}}_{.j.}+{\overline{X}}_{...}{)}^2\sim {\chi }^2((a-1)(b-1),n\sum _{i=1}^a\sum _{j=1}^b{\gamma }_{ij}^2/{\sigma }^2)$

Therefore, the power of the test is $P(F^{\prime }>F_{\alpha })$

If $H_0:{\gamma }_{ij}=0,\forall i,j$ is not rejected, then we continue to test $H_0:{\alpha }_1={\alpha }_2=\cdots ={\alpha }_a=0$ or $H_0:{\beta }_1={\beta }_2=\cdots ={\beta }_b=0$

Two-way ANOVA with a Regression Model

$Y={\beta }_0+\sum _{i=1}^{a-1}{\alpha }_i{D}_i+\sum _{j=1}^{b-1}{\beta }_j{E}_j+\sum _{i=1}^{a-1}\sum _{j=1}^{b-1}{\gamma }_{ij}{D}_i{E}_j+ϵ$ where $D_i$ and $E_j$ are dummy variables representing categories of the two factors, $a$ is the number of categories for the first factor, and $b$ is the number of categories for the second factor

In this setting, a corner-point constraint is used for both factors.

The null hypotheses can be tested by Deviance

Deviance Test for Two-way ANOVA

For a two-way ANOVA with factors $A$ and $B$, we have three null hypotheses to test:

• $H_{0A}:{\alpha }_1={\alpha }_2={\alpha }_{a-1}=0$ i.e. there is no treatment effect of factor $A$
• $H_{0B}:{\beta }_1={\beta }_2={\beta }_{b-1}=0$ i.e. there is no treatment effect of factor $B$
• $H_{0AB}:{\gamma }_{ij}=0,\forall i,j$ i.e. there is no interaction effect between factor $A$ and $B$ They can be tested with the Deviance.

If ${\sigma }^2$ is known, the test statistic for $H_{0A}$ is defined as $\Delta D_A=D_0-D_1=\frac{1}{{\sigma }^2}\left(\frac{1}{bn}\sum _{i=1}^a{y}_{i..}^2-\frac{1}{N}{Y}_{...}^2\right)\sim {\chi }^2(a-1)$ And reject the $H_{0A}$ if $\Delta D_A>{\chi }^2(a-1)$

If ${\sigma }^2$ is unknown, the test statistic for $H_{0A}$ is defined as $F_A=\frac{D_0-D_1}{a-1}/\frac{D_1}{N-ab}\sim F(a-1,N-ab)$ And reject the $H_{0A}$ if $F_A>F(a-1,N-ab)$

If ${\sigma }^2$ is known, the test statistic for $H_{0B}$ is defined as $\Delta D_B=D_0-D_1=\frac{1}{{\sigma }^2}\left(\frac{1}{an}\sum _{j=1}^b{y}_{.j.}^2-\frac{1}{N}{Y}_{...}^2\right)\sim {\chi }^2(b-1)$ And reject the $H_{0B}$ if $\Delta D_B>{\chi }^2(b-1)$

If ${\sigma }^2$ is unknown, the test statistic for $H_{0B}$ is defined as $F_B=\frac{D_0-D_1}{b-1}/\frac{D_1}{N-ab}\sim F(b-1,N-ab)$ And reject the $H_{0B}$ if $F_B>F(b-1,N-ab)$

If ${\sigma }^2$ is known, the test statistic for $H_{0AB}$ is defined as $\Delta D_{AB}=D_0-D_1=\frac{1}{{\sigma }^2}\left(\frac{1}{n}\sum _{i=1}^a\sum _{j=1}^b{y}_{ij.}^2-\frac{1}{bn}\sum _{i=1}^a{Y}_{i..}^2-\frac{1}{an}\sum _{j=1}^b{Y}_{.j.}^2+\frac{1}{N}{Y}_{...}^2\right)\sim {\chi }^2((a-1)(b-1))$ And reject the $H_{0AB}$ if $\Delta D_{AB}>{\chi }^2((a-1)(b-1))$

If ${\sigma }^2$ is unknown, the test statistic for $H_{0AB}$ is defined as $F_{AB}=\frac{D_0-D_1}{(a-1)(b-1)}/\frac{D_1}{N-ab}\sim F((a-1)(b-1),N-ab)$ And reject the $H_{0AB}$ if $F_{AB}>F((a-1)(b-1),N-ab)$