Multiple Comparisons

Definition

Assume that $H_0:{\mu }_1={\mu }_2=\cdots ={\mu }_K$ is rejected, and we interested in the confidence interval for some linear combination of parameters

Let $X_{1j},X_{2j},\dots ,X_{aj}$‘s be random samples from $N({\mu }_j,{\sigma }^2),j=1,2,\dots ,b$ We want to compute the confidence interval of $\sum _{j=1}^b{k}_j{\mu }_j$, where $k_j\in \mathbb{R}$

Type 1 Error

Consider $m$ multiple testing problem $H_{0i}:{\mu }_{1i}={\mu }_{2i},i=1,2,\dots ,m$, and let $E_i$ be an event of rejecting $H_{0i}$ even though $H_{0i}$ is true (type 1 error). If a test satisfies $P(E_i)\le \alpha$ for level $\alpha$, then the test is called to be controlled at $\alpha$ Under the independence of each test, the probability of type 1 error in $m$ multiple testing is $P(\bigcup _{i=1}^m{E}_i)=1-P(\bigcap _{i=1}^m{E}_i^{\complement })=1-{\prod }_{i=1}^{m}P(E_i^{\complement })\le 1-(1-\alpha {)}^m$ Thus, the probability of type 1 error approaches to $1$ as $m$ increase.

Fixed Constants Case

The $k_j$‘s are fixed constants. The $100(1-\alpha )\%$ confidence interval for each $\sum _{j=1}^b{k}_j{\mu }_j$ is $\sum _{j=1}^b{k}_j{\overline{X}}_{.j}\pm t_{\alpha /2}(b(a-1))\sqrt{\left({\sum }_{j=1}^b{k}_j^2\right)\frac{V}{a}}$ where $V=\frac{1}{b(a-1)}\sum _{i=1}^a\sum _{j=1}^b(X_{ij}-{\overline{X}}_{.j}{)}^2$

Scheffe’s Simultaneous Confidence Interval

The $k_j$‘s are allowed to have any real numbers. The simultaneous $100(1-\alpha )\%$ confidence interval for all $\sum _{j=1}^b{k}_j{\mu }_j$ is $\sum _{j=1}^b{k}_j{\overline{X}}_{.j}\pm \sqrt{\frac{b}{a}V{F}_{\alpha }\left(\sum _{j=1}^b{k}_j^2\right)}$ where $V=\frac{1}{b(a-1)}\sum _{i=1}^a\sum _{j=1}^b(X_{ij}-{\overline{X}}_{.j}{)}^2$