Kaplan-Meier Estimator

Definition

Kaplan-Meier Estimator

Consider a Random Censoring case $Y_i=\min (T_i,C_i)$. Assume that $y_1\le y_2\le \cdots \le y_n$, and the distinct failure times are ${\tau }_1<{\tau }_2<\cdots <{\tau }_k$ where $k\le n$. Let $d_j:=\sum _{i=1}^{n}I(y_i={\tau }_j,{\delta }_i=1)$ be the number of deaths at ${\tau }_j$, and $n_j:=\sum _{i=1}^{n}I(y_i\ge {\tau }_j)$ be the number of alive at ${\tau }_j$ where the set$\{y_i|y_i\ge {\tau }_j\}$ is called the risk set at ${\tau }_j$. We only observe $(Y_i,{\delta }_i),i=1,\dots ,n$

The Kaplan-Meier estimator is derived from the expression $S(t)=\prod _{i=1}^k{P}_j,P_j=P(T>{\tau }_j|T>{\tau }_{j-1})$ where ${\tau }_0=0$

General Case

As an estimator of $P_j$, consider ${\hat{P}}_j=\hat{P}(T>{\tau }_j|T>{\tau }_{j-1})=\frac{\hat{P}(T>{\tau }_j)}{\hat{P}(T>{\tau }_{j-1})}=\frac{n_{j+1}/n_1}{n_j/n_1}=\frac{n_j-d_j}{n_j}=1-\frac{d_j}{n_j}$

The Kaplan-Meier estimator is defined with the estimated ${\hat{P}}_j$‘s $\hat{S}(t)=\prod _{j=1}^k{\hat{P}}_j=\prod _{\{j|{\tau }_j\le t\}}\left(1-\frac{d_j}{n_j}\right)$ The cumulative hazard function is estimated by Nelson-Aalen Estimator in the same logic $\hat{\Lambda }(t)=\sum _{\{j|{\tau }_j\le t\}}\frac{d_j}{n_j}$

No Ties Case

When there’s no tie in the observation, $y_1<y_2<\cdots <y_n$, then the failure times are equal to the observation ${\tau }_j=y_i$, death is equal to the censoring indicator $d_j={\delta }_j$, and $n_j=n-j+1$. Thus, the Kaplan-Meier estimator is defined as $\hat{S}(t)=\prod _{\{j|y_j\le t\}}(1-\frac{{\delta }_j}{n-j+1})=\prod _{\{j|{\tau }_j\le t\}}(1-\frac{1}{n-j+1}{)}^{{\delta }_j}$

Properties

Self-Consistency

An estimator $\hat{S}(t)$ is self-consistent if $\begin{array}{rl}\hat{S}(t)& =\frac{1}{n}\sum _{i=1}^n\left[1I(Y_i>t)+0I(Y_i\le t,{\delta }_i=1)+\frac{\hat{S}(t)}{\hat{S}(Y_i)}I(Y_i<t,{\delta }_i=0)\right]\\ & =\frac{1}{n}\left[N_y(t)+\sum _{i:y_i\le t}(1-{\delta }_i)\frac{\hat{S}(t)}{\hat{S}(Y_i)}\right]\end{array}$ where $N_y(t)=\sum _{i=1}^{n}I(Y_i>t)=\#(Y_i>t)$

The Kaplan-Meier estimator is the unique self-consistent estimator for $t<Y_{(n)}$ where $Y_{(n)}$ is the largest observation.

Generalized MLE

The Kaplan-Meier estimator gives the Generalized Maximum Likelihood Estimation of the Survival Function $S$.

Strong Consistency

The Kaplan-Meier estimator uniformly Almost Surely converges to $S(t)$ $\hat{S}(t)\stackrel{a.s.}{\to }S(t),\forall t\in {\mathbb{R}}^{+}$

Proof

Consider a function $S^{\ast }(t)=P(Y>t)$ and decompose it to the sum of the subsurvival functions $S_u^{\ast }(t)$ and $S_c^{\ast }(t)$. $S^{\ast }(t)=S_u^{\ast }(t)+S_c^{\ast }(t)$ where $S_u^{\ast }(t)=P(Y>t,\delta =1)$ is the uncensored case and $S_c^{\ast }(t)=P(Y>t,\delta =0)$ is the censored case.

Then, the survival function $S(t)=P(T>t)$ can be expressed as a function of the subsurvival functions. $S(t)=\Psi (S_u^{\ast },S_c^{\ast },t)$

Define the empirical subsurvival functions ${\hat{S}}_u^{\ast }(t)=\frac{1}{n}\sum _{i=1}^{n}I(Y_i>t,{\delta }_i=1)$ and ${\hat{S}}_c^{\ast }(t)=\frac{1}{n}\sum _{i=1}^{n}I(Y_i>t,{\delta }_i=0)$. The Kaplan-Meier estimator also can be expressed as a function of the empirical subsurvival functions. $\hat{S}(t)=\Psi ({\hat{S}}_u^{\ast },{\hat{S}}_c^{\ast },t)$

By Glivenko-Cantelli theorem, ${\hat{S}}_u^{\ast }(t)\stackrel{a.s.}{\to }{S}_u^{\ast }(t)$ and ${\hat{S}}_c^{\ast }(t)\stackrel{a.s.}{\to }{S}_c^{\ast }(t)$ for all $t\in {\mathbb{R}}^{+}$. Since $\Psi$ is a continuous function of $S_u^{\ast }(t)$ and $S_c^{\ast }(t)$, $\hat{S}(t)=\Psi ({\hat{S}}_u^{\ast },{\hat{S}}_c^{\ast },t)\stackrel{a.s.}{\to }\Psi (S_u^{\ast },S_c^{\ast },t)=S(t),\forall t\in {\mathbb{R}}^{+}$

Asymptotic Normality

Kaplan-Meier estimator has asymptotic normality. $\sqrt{n}(\hat{S}(t)-S(t))\stackrel{D}{\to }N\left(0,S(t{)}^2{\int }_0^t\frac{d{F}_u(X)}{(1-H(x){)}^2}\right)$ where $F_u(t)=P(T\le t,\delta =1)={\int }_0^t(1-G(x))dF(x)$, $C\sim G$, and $Y\sim H$.

The variance of the estimator is estimated by Greenwood’s formula ${\sigma }_S^2(t)\approx \hat{S}(t{)}^2\sum _{\{j|{\tau }_j\le t\}}\frac{d_j}{n_j(n_j-d_j)}$ For the no ties case, the formula is ${\sigma }_S^2(t)\approx \hat{S}(t{)}^2\sum _{\{j|{\tau }_j\le t\}}\frac{{\delta }_j}{(n-j)(n-j+1)}$

Examples

$n=8$ case where $\{\begin{array}{ll}\times & {\delta }_i=1\\ \circ & {\delta }_i=0\end{array}$

Facts

Kaplan-Meier estimator has Self-Consistency and Asymptotic Normality, and it is generalized MLE

If no censoring, Kaplan-Meier estimator is just the Empirical Survival Function.