ANOVA

Confidence Interval

${\mu }_X-{\mu }_Y$ with Normal Distribution

$$P\left(-z_{\frac{\alpha }{2}}\le \frac{(\bar{X}-\bar{Y})-({\mu }_X-{\mu }_Y)}{\sqrt{\frac{{\sigma }_X^2}{n}+\frac{{\sigma }_Y^2}{m}}}\le z_{\frac{\alpha }{2}}\right)=1-\alpha$$

Hypothesis Testing

Two-sample z-test with known variance

$$P\left(\left|\frac{\bar{X}-\bar{Y}}{\sqrt{\frac{{\sigma }_X^2}{n}+\frac{{\sigma }_Y^2}{m}}}\right|>z_{\frac{\alpha }{2}}\right)=\alpha$$

Reject $H_0$ at a confidence level $\alpha$ if

$$|\bar{x}-\bar{y}|>z_{\frac{\alpha }{2}}\sqrt{\frac{{\sigma }_X^2}{n}+\frac{{\sigma }_Y^2}{m}}$$

Two-sample t-test with unknown equal variance

$$S_p^2=\frac{(n-1)S_{n-1,X}^2+(m-1)S_{n-1,Y}^2}{n+m-2}$$

Reject $H_0$ at a confidence level $\alpha$ if

• Two-sided test

$$\bar{x}-\bar{y}>t_{n+m-2,\frac{\alpha }{2}}{S}_p\sqrt{\frac{1}{n}+\frac{1}{m}}$$

• One sided greater test

$$\bar{x}-\bar{y}>t_{n+m-2,\alpha }{S}_p\sqrt{\frac{1}{n}+\frac{1}{m}}$$

• One-sided less test

$$\bar{x}-\bar{y}>t_{n+m-2,\alpha }{S}_p\sqrt{\frac{1}{n}+\frac{1}{m}}$$

Two-sample t-test with unknown unequal variance (Welch’s t-test)

$$v=\left(\frac{{\left(\frac{s_{n-1,X}^2}{n}+\frac{s_{n-1,Y}^2}{m}\right)}^2}{\frac{1}{n-1}{\left(\frac{s_{n-1,X}^2}{n}\right)}^2+\frac{1}{m-1}{\left(\frac{s_{n-1,Y}^2}{M}\right)}^2}\right)$$ $$|\bar{x}-\bar{y}|>t_{v,\alpha /2}\sqrt{\frac{s_{n-1,X}^2}{n}+\frac{s_{n-1,Y}^2}{m}}$$

Use Welch’s t-test by default if no information is given

Power t-test (optional)

power.t.test(delta=2.5, sd=2, sig.level = 0.05, power=0.80, type="two.sample",alternative = "two.sided")

ANOVA

• Compare two or more groups of data/samples

• Factor is a categorical variable denoting the different groups the data come from

• Possible values of factor is called levels

• The main numerical variable of the sample values is called the dependent variable

• If only 1 factor, one-way ANOVA

• If multiple factors, multi-way ANOVA (optional for this course)

Median

1. Order the data points from small to large
2. If n is odd, then median is the $\frac{n+1}{2}$ th data point
3. If n is even, then median is the average of and $\frac{n}{2}$ and $\frac{n}{2}+1$ th data points

Quantile

1. Order the data points from small to large and calculate $k=nq+0.5$
2. If k is an integer, the k-th data point is the quantile
3. If k is not an integer, the quantile is the average $\lfloor k\rfloor$ and $\lfloor k\rfloor +1$ th data point. $\lfloor k\rfloor$ is the largest integer smaller than k.

Boxplot

• Points outside $[Q1-1.5\times IQR,Q3+1.5\times IQR]$ are potential outliers
• LEFT: Smallest value greater than lower quartile minus 1.5 times IQR
• RIGHT: Largest value less than upper quartile plus 1.5 times IQR
  • Think of it as shrinking the 2 bars in order to fit to a data point

One-way ANOVA

1. Group 1: $Y_{1,1},\dots ,Y_{1,n_1}$, iid samples from $N({\mu }_1,{\sigma }_1^2)$
2. Group n: $Y_{n,1},\dots ,Y_{n,n_k}$, iid samples from $N({\mu }_k,{\sigma }_k^2)$
3. Assume samples from different groups are independent
4. Assume variance unknown but equal, i.e. ${\sigma }_1^2=\cdots ={\sigma }_k^2={\sigma }^2$

$$Y_{ij}={\mu }_i+{ϵ}_{ij}\sim N(0,{\sigma }^2)$$

$ϵ$ is noise

$$\begin{array}{rl}& H_0:{\mu }_1=\cdots ={\mu }_k\\ & H_1:\text{some means are different}\end{array}$$

Types of Variance

• Total variance, sum of square total

$$SST=\sum _{i=1}^k\sum _{j=1}^{n_i}(y_{ij}-{\bar{y}}_{..}{)}^2$$

${\bar{y}}_{..}$ is the mean of all the sample data

• Between (group) variance, sum of square treatment

$$S{S}_{Treat}=\sum _{i=1}^k\sum _{j=1}^{n_i}({\bar{y}}_{i.}-{\bar{y}}_{..}{)}^2=\sum _{i=1}^k{n}_i({\bar{y}}_{i.}-{\bar{y}}_{..}{)}^2$$

• Within (group) variance, sum of square error

$$SSE=\sum _{i=1}^k\sum _{j=1}^{n_i}(y_{ij}-{\bar{y}}_{i.}{)}^2$$

• Relations

$$SST=S{S}_{Treat}+SSE$$

F Statistics

If null hypo is true, SST should be close to 0

A large $\frac{S{S}_{Treat}}{SSE}$ support $H_1$

$$F=\frac{S{S}_{Treat}/(k-1)}{SSE/(n-k)}=\frac{M{S}_{Treat}}{MSE}\sim F(k-1,n-k)$$

MS_Treat = Mean square treatment
MSE = Mean square error

n - k because each group have n_each - 1 freedom, sum together we have n_total - k

Reject $H_0$ if F is large

Under $H_0$, $E(M{S}_{Treat})=E(MSE)=E(MST)={\sigma }^2$, $MST=\frac{SST}{n-1}$

Reject H0 if $F>F_{k-1,n-k,\alpha }$

Relation to t-test

if $k=2$, we can apply both two-sided two-sample t-test and ANOVA

• Equivalent if $k=2$, $F=t^2$
• Welch’s t-test is equivalent to Welch’s ANOVA at $k=2$, $F_W=t^2$

Tukey’s Honestly Significant Difference (HSD)

res_aov = aov(time~treatment, data=rat_poison)
TukeyHSD(res_aov)
plot(TukeyHSD(res_aov))