Hypothesis

Terms

Null hypothesis $H_0$

• Assumed to be true
• Tested to be rejected or not to be rejected

Alternative hypothesis $H_1$

• The underlying question
• Complement of $H_0$

Test statistics

• Quantity derived from samples
• e.g. $\bar{X}$ for ${\mu }_X$, $S_{n-1}^2$ for ${\sigma }_X^2$

Errors

• Type I Error

$$P(\text{reject H0 when H0 is true})=\alpha$$

• Type II Error

$$P(\text{not reject H0 when H1 is true})=\beta$$

• Power

$$\text{Power}=1-\beta$$

Only either one of $\alpha$ or $\beta$ of can be minimized

	Not reject $H_0$	Reject $H_0$
$H_0$ true	✔️	Type I error $\alpha$
$H_1$ true	Type II error $\beta$	✔️

Testing

Direct/Indirect comparison

There are two ways to evaluate a test, either by statistics value, e.g. $\frac{\bar{x}-{\mu }_0}{s_{n-1}/\sqrt{n}}<t_{n-1,\alpha }$, or by probability value, i.e. $\text{p-value}<\alpha$

Testing $\mu$ with Known $\sigma$

Test	Hypothesis	Reject $H_0$ at a significance level $\alpha$ if	Rejection Region
Two-sided Test	$H_0:\mu ={\mu }_0$ $H_1:\mu \ne {\mu }_0$	$\bar{x}<{\mu }_0-z_{\alpha /2}\frac{\sigma }{\sqrt{n}}$ $\bar{x}>{\mu }_0+z_{\alpha /2}\frac{\sigma }{\sqrt{n}}$	$\left(-\infty ,{\mu }_0-z_{\alpha /2}\frac{\sigma }{\sqrt{n}}\right)$ $\left({\mu }_0+z_{\sigma /2}\frac{\sigma }{\sqrt{n}},+\infty \right)$
One-sided Less Test (Left)	$H_0:\mu ={\mu }_0$ $H_1:\mu <{\mu }_0$	$\bar{x}<{\mu }_0-z_{\alpha }\frac{\sigma }{\sqrt{n}}$	$(-\infty ,{\mu }_0-z_{\alpha }\frac{\sigma }{\sqrt{n}})$
One-sided Greater Test (Right)	$H_0:\mu ={\mu }_0$ $H_1:\mu >{\mu }_0$	$\bar{x}>{\mu }_0+z_{\alpha }\frac{\sigma }{\sqrt{n}}$	$({\mu }_0+z_{\alpha }\frac{\sigma }{\sqrt{n}},+\infty )$

One-sided Greater Test Example

$$P\left(\frac{\bar{X}-\mu }{\sqrt{{\sigma }^2/n}}>\frac{c-\mu }{\sqrt{{\sigma }^2/n}}\right)=P\left(Z>\frac{c-\mu }{\sqrt{{\sigma }^2/n}}\right)=\alpha$$ $$\{\begin{array}{l}{H}_0:\mu ={\mu }_0\\ H_1:\mu ={\mu }_1>{\mu }_0\end{array}$$ $$P\left(\frac{\bar{X}-{\mu }_1}{\sigma /\sqrt{n}}\le z_{\alpha }+\frac{{\mu }_0-{\mu }_1}{\sigma /\sqrt{n}},\text{ when μ=μ1}\right)=\beta$$

Testing $\mu$ with Unknown $\sigma$

Test	Hypothesis	Reject $H_0$ at a significance level $\alpha$ if
Two-sided Test	$H_0:\mu ={\mu }_0$ $H_1:\mu \ne {\mu }_0$	$\left|\frac{\bar{x}-{\mu }_0}{s_{n-1}/\sqrt{n}}\right|>t_{n-1,\alpha /2}$
One-sided Less Test (Left)	$H_0:\mu ={\mu }_0$ $H_1:\mu <{\mu }_0$	$\frac{\bar{x}-{\mu }_0}{s_{n-1}/\sqrt{n}}<t_{n-1,1-\alpha }=-t_{n-1,\alpha }$
One-sided Greater Test (Right)	$H_0:\mu ={\mu }_0$ $H_1:\mu >{\mu }_0$	$\frac{\bar{x}-{\mu }_0}{s_{n-1}/\sqrt{n}}>t_{n-1,\alpha }$

p-value Example

• p-value is the probability of obtaining a test result that is equal or more extreme (in the direction of rejecting $H_0$) than the actually observed result, under $H_0$
• Reject $H_0$ at a significance level $\alpha$ if $\text{p-value}<\alpha$
• $\frac{\bar{x}-{\mu }_0}{s_{n-1}/\sigma }$ is the t-value, note that $\bar{x}$ is the observed value

One sided

$$P\left(\frac{\bar{X}-{\mu }_0}{s_{n-1}/\sqrt{n}}\ge \text{t-value}\right)=\text{p-value}$$

Two sided

$$P\left(\left|\frac{\bar{X}-{\mu }_0}{s_{n-1}/\sqrt{n}}\right|\ge |\text{t-value}|\right)=2P\left(\frac{\bar{X}-{\mu }_0}{s_{n-1}/\sqrt{n}}\ge |\text{t-value}|\right)=\text{p-value}$$

Power of t-test

Leave 1 variable NULL to find the unknown, delta is $\Delta \mu$

power.t.test(n = NULL, delta = NULL, sd = 1, sig.level = 0.05,
             power = NULL,
             type = c("two.sample", "one.sample", "paired"),
             alternative = c("two.sided", "one.sided"))

Larger ${\sigma }^2$ > smaller power
Larger $\Delta \mu ={\mu }_0-{\mu }_1$> larger power
Larger $n$ > larger power

Testing $\sigma$ with Unknown $\mu$

Test	Hypothesis	Reject $H_0$ at a significance level $\alpha$ if	p-value, $U\sim {\chi }_{n-1}$
Two-sided Test	$H_0:\mu ={\mu }_0$ $H_1:\mu \ne {\mu }_0$	$s_{n-1}^2<{\sigma }_0^2\frac{{\chi }_{n-1,1-\alpha /2}^2}{n-1}$ $s_{n-1}^2>{\sigma }_0^2\frac{{\chi }_{n-1,\alpha /2}^2}{n-1}$	$\text{p-value}=2\times \min \left[P\left(U>\frac{(n-1)s_{n-1}^2}{{\sigma }_0^2}\right),P\left(U<\frac{(n-1)s_{n-1}^2}{{\sigma }_0^2}\right)\right]$
One-sided Less Test (Left)	$H_0:\mu ={\mu }_0$ $H_1:\mu <{\mu }_0$	$s_{n-1}^2<{\sigma }_0^2\frac{{\chi }_{n-1,1-\alpha }^2}{n-1}$	$\text{p-value}=P\left(U<\frac{(n-1)s_{n-1}^2}{{\sigma }_0^2}\right)$
One-sided Greater Test (Right)	$H_0:\mu ={\mu }_0$ $H_1:\mu >{\mu }_0$	$s_{n-1}^2>{\sigma }_0^2\frac{{\chi }_{n-1,\alpha }^2}{n-1}$	$\text{p-value}=P\left(U>\frac{(n-1)s_{n-1}^2}{{\sigma }_0^2}\right)$


CI vs Hypothesis Testing

$$C=1-\alpha$$

Two-sided z-test

$${\mu }_0\notin \left[\bar{X}-z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}},\bar{X}+z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}}\right]$$

Two-sided t-test

$${\sigma }_0^2\notin \left[\frac{(n-1)s_{n-1}^2}{{\chi }_{n-1,\alpha /2}},\frac{(n-1)s_{n-1}^2}{{\chi }_{n-1,1-\alpha /2}}\right]$$