Bayes

Bayes’ Rule

$$P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}$$

Assumes that each input attribute is independent, and given the target attribute is known.
$P(X,Y,Z\mid A)=P(X\mid A)P(Y\mid A)P(Z\mid A)$

Proof

$$\begin{array}{rl}P(B\mid A)& =\frac{P(BA)}{P(A)}\\ P(BA)& =P(AB)=P(B\mid A)P(A)\\ & \\ P(A\mid B)& =\frac{P(AB)}{P(B)}\\ & =\frac{P(B\mid A)P(A)}{P(B)}\end{array}$$

Bayesian Belief Network

Known Parent Rule

A node is conditionally independent of its non-descendent if its parents are known

If A is the node, B is its parent, C is its non-descendent

$$P(A\mid B,C)=P(A\mid B)$$

Sibling Rule

If A, B are siblings of C

$$P(A,B\mid C)=P(A\mid C)P(B\mid C)$$

Conditionally Independent Rule

• A, B, C are random variables
• A is conditionally independent of B given C, i.e. $P(A\mid B,C)=P(A\mid C)$

$$\begin{array}{rl}P(A,B\mid C)& =\frac{P(A,B,C)}{P(C)}\\ & =\frac{P(A,B,C)}{P(B,C)}\frac{P(B,C)}{P(C)}\\ & =P(A\mid B,C)P(B\mid C)\\ & =P(A\mid C)P(B\mid C)\end{array}$$

Add Condition Rule

$$\begin{array}{rl}P(A=\text{Yes})& =P(A=\text{Yes}\mid B=\text{Yes})P(B=\text{Yes})\\ & +P(A=\text{Yes}\mid B=\text{No})P(B=\text{No})\end{array}$$

All rules can be extended for more conditions, simply add the condition behind each probability function