Independent Random Variables
Random variables $F$ and $G$ are independent if:
\[\mathbf{P}(F,G)=\mathbf{P}(F)\times\mathbf{P}(G)\]That is, for all values $r$ and $s$:
\[P(F=r,G=s)=P(F=r)\times P(G=s)\]As one’s dental problems do not influence the weather, the pairs of random variables are each independent:
- $\text{Toothache},\text{Weather}$
- $\text{Catch},\text{Weather}$
- $\text{Cavity},\text{Weather}$
Example - Weather and Dental Problems
The full joint probability distribution:
\[\mathbf{P}(\text{Toothache, Catch, Cavity, Weather})\]has 32 entries. It contains four tables for the dentist variables and one for the four kinds of weather.
The number of entries comes from the $2\times2\times2\times4$ of the 2 outcomes for the 4 dentist variables and the 4 outcomes of the $\text{Weather}$ variable.
Thus we have:
- 8 probabilities for $(\text{Weather}=\text{sunny})$:
- $P(\text{Weather}=\text{sunny}, \text{Toothache}=r_1,$ $\text{Catch} =r_2,\text{Cavity}=r_3)$
- And so on for the other three weather types.
Clearly we can make the independence assumption that for any combination of values of the random variables $\text{Toothache, Catch, Cavity,}$ the probability for $\text{Weather}$. For example:
\[\begin{aligned} & P(\text{Weather}=\text{sunny} \vert\\ & \text{Toothache}=r_1, \text{Catch} =r_2,\text{Cavity}=r_3)\\ =& P(\text{Weather}=\text{sunny}) \end{aligned}\]for all $r_1,r_2,r_3\in\{0,1\}$.
Thus equivalently:
\[\begin{aligned} & P(\text{Weather}=\text{sunny} \vert\\ & \text{Toothache}=r_1, \text{Catch} =r_2,\text{Cavity}=r_3)\\ =& P(\text{Weather}=\text{sunny})\times\\ &P(\text{Toothache}=r_1, \text{Catch} =r_2,\text{Cavity}=r_3) \end{aligned}\]for all $r_1,r_2,r_3\in\{0,1\}$.
This means that they are related proportionally.
We have seen that the join probability distribution
\[\mathbf{P}(\text{Weather, Toothache, Catch, Cavity})\]can be written as:
\[\mathbf{P}(\text{Weather})\times\mathbf{P}(\text{Toothache, Catch, Cavity})\]The 32-element table for four variables can be constructed from one 4-element table and an 8-element table.
Independence Analysis
Unfortunately this type of independence is very rare as generally things that you want to compare are inter-related. This is especially true within a single domain. Conditional independence is much more prevalent and useful.