Independent Random Variables

COMP111 Lectures 2020-12-01

Random variables $F$ and $G$ are independent if:

$$\mathbf{P}(F,G)=\mathbf{P}(F)\times \mathbf{P}(G)$$

That is, for all values $r$ and $s$:

$$P(F=r,G=s)=P(F=r)\times P(G=s)$$

As one’s dental problems do not influence the weather, the pairs of random variables are each independent:

• $\text{Toothache},\text{Weather}$
• $\text{Catch},\text{Weather}$
• $\text{Cavity},\text{Weather}$

Example - Weather and Dental Problems

The full joint probability distribution:

$$\mathbf{P}(\text{Toothache, Catch, Cavity, Weather})$$

has 32 entries. It contains four tables for the dentist variables and one for the four kinds of weather.

The number of entries comes from the $2\times 2\times 2\times 4$ of the 2 outcomes for the 4 dentist variables and the 4 outcomes of the $\text{Weather}$ variable.

Thus we have:

• 8 probabilities for $(\text{Weather}=\text{sunny})$:
  • $P(\text{Weather}=\text{sunny},\text{Toothache}=r_1,$ $\text{Catch}=r_2,\text{Cavity}=r_3)$
• And so on for the other three weather types.

Clearly we can make the independence assumption that for any combination of values of the random variables $\text{Toothache, Catch, Cavity,}$ the probability for $\text{Weather}$. For example:

$$\begin{array}{rl}& P(\text{Weather}=\text{sunny}|\\ & \text{Toothache}=r_1,\text{Catch}=r_2,\text{Cavity}=r_3)\\ =& P(\text{Weather}=\text{sunny})\end{array}$$

for all $r_1,r_2,r_3\in \{0,1\}$.

Thus equivalently:

$$\begin{array}{rl}& P(\text{Weather}=\text{sunny}|\\ & \text{Toothache}=r_1,\text{Catch}=r_2,\text{Cavity}=r_3)\\ =& P(\text{Weather}=\text{sunny})\times \\ & P(\text{Toothache}=r_1,\text{Catch}=r_2,\text{Cavity}=r_3)\end{array}$$

for all $r_1,r_2,r_3\in \{0,1\}$.

This means that they are related proportionally.

We have seen that the join probability distribution

$$\mathbf{P}(\text{Weather, Toothache, Catch, Cavity})$$

can be written as:

$$\mathbf{P}(\text{Weather})\times \mathbf{P}(\text{Toothache, Catch, Cavity})$$

The 32-element table for four variables can be constructed from one 4-element table and an 8-element table.

Independence Analysis

Unfortunately this type of independence is very rare as generally things that you want to compare are inter-related. This is especially true within a single domain. Conditional independence is much more prevalent and useful.