Joint Probability Distribution
Examples of Probabilistic Models
To model a domain using probability theory, one first introduces the relevant random variable. We have seen two basic examples:
-
The weather domain could be modelled using the single random variable $\text{Weather}$ with values:
\[(\text{sunny},\text{rain},\text{cloudy},\text{snow})\] -
The dentist domain could be modelled using the random variables $\text{Toothache, Cavity}$ and $\text{Catch}$ with values 0 and 1 for True or False. We might be interested in:
Probability Distribution of a Single Random Variable
The probability distribution for a random variable gives the probabilities of all the possible values of the random variable.
For example let $\text{Weather}$ be a random variable with values:
\[(\text{sunny},\text{rain},\text{cloudy},\text{snow})\]such that its probability distribution is given by:
- $P(\text{Weather}=\text{sunny})=0.7$
- $P(\text{Weather}=\text{rain})=0.2$
- $P(\text{Weather}=\text{cloudy})=0.08$
- $P(\text{Weather}=\text{snow})=0.02$
Assume the order of the values is fixed. The we write instead:
\[\mathbf{P}(\text{Weather})=(0.7,0.2,0.008,0.02)\]Where the bold $\mathbf{P}$ indicates that the result is a vector of number representing the individual values of $\text{Weather}$.
We can write the values as a vector in the case that the properties are ordered.
Joint Probability Distribution
This is the case when you are using many random variables.
Let $F_1,\ldots,F_k$ be random variable. A joint probability distribution for:
\[F_1,\ldots,F_k\]gives the probabilities:
\[P(F_1=r_1,\ldots,F_k=r_k)\]for the event:
\[(F_1=r_1)\text{ and } \cdots \text{ and }(F_k=r_k)\]that $F_1$ takes value $r_1$, $F_2$ take value $r_2$, and so on up to $k$ , for all possible values $r_1,\ldots,r_k$.
The joint probability distribution is denotes $\mathbf{P}(F_1,\ldots,K_k)$.
Example
A possible joint probabillity distribution $\mathbf{P}(\text{Weather, Cavity})$ for the random varables $\text{Weather}$ and $\text{Cavity}$ is given by the following table:
| $\text{Weather}=$ | $\text{sunny}$ | $\text{rain}$ | $\text{cloudy}$ | $\text{snow}$ |
|---|---|---|---|---|
| $\text{Cavity}=1$ | 0.144 | 0.02 | 0.016 | 0.02 |
| $\text{Cavity}=0$ | 0.576 | 0.08 | 0.064 | 0.08 |
For this to be plausible we should know that the two events are independent of each-other.
Full Joint Probability Distribution
A full joint probability distribution:
\[\mathbf{P}(F_1,\ldots,K_k)\]is a joint probability distribution for all relavent random variables $F_1,\ldots,F_k$ for a domain of interest.
Every probability quesetion about a domain can be answered by tehf ull joint distributin because the probability of every evenint is a sum of the probabilities:
\[P(F_1=r_1,\ldots,F_k=r_k)\]The $r_1,\ldots,r_k$ are often called data points or sample points.
Example
Assume the random variables $\text{Toothache, Cavity, Catch}$ fully describe a visit to a dentist.
The a full joint probability distribution is gien by the following table:
| $\text{Toothache}=1$ | $\text{Toothache}=1$ | $\text{Toothache}=0$ | $\text{Toothache}=0$ | |
|---|---|---|---|---|
| $\text{Catch}=1$ | $\text{Catch}=0$ | $\text{Catch}=1$ | $\text{Catch}=0$ | |
| $\text{Cavity}=1$ | 0.108 | 0.012 | 0.072 | 0.008 |
| $\text{Cavity}=0$ | 0.016 | 0.064 | 0.144 | 0.576 |