UoL CS Notes
Menu

Bayes' Theorem

COMP111 Lectures

First Form

If $P(A)>0$, then:

\[P(B\vert A)=\frac{P(A\vert B)\times P(B)}{P(A)}\]

Proof

We have:

  • $P(A\cap B)=P(A\vert B)\times P(B)$
  • $P(A\cap B)=P(B\vert A)\times P(A)$

Thus:

\[P(A\vert B)\times P(B)=P(B\vert A)\times P(A)\]

By dividing by $P(A)$ we get:

\[P(B\vert A)=\frac{P(A\vert B)\times P(B)}{P(A)}\]

Application - Diagnosis

Assume a patient walks into a doctor’s office complaining of a stiff neck. The doctor knows:

  • Meningitis may cause a patient to have a stiff neck 50% of the time.
    • Causal knowledge.
  • The probability of having meningitis is $\frac{1}{50000}$
  • The probability of having a stiff neck is $\frac{1}{20}$

What is the probability that the patient has meningitis?

Let $A$ be the event that the patient has a stiff neck and $B$ the event that they have meningitis:

\[\begin{aligned} P(B\vert A)&=\frac{P(A\vert B)\times P(B)}{P(A)}=\\ \frac{\frac{1}{2}\times \frac{1}{50000}}{\frac{1}{20}}&=\frac{1}{5000} \end{aligned}\]
  • We can interpret the fact that the patient has a stiff neck as a new observation.
  • Given this observation, we want to classify that patient as either having meningitis or not having meningitis.
  • We have prior knowledge about the unconditional probability of having a stiff neck.
  • We have causal knowledge about the number of times in which meningitis causes a stiff neck.

  • We can then compute the diagnostic probabilities using:

    \[P(B\vert A)=\frac{P(A\vert B)\times P(B)}{P(A)}\]

Alternative Form

You may not have the prior probability for $A$ (the observation). In this case you can use other things that you might know in this alternative form.

If $P(A)>0$, then:

\[\begin{aligned} &P(B\vert A)=\\ &\frac{P(A\vert B)\times P(B)}{P(A\vert B)\times P(B)+P(A\vert \neg B)\times P(\neg B)} \end{aligned}\]

Proof

It suffices to show:

\[P(A)=P(A\vert B)\times P(B)+P(A\vert \neg B) \times P(\neg B)\]

But this follows from:

\[\begin{aligned} P(A)&=P((A\cap B)\cup (A\cap \neg B))\\ &=P(A\cap B)+P(A\cap\neg B)\\ &=P(A\vert B)\times P(B)+P(A\vert \neg B)\times P(\neg B) \end{aligned}\]

Application - Diagnosis

Assume a drug test is:

  • Positive for users 99% of the time.
  • Negative for non-users 99% of the time.

Assume that 0.5% take the drug.

What is the probability that a person whose test is positive (event $A$) takes the drug (event $B$)?

We have:

  • $P(A\vert B)=\frac{99}{100}$
  • $P(\neg A\vert \neg B)=\frac{99}{100}$
  • $P(B)=\frac{1}{200}$

Thus:

  • $P(A\vert \neg B) =\frac{1}{100}$
  • $P(\neg B) =\frac{199}{200}$

Thus:

\[\begin{aligned} &P(B\vert A)=\\ &\frac{P(A\vert B)\times P(B)}{P(A\vert B)\times P(B)+P(A\vert \neg B)\times P(\neg B)}\\ &=\frac{99}{298}\approx0.33 \end{aligned}\]

Due to the low value it means that it is hard to take an action based on the test. This is as a result of the low value of the people who take the drug. This results in many false positives for those that don’t.