A proof of Bayes' theorem

Posted by Diego Assencio on 2013.10.12 under Mathematics (Statistics and probability)

Let $A$ and $B$ be two events and let $P(A|B)$ be the conditional probability of $A$ given that $B$ has occurred. Then Bayes' theorem states that: $$ \boxed{ P(B|A) = \displaystyle\frac{P(A|B)P(B)}{P(A)} = \displaystyle\frac{P(A|B)P(B)}{P(A|B)P(B)+ P(A|B^c)P(B^c)} } $$

In other words, Bayes' theorem gives us the conditional probability of $B$ given that $A$ has occurred as long as we know $P(A|B)$, $P(A|B^c)$ and $P(B) = 1 - P(B^c)$.

The proof of this equation is quite simple. First, consider the following facts: $$P(A|B) := \displaystyle\frac{P(A \cap B)}{P(B)} \Longrightarrow P(A \cap B) = P(A|B)P(B) $$ $$ \begin{eqnarray} P(A) & = & P((A \cap B) \cup (A \cap B^c)) \nonumber\\[5pt] & = & P(A \cap B) + P(A \cap B^c) \nonumber\\[5pt] & = & P(A|B)P(B) + P(A|B^c)P(B^c) \label{post_bcb216aa523d43833789bc008e394275_second_fact} \end{eqnarray} $$ where on equation \eqref{post_bcb216aa523d43833789bc008e394275_second_fact} the fact that $A \cap B$ and $A \cap B^c$ are mutually exclusive events was used. Therefore, since $A\cap B = B\cap A$: $$ \begin{eqnarray} P(B|A) &=& \displaystyle\frac{P(B \cap A)}{P(A)} \nonumber\\[5pt] &=& \displaystyle\frac{P(A|B)P(B)}{P(A)} \nonumber\\[5pt] &=& \displaystyle\frac{P(A|B)P(B)}{P(A|B)P(B)+ P(A|B^c)P(B^c)} \label{post_bcb216aa523d43833789bc008e394275_proof} \end{eqnarray} $$ as we wanted to prove. We can also prove a very interesting formula using Bayes' theorem. From equation \eqref{post_bcb216aa523d43833789bc008e394275_proof} (used with both $B$ and $B^c$) and equation \eqref{post_bcb216aa523d43833789bc008e394275_second_fact}, we have: $$ P(B|A) + P(B^c|A) = \displaystyle\frac{P(A|B)P(B)}{P(A)} + \displaystyle\frac{P(A|B^c)P(B^c)}{P(A)} = \displaystyle\frac{P(A)}{P(A)} = 1 $$ so we obtain: $$ \boxed{ P(B|A) = 1 - P(B^c|A) } $$


MANISH NAIDU on Feb 07, 2017:
thanks a lot for your help...
Chiranjeevi Kanaka on Apr 13, 2018:
given $A \cap B$ and $A \cap B'$ are independent events, but they are actually mutually exclusive events.
Diego Assencio on Apr 13, 2018:
@Chiranjeevi: Fixed, thank you! :-)
Hari on Mar 11, 2020:
Thank you so much sir and madam
Jasmine on Sep 18, 2020:
Thank you - this was easy to follow and concise.
Zachary Njuguna on Dec 22, 2020:
Thank you for the calculative and sober help.
Romhányi István on Jun 13, 2021:
That's all perfect, but easier to remember the following.

The definition of conditional probability is
(1) P(B|A)=P(B^A)/P(A)

Multiply both sides of Bayes formula by P(A)
and you get
(2) P(B|A)*P(A)=P(A|B)*P(B)

Use (1) in both sides of (2) and get
(3) P(B^A)/P(A)*P(A)=P(A^B)/P(B)*P(B)

You get an evidence

Leave a reply

NOTE: A name and a comment (max. 1024 characters) must be provided; all other fields are optional. Equations will be processed if surrounded with dollar signs (as in LaTeX). You can post up to 5 comments per day.