A proof of Bayes' theorem

Posted by Diego Assencio on 2013.10.12 under Mathematics (Statistics and probability)

Let $A$ and $B$ be two events and let $P(A|B)$ be the conditional probability of $A$ given that $B$ has occurred. Then Bayes' theorem states that: $$\boxed{ P(B|A) = \displaystyle\frac{P(A|B)P(B)}{P(A)} = \displaystyle\frac{P(A|B)P(B)}{P(A|B)P(B)+ P(A|B^c)P(B^c)} }$$

In other words, Bayes' theorem gives us the conditional probability of $B$ given that $A$ has occurred as long as we know $P(A|B)$, $P(A|B^c)$ and $P(B) = 1 - P(B^c)$.

The proof of this equation is quite simple. First, consider the following facts: $$P(A|B) := \displaystyle\frac{P(A \cap B)}{P(B)} \Longrightarrow P(A \cap B) = P(A|B)P(B)$$ $$\begin{eqnarray} P(A) & = & P((A \cap B) \cup (A \cap B^c)) \nonumber\\[5pt] & = & P(A \cap B) + P(A \cap B^c) \nonumber\\[5pt] & = & P(A|B)P(B) + P(A|B^c)P(B^c) \label{post_bcb216aa523d43833789bc008e394275_second_fact} \end{eqnarray}$$ where on equation \eqref{post_bcb216aa523d43833789bc008e394275_second_fact} the fact that $A \cap B$ and $A \cap B^c$ are mutually exclusive events was used. Therefore, since $A\cap B = B\cap A$: $$\begin{eqnarray} P(B|A) &=& \displaystyle\frac{P(B \cap A)}{P(A)} \nonumber\\[5pt] &=& \displaystyle\frac{P(A|B)P(B)}{P(A)} \nonumber\\[5pt] &=& \displaystyle\frac{P(A|B)P(B)}{P(A|B)P(B)+ P(A|B^c)P(B^c)} \label{post_bcb216aa523d43833789bc008e394275_proof} \end{eqnarray}$$ as we wanted to prove. We can also prove a very interesting formula using Bayes' theorem. From equation \eqref{post_bcb216aa523d43833789bc008e394275_proof} (used with both $B$ and $B^c$) and equation \eqref{post_bcb216aa523d43833789bc008e394275_second_fact}, we have: $$P(B|A) + P(B^c|A) = \displaystyle\frac{P(A|B)P(B)}{P(A)} + \displaystyle\frac{P(A|B^c)P(B^c)}{P(A)} = \displaystyle\frac{P(A)}{P(A)} = 1$$ so we obtain: $$\boxed{ P(B|A) = 1 - P(B^c|A) }$$

 Chiranjeevi Kanaka on Apr 13, 2018: given $A \cap B$ and $A \cap B'$ are independent events, but they are actually mutually exclusive events.