Conditional Probability

The conditional probability that $E$ occurs given that $F$ has already occurred is denoted by $P (E ∣ F)$ .

It’s not too hard to reason that $P (E ∣ F) = \frac{P ( EF )}{P ( F )}$

The multiplication rule says $P (E_{1} E_{2} E_{3} \dots E_{n}) = P (E_{1}) P (E_{2} ∣ E_{1}) P (E_{3} ∣ E_{1} E_{2}) \dots P (E_{n} ∣ E_{1} \dots E_{n - 1})$

Let $E$ and $F$ be events, since $E = EF \cup E F^{c}$ , we get $P (E) = P (E ∣ F) P (F) + P (E ∣ F^{c}) P (F^{c})$

This is useful, because sometimes it’s easier to calculate conditional probabilities than direct probabilities

Bayes’ theorem states that $P (F ∣ E) = \frac{P ( E ∣ F ) P ( F )}{P ( E )}$
By extension, if $F_{1}, \dots F_{n}$ are mutually exclusive events and we’d like to know which has occurred given $E$ , $P (F_{j} ∣ E) = \frac{P ( E ∣ F _{j} ) P ( F _{j} )}{i = 1 \sum n P ( E ∣ F _{i} ) P ( F _{i} )}$

The above demonstrates the law of total probability, $P (E) = i = 1 \sum n P (E F_{i}) = i = 1 \sum n P (E ∣ F_{i}) P (F_{i})$ where $F_{i}$ are mutually disjoint and cover the sample space $Ω$

The odds of an event are $\frac{P ( H )}{P ( H ^{c} )}$ . After evidence $E$ has been introduced, $\frac{P ( H ∣ E )}{P ( H ^{c} ∣ E )} = \frac{P ( H )}{P ( H ^{c} )} \cdot \frac{P ( E ∣ H )}{P ( E ∣ H ^{c} )}$

It follows that two events are independent if $P (EF) = P (E) P (F)$ . Otherwise, two events are dependent.

$n$ events are independent if any subset of the events are independent. An infinite set of events is independent if any finite subset is independent.

Example: Say Alice flips $n + 1$ coins and Bob flips $n$ coins that have heads with probability $p$ . What’s the probability that Alice flips more?

The probability that Bob gets exactly $k$ coins is $(k n) p^{k} (1 - p)^{n - k}$
If there are $n$ flips for Bob, the probability that Alice wins is $j = k + 1 \sum n + 1 (j n + 1) p^{j} (1 - p)^{n + 1 - j}$

So $P (A wins) = k = 0 \sum n P (A wins ∣ B = k) P (B = k)$
Altogether we get $k = 0 \sum n (j = k + 1 \sum n + 1 (j n + 1) p^{j} (1 - p)^{n + 1 - j} \cdot (k n) p^{k} (1 - p)^{n - k})$
This is fucking ugly
It does turn out that when $p = 1/2$ , $P (A wins) = 1/2$

ALTERNATE way of looking at it,
Let both players flip $n$ coins. At this point, either Alice is winning, Bob is winning, or it’s a tie.
It’s pretty clear that $P (A_{n} > B_{n}) = P (A_{n} < B_{n})$

If she’s already winning then she’s won. If she’s already losing then she’s lost. So the probability she wins in total is $p \cdot P (A_{n} = B_{n}) + 1 \cdot P (A_{n} > B_{n}) + 0 \cdot P (A_{n} < B_{n}) = q + \frac{1}{2} (1 - 2 q) = \frac{1}{2}$

This second argument is very elegant. BUT it is also very fragile. Just about any change to the problem statement would break the logic…

Binyamin's Notes

Explorer