1 · Review of Basic Probability

Drill exercises

D1. State the three axioms of a probability set function. Then show, for any two events \(A\) and \(B\), that \[P(A \cup B) \leq P(A) + P(B),\] and say which axiom you used at each step.

D2. An urn contains 5 red and 7 black balls. Four balls are drawn without replacement. Find the probability that exactly two of them are red.

D3. A diagnostic test for a certain condition has sensitivity \(0.99\) (the probability of a positive test given that the condition is present) and specificity \(0.95\) (the probability of a negative test given that it is absent). The condition is present in \(0.5\%\) of the population.

A randomly chosen person tests positive. What is the probability that the person has the condition? Comment on the size of your answer relative to the sensitivity of the test.

D4. The notes ask, without answering, whether one can give an example of an outcome that gives rise to several different random variables. Answer it: describe a single random experiment, state its sample space, and define three different random variables on it.

D5. Let \(X\) have density \[f(x) = \begin{cases} 2x & 0 < x < 1 \\ 0 & \text{elsewhere.} \end{cases}\] Find the cumulative distribution function \(F_X\), verify that it has the properties a cdf must have, and compute \(P\!\left(\tfrac14 < X < \tfrac12\right)\) in two different ways.

D6. Three functions are proposed as cumulative distribution functions on \(\mathbb{R}\): \[F_1(x) = \begin{cases}0 & x<0\\ x/2 & 0\le x<1\\ 1 & x\ge 1,\end{cases} \quad F_2(x) = \begin{cases}0 & x<0\\ 1-e^{-x} & x\ge 0,\end{cases} \quad F_3(x) = \begin{cases}0 & x<0\\ \sin x & 0\le x<\pi\\ 0 & x\ge \pi.\end{cases}\] For each, decide whether it is a legitimate cdf. Where it is not, name a property that fails and give a value of \(x\) at which it fails.

D7. Let \(X\) take the values \(0\), \(1\) and \(2\) with probabilities \(\tfrac12\), \(\tfrac14\) and \(\tfrac14\). Compute \(E(X)\), \(\operatorname{Var}(X)\) and the moment generating function \(M_X(t)\), and verify that \(M_X'(0) = E(X)\) and \(M_X''(0) = E(X^2)\).

Problems

P1. Prove that for any events \(A_1, \ldots, A_n\), \[P\!\left(\bigcup_{i=1}^{n} A_i\right) \leq \sum_{i=1}^{n} P(A_i).\]

P2. Derive the inclusion–exclusion formula for three events, \[\begin{aligned} P(A \cup B \cup C) = {}& P(A) + P(B) + P(C) \\ &- P(A\cap B) - P(A\cap C) - P(B\cap C) + P(A\cap B\cap C), \end{aligned}\] using only property 5 of the notes and the algebra of sets.

P3. Show that for arbitrary events \(C_1\) and \(C_2\), \[P(C_2 \cap C_1^c) = P(C_2) - P(C_1 \cap C_2),\] and deduce property 3 of the notes (monotonicity) as a special case. State clearly where each axiom is used.

P4. Let \(B\) be an event with \(P(B) > 0\), and define \(Q(A) = P(A \mid B)\) for every event \(A\). Show that \(Q\) is itself a probability set function on the same collection of events \(\mathcal{B}\).

P5. A family has two children. You may assume that each child is a boy or a girl with probability \(\tfrac12\), independently.

Given that at least one of the children is a girl, what is the probability that both are girls?
Given that the elder child is a girl, what is the probability that both are girls?

Write out the sample space and the conditioning event in each case, and explain why the two answers differ.

P6. Urn I contains 2 red and 3 black balls; urn II contains 4 red and 1 black ball. An urn is chosen at random, with equal probabilities, and one ball is drawn from it.

Given that the ball drawn is red, find the probability that it came from urn II.
Now replace the single black ball in urn II by \(b\) black balls, so that urn II contains 4 red and \(b\) black. Show that the probability in (a), viewed as a function of \(b\), is strictly decreasing, and explain why this is what one should expect.

P7. Suppose \(A\) and \(B\) are independent events. Prove that \(A^c\) and \(B\) are independent, and that \(A^c\) and \(B^c\) are independent.

P8. A student argues as follows.

Events \(A\) and \(B\) are mutually exclusive, so they cannot happen together. Learning that \(B\) has occurred therefore tells us nothing about whether \(A\) occurred, and so \(A\) and \(B\) are independent.

Identify the error in this reasoning. Then prove that if \(A\) and \(B\) are mutually exclusive with \(P(A) > 0\) and \(P(B) > 0\), they are necessarily dependent.

P9. Three events \(A\), \(B\) and \(C\) are called mutually independent if \[P(A\cap B) = P(A)P(B), \quad P(A\cap C) = P(A)P(C), \quad P(B\cap C) = P(B)P(C)\] and, in addition, \[P(A\cap B\cap C) = P(A)P(B)P(C).\] Construct three events that satisfy the first three conditions but not the fourth, and conclude that the fourth condition does not follow from the other three.

P10. Let \(X\) be a continuous random variable whose cdf \(F\) is strictly increasing on the set where \(0 < F(x) < 1\). Show that the random variable \(Y = F(X)\) is uniformly distributed on \((0,1)\).

P11. The notes derive the pmf of \(Y = g(X)\) for a discrete random variable \(X\) when \(g\) is one-to-one. Derive the corresponding expression when \(g\) is not one-to-one, and apply it to the case where \(X\) is uniform on \(\{-2,-1,0,1,2\}\) and \(Y = X^2\).

P12. Starting from the definition of the variance, prove that \[\operatorname{Var}(X) = E(X^2) - \mu^2 \qquad\text{and}\qquad \operatorname{Var}(aX + b) = a^2\operatorname{Var}(X)\] for constants \(a\) and \(b\), and say in words why \(b\) disappears while \(a\) is squared.

P13. Let \(X\) have moment generating function \(M_X(t)\), existing for \(|t| < h\), and let \(Y = aX + b\).

Express \(M_Y\) in terms of \(M_X\), and use it to obtain \(E(Y)\) and \(\operatorname{Var}(Y)\).
A random variable has moment generating function \(M(t) = (1-t)^{-1}\) for \(t < 1\). Identify its distribution, and justify the identification by pointing to the result that you use.

P14. Let \(X\) take the value \(2^k\) with probability \(2^{-k}\), for \(k = 1, 2, 3, \ldots\)

Verify that this defines a legitimate probability mass function.
Compute \(E(X)\).
A casino offers you a single play of the game that pays \(X\). What would you be willing to pay for it, and how do you reconcile your answer with (b)?

Advanced problems

A1. Let \(A_1 \subset A_2 \subset A_3 \subset \cdots\) be an increasing sequence of events. Prove that \[P\!\left(\bigcup_{n=1}^{\infty} A_n\right) = \lim_{n \to \infty} P(A_n).\] State and prove the corresponding result for a decreasing sequence \(B_1 \supset B_2 \supset \cdots\).

A2. Show that there is no probability set function on the sample space \(\mathcal{X} = \{1, 2, 3, \ldots\}\) that assigns the same probability to every integer. What does this tell you about the phrase “pick a positive integer at random”?

A3. Let \(F\) be a cumulative distribution function. Prove that \(F\) has at most countably many discontinuities.

A4. Each of \(n\) letters is placed at random into one of \(n\) addressed envelopes, one letter per envelope, with all \(n!\) arrangements equally likely.

Show that the probability that at least one letter reaches its correct envelope is \[\sum_{k=1}^{n} \frac{(-1)^{k+1}}{k!}.\]
Deduce that this probability converges to \(1 - e^{-1}\) as \(n \to \infty\), and comment on why this is surprising.

A5. Construct explicit probabilities for events \(A\), \(B\) and \(C\) such that \[P(A \mid B \cap C) > P(A \mid B^c \cap C) \qquad\text{and}\qquad P(A \mid B \cap C^c) > P(A \mid B^c \cap C^c),\] but nevertheless \[P(A \mid B) < P(A \mid B^c).\] Explain in words how this is possible.

A6. A chord is drawn at random in a circle of radius \(r\). We ask for the probability that the chord is longer than the side of the inscribed equilateral triangle. Compute this probability under each of the following constructions.

Both endpoints of the chord are chosen independently and uniformly on the circumference.
A radius is fixed, and the midpoint of the chord is chosen uniformly along that radius, the chord being perpendicular to it.
The midpoint of the chord is chosen uniformly over the interior of the circle.

Reconcile your three answers.

A7. Let \(X\) be a positive continuous random variable satisfying \[P(X > s + t \mid X > t) = P(X > s) \qquad \text{for all } s, t > 0.\] Writing \(G(x) = P(X > x)\), show that \(G(s+t) = G(s)G(t)\) for all \(s, t > 0\), and conclude that \(G(x) = e^{-\lambda x}\) for some \(\lambda > 0\).

A8. Let \(X\) be a random variable taking values in \(\{0, 1, 2, \ldots\}\).

Prove that \[E(X) = \sum_{k=1}^{\infty} P(X \geq k).\]
State the corresponding formula for a non-negative continuous random variable, and indicate why it should hold.