3 · Multiple Random Variables
Drill exercises
D1. Let \((X,Y)\) have the joint probability mass function \[ \begin{array}{c|ccc} & X=0 & X=1 & X=2 \\ \hline Y=0 & 0.10 & 0.15 & 0.05 \\ Y=1 & 0.20 & 0.25 & 0.25 \end{array} \] Find the marginal probability mass functions of \(X\) and \(Y\), and determine whether \(X\) and \(Y\) are independent.
D2. For the distribution in D1, find the conditional probability mass function of \(Y\) given \(X = 1\), and compute \(E(Y \mid X = 1)\).
D3. For the distribution in D1, compute \(\operatorname{Cov}(X,Y)\) and \(\operatorname{Cor}(X,Y)\).
D4. Let \(f(x,y) = c(x+y)\) for \(0 < x < 1\) and \(0 < y < 1\). Determine \(c\), find both marginal densities and \(E(X)\), and determine whether \(X\) and \(Y\) are independent.
D5. Use the factorisation lemma to determine whether \(X\) and \(Y\) are independent in each case.
\(f(x,y) = 4xy\) for \(0 < x < 1\), \(0 < y < 1\).
\(f(x,y) = 8xy\) for \(0 < x < y < 1\).
D6. Suppose \(\operatorname{Var}(X) = 4\), \(\operatorname{Var}(Y) = 9\) and \(\operatorname{Cov}(X,Y) = -2\). Compute \(\operatorname{Cor}(X,Y)\), \(\operatorname{Var}(X+Y)\), \(\operatorname{Var}(X-Y)\) and \(\operatorname{Var}(2X - 3Y + 5)\).
D7. Let \(X \sim \text{Poisson}(\lambda_1)\) and \(Y \sim \text{Poisson}(\lambda_2)\) be independent. Use moment generating functions to find the distribution of \(X + Y\).
Problems
P1. Construct two joint probability mass functions on \(\{0,1\}^2\) with the same marginals \(f_X = f_Y = (\tfrac12, \tfrac12)\), one with positive covariance and one with negative covariance. Compute both covariances.
P2. Let \(X\) and \(Y\) be random variables with finite variances, and let \(a\) and \(b\) be constants.
Show that \(\operatorname{Cov}(X,Y) = E(XY) - \mu_X\mu_Y\).
Use this to show that \[\operatorname{Var}(aX + bY) = a^2\operatorname{Var}(X) + b^2\operatorname{Var}(Y) + 2ab\operatorname{Cov}(X,Y).\]
P3. Let \(X\) and \(Y\) be random variables with finite variances.
Show that if \(X\) and \(Y\) are independent, then \(\operatorname{Cov}(X,Y) = 0\).
Let \(X \sim \text{Uniform}(-1,1)\) and \(Y = X^2\). Show that \(\operatorname{Cov}(X,Y) = 0\), and explain why this does not contradict (a).
P4. For each of the following joint densities, determine whether \(X\) and \(Y\) are independent, and state what settles it.
\(f(x,y) = 6x^2y\) for \(0 < x < 1\), \(0 < y < 1\)
\(f(x,y) = 2\) for \(0 < x < y < 1\)
\(f(x,y) = xe^{-x(y+1)}\) for \(x > 0\), \(y > 0\)
\(f(x,y) = \tfrac14(1 + xy)\) for \(-1 < x < 1\), \(-1 < y < 1\)
P5. Let \(f(x,y) = 2\) for \(0 < x < y < 1\). Find both marginal densities, both conditional densities, and \(E(Y \mid X = x)\) and \(E(X \mid Y = y)\).
P6. The law of iterated expectation states that \(E\big(E(Y \mid X)\big) = E(Y)\).
Verify it for the distribution in P5.
Prove it for a continuous random vector \((X,Y)\).
P7. Let \(X\) and \(Y\) be independent \(\text{Exponential}(1)\) variables, and let \(U = X + Y\) and \(V = X/(X+Y)\). Find the joint density of \((U,V)\), show that \(U\) and \(V\) are independent, and identify their marginal distributions.
P8. Let \(X \sim N(\mu_1, \sigma_1^2)\) and \(Y \sim N(\mu_2, \sigma_2^2)\) be independent. Use moment generating functions to find the distributions of \(X+Y\) and \(X-Y\).
P9. Let \(X \sim \text{Bin}(n,p)\) and \(Y \sim \text{Bin}(m,p)\) be independent.
Show that \(X + Y \sim \text{Bin}(n+m, p)\).
Give an example of two independent binomial variables with different success probabilities whose sum is not binomial, and prove that it is not.
P10. Show that \(-1 \leq \rho_{XY} \leq 1\) for any \(X\) and \(Y\) with finite and strictly positive variances.
P11. Let \(a\) and \(c\) be non-zero constants and let \(b\) and \(d\) be arbitrary constants. Show that \[\operatorname{Cor}(aX + b,\ cY + d) = \operatorname{sign}(ac)\,\rho_{XY}.\]
P12. Let \(X_1, \ldots, X_n\) be independent with common mean \(\mu\) and variance \(\sigma^2\), and let \(\bar{X} = n^{-1}\sum_{i=1}^n X_i\).
Show that \(E(\bar{X}) = \mu\) and \(\operatorname{Var}(\bar{X}) = \sigma^2/n\).
Show that \(\operatorname{Cov}(\bar{X},\, X_i - \bar{X}) = 0\) for each \(i\).
P13. A student argues as follows.
If \(X\) and \(Y\) are uncorrelated, then \(\operatorname{Var}(X+Y) = \operatorname{Var}(X) + \operatorname{Var}(Y)\). And if the variances add in this way, then \(X\) and \(Y\) are independent.
Identify what is correct and what is incorrect in this argument.
Advanced problems
A1. The standard bivariate normal density with correlation \(\rho\), where \(|\rho| < 1\), is \[f(x,y) = \frac{1}{2\pi\sqrt{1-\rho^2}} \exp\left\{-\frac{x^2 - 2\rho xy + y^2}{2(1-\rho^2)}\right\}, \qquad (x,y) \in \mathbb{R}^2.\]
Show that the marginal distribution of \(X\) is \(N(0,1)\).
Show that \(Y \mid X = x \sim N(\rho x,\ 1-\rho^2)\).
Show that \(\operatorname{Cor}(X,Y) = \rho\), and that \(X\) and \(Y\) are independent if and only if \(\rho = 0\).
A2. Let \(X \sim N(0,1)\), and let \(Z\) take the values \(-1\) and \(1\) with probability \(\tfrac12\) each, independently of \(X\). Put \(Y = ZX\).
Show that \(Y \sim N(0,1)\).
Show that \(\operatorname{Cov}(X,Y) = 0\).
Show that \(X\) and \(Y\) are not independent.
A3. Let \(X\) and \(Y\) be independent \(\text{Uniform}(0,1)\) variables, and let \(M = \min(X,Y)\) and \(N = \max(X,Y)\).
Show that the joint density of \((M,N)\) is \(2\) on \(0 < m < n < 1\).
Find the marginal densities of \(M\) and \(N\), and compute \(\operatorname{Cor}(M,N)\).
A4. Let \(X\) and \(Y\) have finite variances with \(\operatorname{Var}(X) > 0\). Find the constants \(a\) and \(b\) that minimise \(E\big((Y - a - bX)^2\big)\), and show that the minimum value is \(\sigma_Y^2(1 - \rho^2)\).
A5. Let \((X,Y)\) have joint density \(f\), and put \(U = X + Y\).
Show that the density of \(U\) is \[f_U(u) = \int_{-\infty}^{\infty} f(x,\, u - x)\,dx.\]
Use this to find the density of the sum of two independent \(\text{Uniform}(0,1)\) variables.