5 · Limit Theory

Drill exercises

D1. Let \(X\) have mean \(10\) and variance \(4\).

Use Chebyshev’s inequality to bound \(P(|X - 10| \geq 6)\).
For a random sample of size \(n\) from the same population, determine how large \(n\) must be for the corresponding bound on \(P(|\bar{X}_n - 10| \geq 1)\) to be at most \(0.05\).

D2. State the definitions of \(X_n \overset{p}{\to} X\), \(X_n \overset{a.s.}{\to} X\) and \(X_n \overset{d}{\to} X\), and order the three modes of convergence by strength.

D3. Let \(X_1, X_2, \ldots\) be independent \(\text{Bernoulli}(p)\) variables. Show that \(\bar{X}_n \overset{p}{\to} p\), and give a bound on \(P(|\bar{X}_n - p| \geq \varepsilon)\) that does not depend on \(p\).

D4. A random sample of size \(n = 100\) is drawn from a population with mean \(\mu = 3\) and variance \(\sigma^2 = 4\). Use the central limit theorem to approximate \(P(\bar{X}_n > 3.4)\).

D5. Let \(X_n\) be exponentially distributed with rate \(n\). Show that \(X_n \overset{d}{\to} 0\).

D6. Suppose \(X_n \overset{d}{\to} N(0,1)\) and \(Y_n \overset{p}{\to} 3\). Use Slutsky’s theorem to give the limiting distribution of each of \(X_n + Y_n\), \(Y_n X_n\) and \(X_n / Y_n\).

D7. Suppose \(\sqrt{n}(Y_n - \theta) \overset{d}{\to} N(0, \sigma^2)\) with \(\theta = 2\) and \(\sigma^2 = 1\). Use the delta method to find the limiting distribution of \(\sqrt{n}(Y_n^2 - 4)\).

Problems

P1. Let \(X\) have mean \(\mu\) and finite variance \(\sigma^2\). Prove Chebyshev’s inequality: for any \(k > 0\), \[P(|X - \mu| \geq k) \leq \frac{\sigma^2}{k^2}.\]

P2. Let \(X_1, X_2, \ldots\) be a random sample from a population with mean \(\mu\) and finite variance \(\sigma^2\). Prove the weak law of large numbers, and state where the finiteness of \(\sigma^2\) is used.

P3. Let \(X_1, X_2, \ldots\) be a random sample with mean \(\mu\) and variance \(\sigma^2\), and assume \(E(X_i^4) < \infty\).

Apply the weak law of large numbers to \(Y_i = (X_i - \mu)^2\) to show that \(n^{-1}\sum_{i=1}^n (X_i - \mu)^2 \overset{p}{\to} \sigma^2\).
Deduce that \(S_n^2 \overset{p}{\to} \sigma^2\).

P4. State the continuous mapping theorem for convergence in probability. Then let \(\bar{X}_n \overset{p}{\to} \mu\) with \(\mu > 0\), and use the theorem to show that \(\sqrt{\bar{X}_n} \overset{p}{\to} \sqrt{\mu}\) and \(1/\bar{X}_n \overset{p}{\to} 1/\mu\). Explain where the assumption \(\mu > 0\) is needed.

P5. Let \(c\) be a constant. Show that if \(X_n \overset{d}{\to} c\), then \(X_n \overset{p}{\to} c\).

P6. Let \(X_1, X_2, \ldots\) be independent \(\text{Uniform}(0,1)\) variables and let \(X_{(n)} = \max_{i \leq n} X_i\).

Show that \(X_{(n)} \overset{p}{\to} 1\).
Show that \(n\bigl(1 - X_{(n)}\bigr) \overset{d}{\to} \text{Exponential}(1)\).

P7. Let \(X_1, \ldots, X_n\) be independent \(\text{Exponential}(1)\) variables and let \(X_{(1)} = \min_i X_i\). Show that \(nX_{(1)}\) is exactly \(\text{Exponential}(1)\) distributed for every \(n\), not only in the limit, and compare this with P6(b).

P8. Let \(X_1, \ldots, X_{50}\) be independent \(\text{Poisson}(2)\) variables.

Identify the exact distribution of \(\sum_{i} X_i\), and use the central limit theorem to approximate \(P\bigl(\sum_i X_i \leq 110\bigr)\).
Repeat the approximation with a continuity correction, and compare both answers with the exact value \(0.8529\).

P9. Let \(X_1, X_2, \ldots\) be a random sample with mean \(\mu\), variance \(\sigma^2\) and moment generating function existing in a neighbourhood of zero. Put \(Y_i = (X_i - \mu)/\sigma\).

Show that \(\sqrt{n}(\bar{X}_n - \mu)/\sigma = n^{-1/2}\sum_{i=1}^n Y_i\), and that its moment generating function is \(\bigl(M_Y(t/\sqrt{n})\bigr)^n\).
Show that \(M_Y(0) = 1\), \(M_Y'(0) = 0\) and \(M_Y''(0) = 1\).

P10. Let \(X_1, \ldots, X_n\) be a random sample with mean \(\mu\) and finite variance \(\sigma^2\). Using the central limit theorem, the consistency of \(S_n\) and Slutsky’s theorem, show that \[\frac{\sqrt{n}(\bar{X}_n - \mu)}{S_n} \overset{d}{\to} N(0,1).\]

P11. Let \(X_1, X_2, \ldots\) be independent \(\text{Bernoulli}(p)\) with \(0 < p < 1\), so that \(\sqrt{n}(\bar{X}_n - p) \overset{d}{\to} N\bigl(0, p(1-p)\bigr)\). Use the delta method to find the limiting distribution of

\(\sqrt{n}\bigl(\bar{X}_n^2 - p^2\bigr)\),
\(\sqrt{n}\left(\log\dfrac{\bar{X}_n}{1 - \bar{X}_n} - \log\dfrac{p}{1-p}\right)\).

P12. Let \(X_1, X_2, \ldots\) be independent \(\text{Poisson}(\lambda)\) variables, so that \(\sqrt{n}(\bar{X}_n - \lambda) \overset{d}{\to} N(0, \lambda)\). Find a function \(g\) for which the asymptotic variance of \(\sqrt{n}\bigl(g(\bar{X}_n) - g(\lambda)\bigr)\) does not depend on \(\lambda\).

P13. A student argues as follows.

If \(X_n \overset{d}{\to} X\), then for large \(n\) the random variable \(X_n\) is close to \(X\). So convergence in distribution means that the difference \(X_n - X\) becomes small.

Identify what is incorrect in this argument, and give a counterexample.

Advanced problems

A1. Divide \([0,1]\) into intervals in the following order: first \([0,1]\); then \([0,\tfrac12]\) and \([\tfrac12,1]\); then \([0,\tfrac13]\), \([\tfrac13,\tfrac23]\) and \([\tfrac23,1]\); and so on, so that the \(k\)-th block consists of the \(k\) intervals of length \(1/k\). Let \(I_n\) denote the \(n\)-th interval in this list, let \(\omega\) be drawn uniformly on \([0,1]\), and set \(X_n = \mathbf{1}\{\omega \in I_n\}\).

Show that \(X_n \overset{p}{\to} 0\).
Show that for no \(\omega\) does the sequence \(X_n(\omega)\) converge, so that \(X_n\) does not converge to \(0\) almost surely.

A2. Let \(X_1, X_2, \ldots\) be a random sample with mean \(0\) and variance \(\sigma^2\), so that \(\sqrt{n}\,\bar{X}_n \overset{d}{\to} N(0, \sigma^2)\). Take \(g(y) = y^2\), for which \(g'(0) = 0\), so the delta method does not apply.

Show that \(n\bar{X}_n^2 \overset{d}{\to} \sigma^2\chi^2_1\), and explain why the condition \(g'(\theta) \neq 0\) appears in the delta method.

A3. Let \(X_1, X_2, \ldots\) be independent \(\text{Exponential}(1)\) variables and let \(X_{(n)} = \max_{i \leq n} X_i\). Show that \[P\bigl(X_{(n)} - \log n \leq x\bigr) \longrightarrow \exp\bigl(-e^{-x}\bigr)\] for every real \(x\), and comment on why the maximum has to be centred here rather than scaled as in P6.

A4. Show that convergence in probability implies convergence in distribution.