BEA529 Probability & Statistical Inference
NHH · Autumn 2026

4 · Sampling and Elementary Statistical Inference

← Back to schedule

Drill exercises

D1. Let \(X_1, \ldots, X_n\) be a random sample from an exponential distribution with mean \(\beta\). Write down the joint density of the sample, and compute \(P(X_1 > c, \ldots, X_n > c)\) for a constant \(c > 0\).

D2. A random sample of size \(n = 25\) is drawn from a population with standard deviation \(\sigma = 10\). Compute \(\operatorname{Var}(\bar{X})\) and the standard deviation of \(\bar{X}\). How large must \(n\) be for that standard deviation to be halved?

D3. Let \(X_1, \ldots, X_n\) be a random sample from a population with mean \(\mu\) and variance \(\sigma^2\). Show that \[T_1 = \bar{X}, \qquad T_2 = \frac{X_1 + X_2}{2}, \qquad T_3 = X_1\] are all unbiased estimators of \(\mu\), and compare their variances.

D4. For the sample \(2, 4, 4, 6, 9\), compute \(\bar{x}\) and \(s^2\), first from the definition of \(s^2\) and then from the identity \(\sum_i x_i^2 - n\bar{x}^2\).

D5. Let \(X_1, \ldots, X_n\) be a random sample from \(N(\mu, \sigma^2)\). Use the moment generating function of \(\bar{X}\) to identify its distribution.

D6. State the distribution of each of the following.

  1. \(\sum_{i=1}^{5} Z_i^2\), where \(Z_1, \ldots, Z_5 \overset{\text{iid}}{\sim} N(0,1)\).

  2. \(U + V\), where \(U \sim \chi^2_4\) and \(V \sim \chi^2_7\) are independent.

  3. \(Z^2\), where \(Z \sim N(0,1)\).

D7. Let \(X_1, \ldots, X_{16}\) be a random sample from \(N(\mu, \sigma^2)\). State the sampling distribution of each of \[\frac{\bar{X} - \mu}{\sigma/4}, \qquad \frac{15 S^2}{\sigma^2}, \qquad \frac{\bar{X} - \mu}{S/4}.\]

Problems

P1. Let \(x_1, \ldots, x_n\) be any numbers with mean \(\bar{x}\).

  1. Show that \(\sum_{i=1}^n (x_i - a)^2\) is minimised at \(a = \bar{x}\).

  2. Show that \(\sum_{i=1}^n (x_i - \bar{x})^2 = \sum_{i=1}^n x_i^2 - n\bar{x}^2\).

P2. Let \(X_1, \ldots, X_n\) be a random sample and let \(g\) be a function for which \(E\,g(X_i)\) and \(\operatorname{Var} g(X_i)\) exist. Show that \[E\Bigl(\sum_{i=1}^n g(X_i)\Bigr) = n\,E\,g(X_1), \qquad \operatorname{Var}\Bigl(\sum_{i=1}^n g(X_i)\Bigr) = n\operatorname{Var} g(X_1),\] and state which of the two requires the observations to be independent.

P3. Let \(X_1, \ldots, X_n\) be a random sample from a population with mean \(\mu\) and variance \(\sigma^2\). Show that \(E(S^2) = \sigma^2\).

P4. Let \(X_1, \ldots, X_n\) be a random sample from a population with moment generating function \(M_X(t)\).

  1. Show that the moment generating function of \(\bar{X}\) is \(M_{\bar{X}}(t) = \bigl(M_X(t/n)\bigr)^n\).

  2. Use this to find the distribution of \(\bar{X}\) when the population is \(\text{Gamma}(\alpha, \beta)\).

P5. Let \(X_1, \ldots, X_n\) be a random sample from \(\text{Bernoulli}(p)\).

  1. Show that \(\sum_{i=1}^n X_i \sim \text{Bin}(n,p)\), and that \(\bar{X}\) is an unbiased estimator of \(p\).

  2. Show that \(\bar{X}(1 - \bar{X})\) is not an unbiased estimator of \(p(1-p)\), and determine the factor by which its expectation differs.

P6. The sample variance \(S^2\) is an unbiased estimator of \(\sigma^2\). Show that \(S\) is not an unbiased estimator of \(\sigma\), and determine in which direction it errs.

P7. Let \(Z \sim N(0,1)\). Show that \(Z^2 \sim \chi^2_1\).

P8. Let \(X_1, X_2\) be a random sample of size \(2\) from \(N(\mu, \sigma^2)\).

  1. Show that \(S^2 = (X_2 - X_1)^2/2\).

  2. Deduce that \(S^2/\sigma^2 \sim \chi^2_1\).

P9. Let \(X_1, \ldots, X_n\) be a random sample from \(N(\mu, \sigma^2)\). The \(t\)-statistic can be written as \[\frac{\bar{X} - \mu}{S/\sqrt{n}} = \frac{(\bar{X}-\mu)/(\sigma/\sqrt{n})}{\sqrt{S^2/\sigma^2}}.\] Identify the distribution of the numerator and of the denominator, state the further property of the sample that the argument requires, and conclude that the statistic is \(t_{n-1}\) distributed.

P10. Let \(X_1, \ldots, X_n\) be independent with common cdf \(F\) and density \(f\). Derive the cdf and the density of \(X_{(n)} = \max_i X_i\) and of \(X_{(1)} = \min_i X_i\).

P11. Let \(X_1, \ldots, X_n\) be a random sample from \(\text{Uniform}(0, \theta)\).

  1. Find the density of \(X_{(n)}\) and compute \(E\bigl(X_{(n)}\bigr)\).

  2. Show that \(\frac{n+1}{n}X_{(n)}\) is an unbiased estimator of \(\theta\).

P12. Suppose \(n\) serial numbers are drawn without replacement from \(\{1, 2, \ldots, N\}\), and let \(m\) denote the largest number observed. It can be shown that \(E\bigl(X_{(n)}\bigr) = n(N+1)/(n+1)\).

  1. Show that \(\hat{N} = \frac{n+1}{n}m - 1\) is an unbiased estimator of \(N\).

  2. Evaluate \(\hat{N}\) for the observed serial numbers \(19, 40, 42, 60\).

P13. A student argues as follows.

\(\bar{X}\) and \(S^2\) are computed from the same \(n\) observations, and the definition of \(S^2\) refers to \(\bar{X}\). They can therefore not be independent.

Identify what is correct and what is incorrect in this argument.

P14. Let \(X_1, \ldots, X_n\) be a random sample from the exponential distribution with mean \(\beta\), and let \(\lambda = 1/\beta\) be the rate. By Theorem 5.2.6, \(\bar{X}\) is an unbiased estimator of \(\beta\), and a natural estimator of \(\lambda\) is \(1/\bar{X}\). Recall that the exponential distribution with mean \(\beta\) is the \(\text{Gamma}(1, \beta)\) distribution, and that \(\int_0^\infty t^{a-1}e^{-t/\beta}\,dt = \Gamma(a)\beta^a\) for \(a > 0\).

  1. Show that \(T = \sum_{i=1}^n X_i \sim \text{Gamma}(n, \beta)\).

  2. For \(n \geq 2\), show that \(E(1/T) = 1/\bigl((n-1)\beta\bigr)\).

  3. Deduce that \(1/\bar{X}\) is a biased estimator of \(\lambda\), state the direction of the bias, and give an unbiased estimator of \(\lambda\).

  4. Show that \(E(1/X_1) = \infty\), so that the case \(n = 1\) has to be excluded in (b).

P15. Let \(X_1, \ldots, X_n\) be a random sample from \(N(\mu, \sigma^2)\). The sample variance divides \(\sum_{i=1}^n (X_i - \bar{X})^2\) by \(n - 1\), which makes it unbiased for \(\sigma^2\). This problem finds the divisor that gives the smallest mean squared error, where \(\text{MSE}(T) = E\bigl((T - \sigma^2)^2\bigr) = \operatorname{Var}(T) + \bigl(E(T) - \sigma^2\bigr)^2\). The \(\chi^2_p\) distribution has moment generating function \((1 - 2t)^{-p/2}\) for \(t < 1/2\).

  1. Show that a \(\chi^2_p\) variable has mean \(p\) and variance \(2p\).

  2. Use Theorem 5.3.1(c) to show that \(\operatorname{Var}(S^2) = 2\sigma^4/(n-1)\).

  3. For a constant \(c > 0\), find \(\text{MSE}(cS^2)\) as a function of \(c\), and find the value of \(c\) that minimises it. Which divisor of \(\sum_{i=1}^n (X_i - \bar{X})^2\) does this correspond to? Compare its MSE with the MSEs for the divisors \(n - 1\) and \(n\).

P16. Let \(X_1, \ldots, X_n\) be a random sample from \(N(\mu_X, \sigma^2)\), and let \(Y_1, \ldots, Y_m\) be an independent random sample from \(N(\mu_Y, \sigma^2)\), with the same variance \(\sigma^2\). Both sample variances \(S_X^2\) and \(S_Y^2\) estimate \(\sigma^2\), and they can be combined into the pooled variance \[S_p^2 = \frac{(n-1)S_X^2 + (m-1)S_Y^2}{n + m - 2}.\] Recall that if \(U \sim N(0,1)\) and \(V \sim \chi^2_p\) are independent, then \(U/\sqrt{V/p}\) has a \(t_p\) distribution.

  1. Show that \(S_p^2\) is an unbiased estimator of \(\sigma^2\).

  2. Show that \((n + m - 2)S_p^2/\sigma^2 \sim \chi^2_{n+m-2}\).

  3. Show that the two-sample \(t\)-statistic \(T = \dfrac{(\bar{X} - \bar{Y}) - (\mu_X - \mu_Y)}{S_p\sqrt{1/n + 1/m}}\) has a \(t_{n+m-2}\) distribution.

P17. Let \(X_1, \ldots, X_n\) be a random sample from \(\text{Uniform}(0, \theta)\). In P11 you found the density of \(X_{(n)}\) and showed that \(\frac{n+1}{n}X_{(n)}\) is an unbiased estimator of \(\theta\). Since \(E(X_i) = \theta/2\), the estimator \(2\bar{X}\) is unbiased as well. This problem compares the two by their variances. Recall that \(\operatorname{Var}(X_i) = \theta^2/12\).

  1. Show that \(\operatorname{Var}(2\bar{X}) = \theta^2/(3n)\).

  2. Compute \(E\bigl(X_{(n)}^2\bigr)\), and show that \(\operatorname{Var}\bigl(\frac{n+1}{n}X_{(n)}\bigr) = \theta^2/\bigl(n(n+2)\bigr)\).

  3. Compute the ratio of the two variances. Which estimator do you prefer, and how does the difference depend on \(n\)?

Advanced problems

A1. Let \(X_1, X_2\) be a random sample of size \(2\) from \(N(\mu, \sigma^2)\).

  1. Show that \(X_1 + X_2\) and \(X_2 - X_1\) are independent.

  2. Deduce that \(\bar{X}\) and \(S^2\) are independent.

A2. Write \(\bar{X}_k\) and \(S_k^2\) for the sample mean and sample variance of the first \(k\) observations. Establish the recursion \[(n-1)S_n^2 = (n-2)S_{n-1}^2 + \frac{n-1}{n}\bigl(X_n - \bar{X}_{n-1}\bigr)^2.\]

A3. For a random sample of size \(n\) from a continuous distribution with cdf \(F\) and density \(f\), the density of the \(j\)-th order statistic is \[f_{X_{(j)}}(x) = \frac{n!}{(j-1)!\,(n-j)!}\,F(x)^{j-1}\bigl(1-F(x)\bigr)^{n-j}f(x).\]

  1. Give the counting argument behind this expression.

  2. Apply it to the median of a random sample of size \(3\) from \(\text{Uniform}(0,1)\), and identify the resulting distribution.

A4. Let \(X_1, \ldots, X_n\) be a random sample from a population with mean \(\mu\) and variance \(\sigma^2\), and consider estimators of the form \(\sum_{i=1}^n a_i X_i\) with \(\sum_{i=1}^n a_i = 1\). Show that the variance is minimised by \(a_i = 1/n\) for every \(i\), that is, by the sample mean.

A5. Let \(X_1, \ldots, X_n\) and \(Y_1, \ldots, Y_m\) be independent random samples from \(N(\mu_X, \sigma_X^2)\) and \(N(\mu_Y, \sigma_Y^2)\) respectively.

  1. Show that the ratio \(\dfrac{S_X^2/\sigma_X^2}{S_Y^2/\sigma_Y^2}\) has an \(F_{n-1,\,m-1}\) distribution.

  2. Explain how this justifies a test of the hypothesis \(\sigma_X^2 = \sigma_Y^2\).