Practice Midterm 1

QTM 285-1

These solutions give examples of full-credit answers. Where we derived a formula in homework, citing it and applying it is enough on an exam.

Problem 1

Suppose a friend has polled 100 of Georgia’s registered voters about their intention to vote in the 2024 election. They conducted this poll by sampling with replacement from the population of all of Georgia’s registered voters. And they report what they say is a 95% confidence interval for turnout: [.58,.62].

Part A

Has your friend calibrated their interval estimate correctly, i.e., are they really using an interval estimator that has 95% coverage? Say yes or no, then explain your answer.

Solution

No. The interval is centered at .60, giving an estimated standard error of \sqrt{(.6)(.4)/100}\approx .05. A 95% interval should extend about two standard errors, or .10, to each side. Their interval extends only .02.

Part B

If they wanted an interval estimate that was accurate to within 1%, i.e. an interval of the form \hat\theta \pm .01, roughly how many voters should they have polled? Choose the closest number from this list: 100, 1000, 10000, 100000. Circle you choice.

Solution

10,000. The half-width needs to shrink tenfold, from .10 to .01. Because the standard error decreases as 1/\sqrt n, we need 10^2=100 times as many voters: 100\times100=10{,}000.

Problem 2

A population of m=200 people is arranged in a circle, and their responses alternate as you go around it: 1, 0, 1, 0, \ldots. So the proportion of 1s in the population is \theta=1/2. We’ll compare three ways of drawing a sample of n=100 people from it. In each case, we estimate \theta by the proportion of 1s in the sample, \hat\theta=\frac{1}{100}\sum_{i=1}^{100} Y_i, and report the interval estimate \hat\theta \pm 1.96\sqrt{\hat\theta(1-\hat\theta)/n}.

Draw 100 people uniformly at random with replacement.
Draw 100 people uniformly at random without replacement.
Choose one starting person uniformly at random, and take that person and the next 99 people around the circle.

One of these intervals has coverage of roughly 95%, and the other two have noticeably more. Which is which? Explain.

Solution

With replacement: roughly 95%. The interval uses the standard error for this sampling design and extends roughly two standard errors to each side.
Without replacement: more than 95%. The HW 4 variance formula multiplies the with-replacement variance by (m-n)/(m-1)=100/199\approx1/2. The standard error is therefore about 1/\sqrt2 as large, but the reported interval has approximately the same width, so it covers more often.
The consecutive run: 100%. Every run of 100 people contains 50 ones. The estimate is always 1/2, exactly the population proportion, so the interval always covers it.

Problem 3

A coworker has a sample of n=100 household incomes Y_1 \ldots Y_{100}, drawn with replacement from a population, and wants a 95% interval estimate for the population’s mean income. They bootstrap: they draw 10{,}000 samples of size 100 with replacement from their sample, take the mean \bar Y^\star of each, and report \bar Y \pm 1.96\,\text{se}, where \text{se} is the standard deviation of these bootstrap means. They’ve done everything right. But their boss, who has a habit of correcting people without taking the time to understand what they’re doing, glances at it and says:

For the pictures, take the sample to be simulated from a log-normal income population with median 60{,}000 and log-scale standard deviation 0.5.

You forgot to divide by \sqrt n. Rookie mistake.

To sort this out, your coworker draws two pictures on the same axis: the distribution of the incomes in their sample on the left, and the bootstrap distribution of \bar Y^\star on the right.

Histogram of 100 simulated log-normal household incomes, from about 20 thousand to 200 thousand dollars, with the sample mean marked by a vertical line.

Histogram of 10,000 bootstrap sample means, tightly clustered around the sample mean on the same narrowed dollar axis.

Part A

Is the boss right? Say what each picture is a picture of, and explain why the standard deviation of the picture on the right is, or isn’t, the right standard error for an interval estimate of the mean income.

Solution

No. The left picture shows the incomes in the sample. If all we had were that spread, we’d divide it by \sqrt n to get the spread of the mean of n households, because averaging n independent draws shrinks the spread by that factor. The right shows means of bootstrap samples, each an average of a sample of 100 incomes. Bootstrapping the mean gives us an estimate of the sampling distribution of the mean when we’ve sampled with replacement, so its standard deviation already estimates the mean’s standard error. Effectively, the division by \sqrt n has already happened—it’s there in the bootstrapped mean’s standard deviation.

Part B

Your coworker wants to end their explanation with a flourish: “If you want to think of the standard deviation of Y_1 \ldots Y_n—the one you’d divide by \sqrt n—in bootstrap terms, it’s the bootstrap standard deviation of an estimator you’d never actually use. It’s the standard deviation of …”

Finish the sentence. What simple estimator has bootstrap standard deviation \sigma=\sqrt{\frac{1}{n}\sum\{Y_i - \bar Y\}^2}, the same standard deviation as the sample Y_1 \ldots Y_n?

Tip. There’s more than one. Any one you can think of is fine.

Solution

Use the first household’s income, Y_1, as the estimate of the population mean.

When we bootstrap this estimator, our estimates are points drawn independently from the sample, i.e. Y_1^\star=Y_i with probability 1/n. The expected value of Y_1^\star is \bar Y. To calculate its variance, we average the squared distances from \bar Y, weighting each possible draw by its probability 1/n: \begin{aligned} \operatorname{Var}^{\star}(Y_1^\star) &=\mathbb E[(Y_1^\star-\bar Y)^2] \\ &=\sum_{i=1}^{n}\frac{1}{n}(Y_i-\bar Y)^2 \\ &=\frac{1}{n}\sum_{i=1}^{n}(Y_i-\bar Y)^2. \end{aligned} This is the sample variance.

Here \operatorname{Var}^{\star} means variance over bootstrap draws, holding the observed sample fixed.

Problem 4

Two polling firms each survey the same population in exactly the same way, independently of one another: each calls 1,000 of Georgia’s registered voters, drawn with replacement, and reports the proportion who say they’ll turn out. The plot below shows what a single survey of this kind produces, with two particular surveys marked and the actual turnout rate \theta drawn in green.

Sampling distribution of a turnout survey of 1,000 voters: a histogram centered on the green target line at .7, with two highlighted dots for particular surveys and a double-headed arrow between them.

Part A

Typically, how far apart are the two surveys’ estimates? That is, what is the root-mean-squared length of the arrow drawn between them? Give your answer as a multiple of one survey’s standard error, and show your calculation.

Solution

The root-mean-squared distance is \sqrt2\,\mathrm{se}, about 1.4\,\mathrm{se}. Write A and B for the two estimates. Both have expectation \theta. Because they are independent, the expected product of their centered values is zero: \begin{aligned} \mathbb E\{(A-B)^2\} &=\mathbb E\bigl[\{(A-\theta)-(B-\theta)\}^2\bigr] \\ &=\mathbb E\{(A-\theta)^2\} -2\mathbb E\{(A-\theta)(B-\theta)\} +\mathbb E\{(B-\theta)^2\} \\ &=\mathrm{se}^2-2\times0\times0+\mathrm{se}^2 =2\,\mathrm{se}^2. \end{aligned} Taking the square root gives the answer.

Part B

The two firms report estimates 6 percentage points apart. Is that a surprising amount of disagreement?

Solution

Yes. One survey’s standard error is \sqrt{(.7)(.3)/1000}\approx .0145, so the standard deviation of the difference is \sqrt2\times .0145\approx .02. A six-percentage-point gap is about three standard deviations.

Additional explanation (not expected in your answer). We did not provide the sampling distribution of the difference. Here it is, computed from the two independent polls.

You can work out the distribution of a sum or difference of two independent binomials, or two independent random variables generally, by writing out the joint distribution of the two partial sums and marginalizing. It’s called convolution. You did it in a small example in HW3, Exercise 11: Splitting the Street. At sample sizes like this, you’d really want a computer for that.

About 0.37% of independent pairs of polls differ by at least six percentage points—the two pink tails.

Problem 5

You have a sample Y_1,Y_2...,Y_n drawn with replacement from a population with mean \mu and standard deviation \sigma. The usual estimate for \mu is the sample mean \bar{Y}_0 = \frac{1}{n}\sum_{i=1}^n Y_i. But one of your coworkers has a clever idea to reduce variance—instead of dividing by n, they divide by n+10. This is their estimator: \bar{Y}_{10} = \frac{1}{n+10}\sum_{i=1}^n Y_i. You try to convince them it’s a bad idea by being a bit hyperbolic—you argue that if that’s a good idea, then why not divide by n+100? They, unfortunately, love this idea. You now have to explain why this is a bad idea.

Your coworker has calculated formulas for the bias and standard deviation of the ‘divide by n+k estimator’ \bar{Y}_k=\frac{1}{n+k}\sum_{i=1}^n Y_i. \begin{aligned} \mathop{\mathrm{E}}[\bar{Y}_k] - \mu &= -\frac{k}{n+k}\mu \\ \mathop{\mathrm{sd}}[\bar{Y}_k] &= \frac{\sigma}{\sqrt{n}} \times \frac{n}{n+k}. \end{aligned}

Formula Derivation. Optional Reading.

\begin{aligned} \mathop{\mathrm{E}}\sb*{\bar{Y}_k} - \mu &= \mathop{\mathrm{E}}\sb*{\frac{n}{n+k}\bar Y_0} - \mu \\ &= \frac{n}{n+k}\mathop{\mathrm{E}}[\bar Y_0] - \mu \\ &=\frac{n}{n+k}\mu - \mu \\ &= \p*{\frac{n}{n+k} - 1}\mu \\ &= \p*{\frac{n}{n+k} - \frac{n+k}{n+k}}\mu \\ &= -\frac{k}{n+k}\mu \\ & \\ \mathop{\mathrm{\mathop{\mathrm{V}}}}\sb*{\bar{Y}_k} &= \mathop{\mathrm{\mathop{\mathrm{V}}}}\sb*{\frac{n}{n+k} \bar {Y}_0} \\ &= \p*{\frac{n}{n+k}}^2 V[\bar{Y}_0] \\ &= \frac{n^2}{(n+k)^2} \frac{\sigma^2}{n} \\ &= \frac{ n\sigma^2}{(n+k)^2}=\frac{\sigma^2}{n} \times \frac{n^2}{(n+k)^2} \end{aligned}

And they’ve drawn the sampling distribution of \bar{Y}_0, \bar{Y}_{10}, and \bar{Y}_{100} (top, middle, and bottom rows respectively) for sample sizes n=100 and n=10,000 (left and right columns respectively) for \mu=0 and \sigma=1. The solid line is the estimator’s expected value \mathop{\mathrm{E}}[\bar{Y}_k], the dashed lines are \mathop{\mathrm{E}}[\bar Y_k] \pm \mathop{\mathrm{sd}}[\bar Y_k], and the dotted lines are \mathop{\mathrm{E}}[\bar Y_k] \pm 2 \mathop{\mathrm{sd}}[\bar Y_k].

Six sampling distributions of the divide-by-n-plus-k estimator for k equal to 0, 10, and 100 (rows) at n equal to 100 and 10,000 (columns), all centered at zero, each narrower than the one above it.

On the basis of those drawings, your coworker proposes using \bar{Y}_{100} to estimate \mu.

Exercise 1

Suppose your sample size is n=100. Explain why your coworker’s original proposal to use \bar Y_{10} and your hyperbolic suggestion to use \bar Y_{100} appear, from the drawings, to be a way to increase precision without increasing bias. To explain why this is misleading, describe what happens in the case that \mu=11/10 and \sigma=1 by doing the following.

Modifying the drawings to show the sampling distributions in this new case.
Stating an approximation to the coverage of each of the confidence intervals \bar{Y}_{k} \pm 1.96\ \mathop{\mathrm{sd}}[\bar{Y}_k] for k=0, k=10, and k=100.

Tip. You can ‘change the location’ of the distributions in the drawing above by changing the labels on the x-axis.

Solution

The estimator is a weighted average of the sample mean and zero, with weight n/(n+k) on the sample mean. This is equivalent to adding k fake observations, all zero. It shrinks the sample mean toward zero.

In the original pictures, \mu=0. All three estimators are unbiased, and increasing k reduces their standard error, so the pictures show increased precision without bias.

Start by relabeling the original plots in SE units: write \mu at the solid line, \mu\pm1\,\mathrm{se} at the dashed lines, and \mu\pm2\,\mathrm{se} at the dotted lines. Changing \mu does not change an estimator’s SE, so one SE remains the same distance on the paper.

For n=100, the plots for k=0,10,100 are shown from top to bottom.

For \mu=11/10, the supplied formulas give \frac{\mathrm{bias}}{\mathrm{se}} =-\frac{k\mu}{\sigma\sqrt n} =-\frac{1.1k}{10}. Keep each curve and its reference lines where they are. Negative bias means the curve’s center is to the left of the target, so draw the green \mu line to its right:

Estimator	Bias/SE	Green target line	Approximate coverage
\bar Y_0	0	At the center	95%
\bar Y_{10}	-1.1	Just beyond the right dashed line: 1.1\,\mathrm{se} from the center	80%
\bar Y_{100}	-11	Far to the right: 11\,\mathrm{se} from the center	0%

The k=10 case corresponds to the 80% point on the coverage curve in Bias and Coverage. For k=100, an interval reaching about two standard errors to either side almost never reaches the target eleven standard errors away.