Populations and Samples

From a population to a sample

Statistics often begins with a question about a large group. The population is the entire group that the question concerns; it is not necessarily a population of people. If a manufacturer wants to estimate the lifetime of a particular batch of bulbs, every bulb in that batch is the population. If a head teacher wants to know what pupils at one school think of its canteen, the population is all pupils at that school.

A sample is the smaller group that is actually investigated. It must be drawn from the relevant population: surveying pupils at a different school would not provide a sample of the population in the head teacher’s question.

A census attempts to collect data from every member of the population. This can remove uncertainty caused by selecting only a sample, but it may be slow, expensive or impractical. Some tests also destroy the item being tested, so testing every item would leave nothing to use. A sample is usually more manageable, but its results provide estimates rather than certainty about the whole population.

Inferring properties of the population

An inference uses what is observed in a sample to draw a conclusion about the population. For example, the proportion of sampled households that own two or more cars can be used to estimate the corresponding proportion among all households in a town.

If the sample contains $n$ members and $x$ have the property of interest, the sample proportion is

$$\frac{x}{n}.$$

An estimate of the number in a population of size $N$ with that property is therefore

$$\text{estimated number}=\frac{x}{n}\times N.$$

This calculation assumes that the sample is representative of the population. The answer is an estimate: a different sample is likely to produce a slightly different proportion. A larger suitable sample will usually reduce this chance-to-chance variation, but size alone cannot repair a biased method.

The same principle applies to a distribution. If the sample contains a spread of ages, journey times or test scores, that pattern can be used to estimate the spread in the population—provided the sample represents the population reasonably well.

Simple random sampling

In a simple random sample, every member of the population has an equal chance of selection. A practical procedure is:

1. obtain a complete list of the population, called a sampling frame;
2. give every member a different number;
3. use a random-number generator or random-number table to select numbers from the full range;
4. choose the corresponding members, ignoring repeated numbers if a sample of distinct members is required;
5. continue until the required sample size has been reached.

Merely choosing people who look varied, asking friends or selecting the first names on a register is not simple random sampling. Human choice can favour some members, even unintentionally.

Random selection reduces selection bias, but it does not guarantee that one particular sample will perfectly match the population. A genuinely random sample can be unrepresentative purely by chance. This random variation is different from bias: variation changes unpredictably from sample to sample, whereas bias repeatedly pushes results in a particular direction.

Recognising sampling bias

A sampling method is biased when some members of the population are more likely to be included than others in a way that could distort the result. Asking people outside a gym about exercise is likely to over-represent people who exercise. An online voluntary poll may over-represent people who use that website and those with strong views. Asking shoppers only on a weekday morning can miss many people who are at work then.

A useful criticism identifies three things:

• who is over-represented or excluded;
• why the selection method causes this;
• how that could push the result.

Bias may also come from an incomplete sampling frame or from non-response if those who do not answer differ systematically from those who do. Taking more observations from the same biased source produces a more precise description of that source, not necessarily a better estimate of the intended population.

Judging an inference

Before accepting a conclusion drawn from a sample, ask:

• Was the correct population defined?
• Could every member be selected by the stated method?
• Is the sampling frame complete?
• Is the sample large enough to give a reasonably stable estimate?
• Might the place, time, wording or voluntary nature of the survey create bias?
• Is the conclusion presented as an estimate rather than an exact population fact?

A sound inference depends on both the selection method and the amount of data. Random selection helps prevent systematic bias; an adequate sample size helps reduce random variation.

Core distinction

• Population: the complete group being studied.
• Sample: the smaller group from which data are collected.
• Census: data are sought from every member of the population.
• Sample results are used to make an inference about the population.

Estimating from a sample

$$\text{sample proportion}=\frac{\text{number with the property}}{\text{sample size}}$$ $$\text{estimated population number}=\text{sample proportion}\times\text{population size}$$

The estimate assumes that the sample is representative. Different samples may give different estimates.

Simple random sample

1. Obtain a complete sampling frame.
2. Number every member.
3. Generate random numbers across the full range.
4. Select the corresponding distinct members until the sample is full.

Every population member must have an equal chance of selection.

Bias and variation

• Bias: the method systematically favours or excludes part of the population.
• Random variation: even an unbiased random sample may differ from the population by chance.
• A larger appropriate sample usually reduces random variation.
• A larger sample does not remove bias.

When evaluating a method, identify who is over-represented or omitted and explain the likely effect on the conclusion.