Planning data collection

A source can give a distorted picture if it mainly includes one type of person or one viewpoint. For example, in a survey about a school sports club, asking only current club members would not fairly represent all pupils: it excludes pupils who chose not to join. Neutral question wording cannot fix that problem.

For secondary data, consider who collected it, why they collected it, which people it covers, how it was collected and how recent it is. A source with an interest in a particular outcome needs careful scrutiny. Acknowledge the source when using its data.

Time affects both feasibility and quality. Interviews take time to arrange and conduct; long questionnaires take time to complete; open answers take time to interpret. A tight deadline may limit how many people you can reach. Also consider when collection happens: an observation at one time of day may not represent behaviour at other times. Plan a collection period suitable for the conclusion you want to draw.

Write neutral, precise questions

A leading question encourages a particular answer. ‘How delighted are you with our excellent new app?’ assumes that the respondent likes the app. ‘How satisfied or dissatisfied are you with the new app?’ is more neutral.

Questions should use short, familiar wording and make clear what is being asked. A question such as ‘How often do you use the app?’ leaves the time period unclear. A more useful closed question is:

During the last seven days, on how many days did you use the app? Tick one answer.

• 0 days
• 1–2 days
• 3–4 days
• 5–6 days
• 7 days

Its categories cover every possible number of days and do not overlap. Someone who used the app on two days has exactly one suitable answer.

For continuous quantities, boundaries need particular care. For example, time categories $0 \leq t < 10$, $10 \leq t < 20$ and $t \geq 20$, with measured in minutes, place a time of exactly 10 minutes in the second category only. Include options such as ‘other’, ‘not applicable’ or ‘don't know’ where they are genuinely needed.

Sensitive questions may be left unanswered or answered dishonestly. Consider whether the information is necessary and whether anonymous collection would make respondents more comfortable.

Choose between open and closed questions

An open question allows an answer in the respondent's own words. ‘What would you change about the app?’ may reveal an unexpected problem and provide detailed reasons. However, answers vary, making them slower to group, compare and summarise.

A closed question supplies response options. These answers are usually quicker to record and count, but the options may restrict what respondents can express. An opinion scale might offer ‘very satisfied’, ‘satisfied’, ‘neither satisfied nor dissatisfied’, ‘dissatisfied’ and ‘very dissatisfied’. Such scales are convenient, although some respondents may tend to choose middle responses.

The choice depends on the aim: closed questions help count how common an answer is; open questions help explore explanations. A questionnaire can use both.

Compare questionnaires and interview techniques

An interviewer asks questions and records the responses, either face to face or by telephone. A self-completed questionnaire can be printed or online and may be anonymous.

Interviews often obtain more complete answers from people who take part, but the time involved can limit sample size. Questionnaires can reach a larger sample, yet a low response rate may leave some groups under-represented.

Interviewer bias occurs when an interviewer's expectations or manner affect the answers. Asking neutrally and using consistent wording helps reduce this influence. Anonymous self-completion removes the interviewer from the answering process, but does not remove bias caused by leading questions or an unsuitable choice of respondents.

Prepare a usable data collection sheet

A data collection sheet should have clear headings, units where needed, and enough space for each observation. Decide whether you need individual records or only category totals.

For direct observation of pedestrians' phone use, a tally sheet could have these headings:

Record the location, date and observation period alongside the table. Each pedestrian observed is tallied once in the appropriate row; the frequency is the final count for that row.

For an experiment with repeated measurements, keep each object's results together. Suppose a researcher tests eight dogs at home, asking each dog to perform the same ten commands on two visits, two days apart. A suitable sheet has one row for each dog, numbered 1–8, and columns headed Dog identifier, Commands completed on visit 1 (out of 10) and Commands completed on visit 2 (out of 10). This preserves the pairing between each dog's two results rather than mixing them together.

Select observation or an experiment when appropriate

Direct observation records what happens rather than relying on what people report. It may leave natural behaviour unaffected, but the observer has little control over the situation.

An experiment investigates how changes in one variable affect another. Other variables that could affect the result should be controlled where possible.

• Laboratory experiment: a controlled environment makes conditions easier to control and repeat, but behaviour may be less natural.
• Field experiment: takes place in the usual environment while the researcher controls aspects of the investigation. Behaviour may be more natural, but other influences are harder to control and conditions harder to repeat.
• Natural experiment: observes changes without the researcher controlling the situation or variables. It can reflect natural behaviour, but offers little control over other influences.

The dog investigation is a field experiment: the dogs are in their home environments, but the researcher controls the commands they are asked to perform. The best method depends on whether control, natural behaviour, time or cost matters most for the investigation.

Test the plan before full collection

A pilot survey gives a questionnaire to a small sample before the main survey. It can reveal confusing wording, missing response options, overlapping categories or a questionnaire that takes too long. Revise these problems before using the questionnaire more widely.

A pre-test runs an experiment on a small scale. It checks whether the procedure works and whether the measurements can be recorded as intended. Problems can then be corrected before committing time and resources to the full experiment.

Both are trials of the collection method, not substitutes for the main investigation. Their purpose is to discover preventable difficulties early.

• Use neutral, clear wording and a specific time period.
• Response options must cover all possibilities without overlap.
• Open: detailed, unrestricted answers; slower to summarise.
• Closed: supplied options; easier to count, but may restrict answers.

Collection methods

• Interview: clarification and more complete answers; costly in time, with possible interviewer bias or less honest sensitive answers.
• Anonymous questionnaire: cheap, wide reach and potentially greater honesty; lower response and no immediate clarification.
• Observation: records behaviour directly; limited control.
• Laboratory experiment: greater control and repeatability; less natural behaviour.
• Field experiment: usual environment with some researcher control; harder to control other influences.
• Natural experiment: no researcher control over the situation or variables; natural conditions but little control.

Recording and trials

• Collection sheets need clear categories or variable headings, units and space for all observations. Keep paired measurements together.
• Pilot survey: trial a questionnaire on a small sample, identify problems, then revise it.
• Pre-test: trial an experiment on a small scale, identify procedural or recording problems, then improve it.