Evaluating Data and Drawing Conclusions

What makes data convincing?

A clear pattern is only the beginning of a scientific conclusion. Before deciding what results mean, we need to ask how well the measurements were made, whether the pattern can be obtained again, and whether errors could explain it.

Consider an investigation into light intensity at different distances from a lamp. A prediction is that light intensity decreases as distance increases. Measurements might support this prediction, but a convincing evaluation must consider both the pattern and the quality of the evidence behind it.

Accuracy and precision answer different questions

Accuracy describes how close a result is to the true or accepted value. If a reference mass is 100 g and a balance reads 108 g, that reading is inaccurate.

Precision describes how closely repeated measurements agree. Suppose the balance gives readings of 108 g, 108 g and 109 g for that reference mass. These readings are close together, so they are precise, but they are not accurate. Consistency alone does not show that a measurement is correct.

On the diagram, distance from the centre represents error relative to the true value, while the spread of the marks represents precision. A tight cluster away from the centre shows why precise results can still be inaccurate.

To assess accuracy, compare a measurement with a suitable accepted value or check the instrument against a known standard. Small scatter around a line of best fit shows consistency with the pattern, but it does not establish accuracy: all readings could share the same bias.

Repeatability and reproducibility

Results are repeatable when the same investigator obtains closely agreeing measurements using the same method and equipment. Repeating measurements at each lamp distance allows this to be assessed. Small ranges between repeats, shown by short range bars on a graph, provide evidence of good repeatability.

Results are reproducible when different investigators, or different equipment or methods, produce agreeing results. For example, another group could investigate the same lamp distances using a different light meter. Agreement would strengthen confidence that the findings are not peculiar to one instrument or investigator.

These are different tests. One student taking many similar readings demonstrates repeatability, not necessarily reproducibility. Both can be good even when a shared flaw makes measurements inaccurate.

Random and systematic errors

A random error causes unpredictable variation between readings. In the lamp investigation, slight changes in the sensor's position or fluctuating background light could make some readings higher and others lower. This increases scatter and reduces precision.

Repeating measurements and calculating a mean reduces the influence of random variation on the final estimate. It does not remove the cause of that variation or guarantee that the estimate is accurate.

A systematic error shifts measurements consistently in a particular direction. For example, measuring every distance from the wrong reference point would give consistently incorrect distances. A light meter with an incorrect zero could consistently over-read.

Repeating the same flawed measurement does not correct a systematic error. The reference point must be corrected, or the instrument checked and adjusted. A useful evaluation therefore identifies a specific possible error rather than simply blaming ‘human error’.

Deciding whether an outlier should be kept

An outlier is a result that is unusually different from other readings or from the general pattern. It is a reason to investigate, not an automatic reason to delete data.

In a set of light-intensity measurements, the first three readings at 10 cm were 301.56, 305.53 and 300.65 lux. Elsewhere in the same row there was a reading of 134.13 lux. This much lower reading deserves investigation. However, the first three columns belonged to one investigator, so the whole row should not automatically be treated as repeats by that person under identical conditions.

Check the recording, instrument and experimental conditions, then repeat the measurement if possible. A sensor temporarily shaded by someone's hand would be a plausible explanation for an unusually low reading, but it should not be claimed as the cause without evidence.

Excluding a reading can be justified if there is evidence of a mistake or a condition that invalidated the measurement. Record the original result and explain why it was excluded. If no such evidence exists, retaining it and discussing its effect on the mean and uncertainty may be more appropriate. A genuine unexpected result can reveal a weakness in the hypothesis rather than a fault in the data.

Improvements should address a particular weakness

An improvement is useful when its benefit is explained. For the light investigation:

• Clamp the sensor and use a fixed distance reference point: this reduces positioning variation between repeats and improves precision and repeatability.
• Check the meter against a suitable reference and correct any zero error: this addresses instrumental bias and improves accuracy.
• Keep background lighting constant: this reduces variation unrelated to lamp distance, making the comparison more meaningful.
• Take repeats at every distance: this reveals scatter and possible outliers and reduces the influence of random variation on the mean.

If using a lamp, avoid touching a hot bulb and allow it to cool before adjusting it. Keep electrical leads away from walkways so they do not create a trip hazard.

Further investigation should answer a remaining question. More closely spaced distances could explore where the intensity changes most rapidly. Repeating the investigation with another lamp could test whether the same decreasing pattern occurs with a different light source. Neither is simply a substitute for correcting an existing measurement error.

Moving from observations to a reasoned conclusion

Data may appear as numbers, a graph, an apparatus diagram or symbols. Start by identifying what each value, axis, label or symbol represents, including its unit. An apparatus diagram can reveal a possible measurement problem, such as an unclear distance reference point; a graph can reveal a trend or unusually scattered readings.

In the light investigation, one investigator's first three readings were about 301–306 lux at 10 cm and about 186–190 lux at 40 cm. These numerical observations support the inference that light intensity decreases as distance increases. On a graph, the same relationship would appear as a downward trend. Several distances provide stronger evidence for a pattern than one pair of readings.

A reasoned conclusion links the claim to evidence and acknowledges limitations: ‘Light intensity decreased as distance increased over the distances tested. Closely agreeing repeats strengthen confidence in this trend, although checking the meter's calibration would be needed to assess measurement accuracy.’

How far does the evidence support the hypothesis?

Confidence increases when results match a prediction, repeats agree, other investigators obtain similar findings, and plausible measurement errors have been addressed. Confidence decreases when results repeatedly contradict the prediction, show substantial unexplained scatter, or depend on a flawed method.

Uncertain data may be inconclusive rather than evidence that a hypothesis is wrong. Equally, agreement with a prediction does not prove a hypothesis: the investigation covers only particular conditions, and another explanation may also fit the observations. State how strongly the evidence supports the claim, and limit the conclusion to what was actually tested.

Judge data quality

• Accuracy: closeness to the true or accepted value.
• Precision: closeness between repeated measurements.
• Repeatability: agreement using the same investigator, method and equipment.
• Reproducibility: agreement between different investigators, equipment or methods.