Mathematics for Chemists / Statistics and data

Sampling and hypothesis testing

Sample means, standard error, confidence intervals and t-tests for comparing chemical measurements.

Undergrad

In a nutshell

Suppose a laboratory measures the iron content of a tablet five times and gets an average slightly different from the value printed on the label. Is the tablet really different, or is the gap just the normal scatter of the measurements? Hypothesis testing gives a disciplined way to answer. We start by assuming there is no real difference (the null hypothesis), then ask how likely it would be to see a gap this large by chance alone. If that probability is very small, we reject the assumption. The tools are the standard error, which says how precisely a mean is known, confidence intervals, which give a range for the true value, and Student's t-test, designed for the small numbers of repeats typical in chemistry. These methods underpin quality control, method validation and forensic analysis.

Samples and populations

A set of n replicate measurements is a sample from an imagined infinite population with true mean \mu and standard deviation \sigma. We estimate them by the sample mean \bar x and sample standard deviation s (divisor n - 1). The mean itself varies from sample to sample, with standard error

\mathrm{SE} = \frac{s}{\sqrt n}.

By the central limit theorem, sample means are close to normally distributed even when individual measurements are not, provided n is not tiny.

Confidence intervals

Because s is only an estimate, small samples use Student's t distribution, which has heavier tails than the normal distribution. The confidence interval for \mu is

\mu = \bar x \pm t_{n-1}\frac{s}{\sqrt n},

where t_{n-1} depends on the confidence level and the degrees of freedom. Two-tailed 95% values are approximately 4.30 (n - 1 = 2), 2.78 (4), 2.26 (9) and 1.96 for very large samples. A 95% interval means that, if the procedure were repeated many times, 95% of the intervals so built would contain \mu.

Structure of a test

1. State a null hypothesis H_0 (no difference) and an alternative H_1. 2. Choose a significance level, commonly 5%. 3. Calculate a test statistic from the data. 4. Compare it with the critical value, or compute a p-value. If |t| > t_\text{crit} (or p < 0.05), reject H_0.

Rejecting H_0 when it is true is a type I error (probability equal to the significance level); failing to reject a false H_0 is a type II error. "Not significant" means the data are consistent with no difference, not that there is none.

Comparing a mean with a reference value

t = \frac{\bar x - \mu_0}{s/\sqrt n}, \qquad \nu = n - 1.

Worked example. A certified reference material contains 10.00 mg g^{-1} of an analyte. Five determinations give \bar x = 10.12 and s = 0.10 mg g^{-1}. Then SE = 0.10/\sqrt5 = 0.045 and t = 0.12/0.045 = 2.68. The critical value for four degrees of freedom at 95% is 2.78, so the difference is not significant at this level: there is no evidence of bias. With ten replicates and the same mean and s, t would be 3.79 against a critical value of 2.26, and the bias would be detected. More data increase the power of a test.

Comparing two means

For two methods with similar standard deviations, a pooled s_\text{p} gives

t = \frac{\bar x_1 - \bar x_2}{s_\text{p}\sqrt{1/n_1 + 1/n_2}}, \qquad \nu = n_1 + n_2 - 2.

An F-test (F = s_1^2/s_2^2) checks first whether the variances are comparable. If the same samples are measured by both methods, a paired t-test on the differences removes sample-to-sample variation and is far more sensitive.

Caveats

Suspected outliers should be judged by a formal test such as Grubbs', not discarded by eye. Repeated testing inflates false positives, a significant difference may be chemically trivial, and t-tests assume random, independent, roughly normal errors.

Key points

The standard error of the mean, s/\sqrt n, measures how precisely the mean is known.
Confidence interval: \bar x \pm t_{n-1}s/\sqrt n, using Student's t for small samples.
A t-test compares |t| with a critical value; if larger, reject the null hypothesis of no difference.
Not significant means no evidence of a difference, not proof that there is none.
Paired tests, F-tests and outlier tests suit different experimental designs.

Check yourself

5 questions

1Four replicates have s = 0.20. What is the standard error of the mean?
SE = s/\sqrt n = 0.20/2 = 0.10.
2Why is Student's t used instead of the normal distribution for small samples?
The t distribution widens the interval to allow for the uncertainty in estimating \sigma from few data.
3A t-test gives |t| = 1.9 with a critical value of 2.78. What should you conclude?
|t| is below the critical value, so the null hypothesis is not rejected; that is not proof of equality.
4The same eight samples are analysed by two methods. Which test is most appropriate to compare the methods?
Pairing removes the variation between samples, isolating the difference between methods.
5What is a type I error?
A type I error is a false positive; its probability equals the chosen significance level.

Recent preprints on this topic

Loading…

Matched by keyword from ChemRxiv via Crossref. Not peer-reviewed; read critically.