Measurement, Uncertainty & Significant Figures · Guide · glidepathscience.org
GuideMeasurement, Uncertainty & Significant Figures
Contents
This Guide explains the ideas and equations behind every Walkthrough step, Concept Challenge question and Predict round. Part 1 prepares the Intro track and Part 2 the Advanced track. Each section works an example that is not one of the sim’s own problems.
Part 1 · The essentials
Math you’ll need: arithmetic with decimals and percents, powers and square roots, and rearranging a simple formula. One optional box uses a derivative.
A measurement is a range
Ask two careful people to measure the same table and they will rarely write down the same number. One lines the tape up a little differently, one judges the table’s edge a little further along, and the last digit comes out different. Neither has made a mistake. Every measurement leaves some doubt in its last digit, and an honest result says how much: it gives a best value and a range around it,
$x = \bar{x} ± δx$
read as “the quantity is probably within $δx$ of $\bar{x}$.” The number $δx$ is the uncertainty. It is not a mistake to be fixed; it is part of the result, as important as the value itself, because it tells the reader how far to trust the value and whether it agrees with someone else’s. A length of 1.524 m with no uncertainty attached cannot be compared with anything.
Two words are worth keeping apart from the start. A measurement is precise when repeated readings cluster tightly, and accurate when they cluster around the true value. A ruler with a worn end can give beautifully precise readings that are all wrong by the same amount; a hurried timer can scatter widely around the right answer. The sections that follow deal with both.
Where does the doubt come from? First from the instrument itself, so we begin with reading a scale.
Reading a scale
The smallest division of a scale is its resolution. A meter stick marked in millimeters has a resolution of 1 mm. You can often judge where an edge falls between two marks to about a tenth of a division, but how good that judgment is depends on the edge, the light and your eye. A safe and widely used default is to take the reading uncertainty as half the smallest division: ±0.5 mm on a millimeter scale. This sim uses that rule, half the resolution, for every instrument.
Two habits keep a reading honest. Look straight at the mark: seen from the side, a mark that is not touching the object seems to shift, an error called parallax. And check the zero. If a ruler’s end is worn, or a caliper reads 0.10 mm with its jaws closed, every reading is shifted by the same amount. That is a zero error, a kind of systematic error: repeating the measurement cannot remove it, but subtracting it can. With a worn ruler, start the object at a clear mark, say 10 mm, and subtract that mark from the far reading.
A vernier caliper reads finer than its main scale by adding a sliding scale. On the one in this sim, ten vernier lines span 9 mm, so each falls 0.1 mm short of a millimeter mark. Read the main scale just before the vernier’s zero; then find the single vernier line that lines up exactly with a main-scale mark. If it is line $n$, add $n$ × 0.1 mm. The eye is poor at judging a tenth of a millimeter directly but very good at seeing which pair of lines meets, and that is the trick.
reading = main scale before the zero + (matching line) × least count
A micrometer goes a step further. Its spindle advances 0.5 mm per turn, and the thimble around it carries 50 divisions, so each division is 0.01 mm. The sleeve shows whole millimeters above a line and half millimeters below it; the thimble division on the line adds the hundredths. Forgetting a half-millimeter mark that is showing is the classic slip.
Timing by hand
A stopwatch that shows hundredths of a second has a resolution of 0.01 s, but that is rarely what limits a hand-timed measurement. You press the button a little late at the start and a little late at the stop, and the two delays never quite match. A simple reaction to a surprise signal takes about 0.2 s; when you watch a pendulum and anticipate its swing, the mismatch between start and stop is typically a few hundredths to a tenth of a second. Either way it is far larger than the display’s ±0.005 s. Resolution is not the same as uncertainty.
The cure is to time many repetitions together. The reaction errors happen only at the start and the end, so they are shared among all the swings: timing 20 swings and dividing by 20 divides the timing uncertainty by 20.
Repeated readings, absolute and relative uncertainty
Repeat a measurement and the readings scatter. Their mean is a better estimate than any one of them, because random errors push some readings up and some down and partly cancel when you average:
$\bar{x} = (x_1 + x_2 + … + x_N)/N$
How far does a single reading stray? A quick measure that needs no statistics is the spread, half the range: $(x_{max} − x_{min})/2$. Report the result as the mean ± the spread, but never as less than the reading uncertainty of the instrument. (The Advanced track replaces the spread with the standard deviation, and shows that the mean itself is better known than any single reading.)
The ± is the absolute uncertainty, in the units of the quantity. Divided by the value it becomes the relative uncertainty, a pure number usually given as a percent:
relative uncertainty $= δx/x$
The relative uncertainty is what tells you how good a measurement is regardless of its units: ±1 mm is superb for the length of a room and poor for the thickness of a coin. It is also what carries through multiplication and division, as the next section shows.
Carrying uncertainty through a calculation
Most results are calculated from several measurements, and each brings its uncertainty along. The simplest rules ask what happens if every error is at its worst and pushes the result the same way. They give an upper bound, and they need no statistics.
Sums and differences: absolute uncertainties add. If $q = a + b$ or $q = a − b$, then
$δq = δa + δb$
Uncertainties add even when the values subtract. A length found as the difference of two positions carries the doubt of both ends, and a small difference of two large, close numbers can be mostly uncertainty.
Products and quotients: relative uncertainties add. If $q = ab$ or $q = a/b$, then
$δq/q = δa/a + δb/b$
Seconds cannot be added to meters, but percents can: find each input’s relative uncertainty, add them, and multiply by the result to get its absolute uncertainty.
Powers: the relative uncertainty is multiplied by the power. If $q = x^n$, then $δq/q = |n|\,δx/x$. A squared quantity counts twice and a square root counts half. In $g = 4π^2L/T^2$, the period’s relative uncertainty counts twice, which is why a pendulum experiment is won or lost on its timing.
The rules give a result’s uncertainty properly. Significant figures are a shortcut that tries to do the same with no arithmetic at all.
Significant figures
The significant figures of a number are the digits that carry information about the measurement: every nonzero digit, zeros between nonzero digits, and zeros at the end of a number that has a decimal point. Leading zeros only place the decimal point, so 0.004050 has four significant figures. Scientific notation removes the doubt about trailing zeros: 3.20 × 10³ has three, while 3200 on its own is ambiguous (this sim counts its trailing zeros as not significant).
The usual rules are a shorthand for propagation. Writing a number with $n$ significant figures promises an uncertainty of about half a unit in its last place. In a sum, the answer keeps the decimal places of the least precise term; in a product or quotient, it keeps as many significant figures as the input with the fewest. These mimic the rules of the previous section roughly, and that is their weakness.
Sig figs can mislead in both directions. The rule for a product throws away the size of the leading digit, so it can claim too much or too little. And rounding in the middle of a calculation adds a fresh error each time. When it matters, propagate the uncertainty, round it, and round the value to the same decimal place. This sim rounds an uncertainty to one significant figure, or two when its first digit is 1, and keeps every digit until the end.
Do two results agree?
Two results for the same quantity are never exactly equal. The question is whether their difference is larger than their uncertainties can explain. The uncertainty of a difference is the sum of the two uncertainties (the rule for differences), so two results agree when
$|a − b| ≤ δa + δb$
which is the same as saying their ranges overlap. If they do not overlap, at least one of the results has an error that its uncertainty does not cover: a systematic error, a mistake, or an uncertainty estimated too small. Agreement needs no true value: two results can be compared with each other directly. An accepted value from a handbook is usually treated as exact, so a measurement agrees with it when the gap is within the measurement’s own uncertainty. A result given with no uncertainty cannot be judged at all.
Part 2 · Going further
Math you’ll need: everything in Part 1, plus partial derivatives, square roots of sums of squares, and the idea of a probability distribution.
Standard uncertainty: Type A and Type B
The international Guide to the Expression of Uncertainty in Measurement (the GUM) gives the field its vocabulary. The quantity being measured is the measurand. Its true value is an ideal: the error of a reading, its difference from the true value, can never be known exactly (if it could, it would be corrected). What can be evaluated is the standard uncertainty $u$, a standard deviation that describes the spread of values that could reasonably be attributed to the measurand.
The GUM evaluates a standard uncertainty in two ways. Type A uses the statistics of repeated readings. Type B uses anything else: a calibration certificate, a manufacturer’s specification, or a display’s resolution. The labels describe how the uncertainty is evaluated, not whether the error is random or systematic.
A display that rounds to a step $Δ$ leaves the true value anywhere within $±Δ/2$ of what it shows, with no reason to prefer one place over another. A uniform spread of half-width $a$ has standard deviation $a/\sqrt3$, so
$u = Δ/\sqrt{12}$
Part 1’s “half the resolution” is the half-width $a$ itself, a looser bound; the standard uncertainty is about 58 % of it.
The standard error, the Gaussian and the central limit theorem
For $N$ repeated readings, the sample standard deviation measures how far a single reading strays from the mean:
$s = \sqrt{\sum(x_i − \bar{x})^2/(N − 1)}$
(The $N − 1$ accounts for the mean having been estimated from the same readings.) The mean itself strays much less. Its standard deviation, the standard error of the mean, is
$s_{\bar{x}} = s/\sqrt{N}$
Taking more readings never shrinks $s$, the scatter of single readings; it shrinks $s/\sqrt{N}$, slowly. Halving the standard error takes four times the readings, and a tenfold improvement takes a hundred times. Beyond some point the Type B part (a resolution, a calibration) dominates and more readings stop helping.
Why the Gaussian (normal) distribution? Most random error is the sum of many small independent effects. The central limit theorem says that such a sum tends to a Gaussian whatever the shapes of its parts, and the variances of independent parts add. For a Gaussian, the share of readings within $±kσ$ of the mean is $\operatorname{erf}(k/\sqrt2)$: 68.3 % within 1σ, 95.4 % within 2σ and 99.7 % within 3σ. So a ±1σ error bar on a Gaussian result should miss the true value about a third of the time; error bars are statements about probability, not guarantees. Those shares belong to the Gaussian. A standard uncertainty is a scale, a standard deviation; a coverage interval (“95 %”) needs the distribution too: ±1.96σ for a Gaussian, ±1.65σ for a uniform spread, and wider (a Student t factor) when σ itself comes from a few readings.
outside $±kσ$: $1 − \operatorname{erf}(k/\sqrt2)$
Propagation in quadrature, and its limits
Part 1’s simple rule assumes every error is at its worst together. For independent random errors that almost never happens: some push up while others push down. Expanding $f$ to first order and using the fact that the variances of independent quantities add gives the general rule. With sensitivity coefficients $c_i = ∂f/∂x_i$ evaluated at the best values,
$u_f^2 = \sum_i \left(∂f/∂x_i\right)^2u_i^2$
For a product or a power this becomes relative uncertainties added in quadrature: for $g = 4π^2L/T^2$, $u_g/g = \sqrt{(u_L/L)^2 + (2u_T/T)^2}$. Squaring makes the largest contribution dominate, so the first improvement to make is always to the input with the largest share.
Correlated inputs. When two inputs share an error, such as one calibration that affects both readings, the variances gain a covariance term,
$u_f^2 = \sum c_i^2u_i^2 + 2\sum c_ic_jr_{ij}u_iu_j$
with $r_{ij}$ the correlation coefficient. In a difference ($c_1 = −1$, $c_2 = +1$) a perfectly shared error cancels; in a sum it adds directly. That is why differences are best measured with one instrument.
When first order fails. The rule uses only the straight-line part of $f$. If an input’s relative uncertainty is large and $f$ curves, the mean of $f$ is no longer $f$ of the mean, and the distribution of the result is skewed. For $f = x^2$ with a Gaussian $x$, the mean is $\bar{x}^2 + σ^2$. A Monte Carlo propagation avoids the approximation altogether: draw every input from its distribution many times, compute $f$ each time, and read the mean, the standard deviation and a 95 % interval straight from the results.
Combining results: the weighted mean
Several measurements of one quantity, of unequal quality, should not count equally. Weighting each by $1/u_i^2$ gives the combination with the smallest uncertainty:
$\bar{x}_w = \sum(x_i/u_i^2)/\sum(1/u_i^2)$, $u_w = \left(\sum 1/u_i^2\right)^{−1/2}$
The weighted mean lies between its inputs, nearer the precise ones, and its uncertainty is smaller than the smallest input’s. Combine only results that agree: compute $χ^2 = \sum((x_i − \bar{x}_w)/u_i)^2$ first, which should be no more than about the number of results minus one.
Fitting a line: the slope’s uncertainty and χ²
A straight-line fit by least squares chooses the line $y = a + bx$ that minimizes
$χ^2 = \sum\left((y_i − a − bx_i)/σ_i\right)^2$
When every point has the same error bar $σ$, the slope’s standard uncertainty is
$σ_b = σ/\sqrt{\sum(x_i − \bar{x})^2}$
It depends on the error bars and on how widely the $x$ values are spread, not on the scatter of these particular points, which is why spreading the measurements out improves a slope. After the fit, $χ^2$ tests the error bars. With $N$ points and two fitted parameters there are $ν = N − 2$ degrees of freedom, and $χ^2/ν$ should be close to 1. Much larger means the points scatter more than their error bars allow: the bars are too small, or the model is wrong. Much smaller means the bars are too large.
$χ^2/ν$, $ν = N − 2$
Systematic error and calibration
Random errors scatter readings both ways and shrink in the mean. A systematic error pushes every reading the same way, and no number of readings reveals or removes it: the readings can be beautifully precise and still wrong. Systematic errors are found by calibration, measuring a standard whose value is known much better than the instrument’s, and are removed by correcting for the offset found. The correction carries its own uncertainty, which stays in the result and adds in quadrature to the random part.
corrected = reading − offset, $u = \sqrt{u_{reading}^2 + u_{offset}^2}$
Reference
Common mistakes
Part 1
- Taking a display’s resolution as the uncertainty of a hand-timed interval: reaction time dominates.
- Reading the main-scale mark where the vernier lines meet, instead of the main scale before the vernier’s zero.
- Averaging away a zero error: it shifts every reading the same way; subtract it.
- Subtracting uncertainties when values subtract: for sums and differences, they add.
- Adding absolute uncertainties of a product: for products and quotients, relative ones add.
- Forgetting the power: a squared quantity counts its relative uncertainty twice.
- Rounding in the middle of a calculation, or trusting sig figs where the uncertainty is known.
- Calling two results in agreement because both are “close”: compare the gap with the uncertainties.
Part 2
- Taking half the resolution as a standard uncertainty: the uniform spread gives $Δ/\sqrt{12}$.
- Using the standard deviation of single readings as the uncertainty of their mean: divide by $\sqrt{N}$.
- Adding independent standard uncertainties directly instead of in quadrature.
- Treating a shared calibration error as independent at both ends of a difference.
- Trusting first-order propagation when an input’s relative uncertainty is large and $f$ curves.
- Dividing χ² by $N$ instead of the degrees of freedom $N − 2$.
The equation sheet
| Part 1 | |
|---|---|
| result | $x = \bar{x} ± δx$ |
| mean | $\bar{x} = \sum x_i/N$ |
| relative | $δx/x$ |
| sums | $q = a ± b$: $δq = δa + δb$ |
| products | $q = ab$ or $a/b$: $δq/q = δa/a + δb/b$ |
| powers | $q = x^n$: $δq/q = |n|\,δx/x$ |
| agreement | agree if $|a − b| ≤ δa + δb$ |
| Part 2 | |
|---|---|
| Type B | $u = Δ/\sqrt{12}$ |
| standard error | $s_{\bar{x}} = s/\sqrt{N}$ |
| Gaussian | outside $±kσ$: $1 − \operatorname{erf}(k/\sqrt2)$ |
| propagation | $u_f^2 = \sum c_i^2u_i^2 + 2\sum c_ic_jr_{ij}u_iu_j$ |
| weighted mean | $\bar{x}_w = \sum(x_i/u_i^2)/\sum(1/u_i^2)$ |
| slope | $σ_b = σ/\sqrt{\sum(x_i − \bar{x})^2}$ |
| goodness of fit | $χ^2/ν$, $ν = N − 2$ |
$\bar{x}$ mean · $δx$ uncertainty (Part 1) · $u$ standard uncertainty · $Δ$ resolution · $s$ sample standard deviation · $N$ number of readings · $c_i = ∂f/∂x_i$ · $r_{ij}$ correlation · $σ$ error bar · $ν$ degrees of freedom.
Sources and further reading
The vocabulary and the Type A and Type B evaluations follow the GUM (JCGM 100:2008) and the International Vocabulary of Metrology (JCGM 200:2012); Monte Carlo propagation follows JCGM 101:2008; NIST Technical Note 1297 is a short practical summary. For a fuller treatment, J. R. Taylor’s An Introduction to Error Analysis and Bevington and Robinson’s Data Reduction and Error Analysis for the Physical Sciences. Open introductory physics textbooks, OpenStax’s among them, open with a chapter on units and measurement that covers significant figures. The Field Manual (H) lists the model’s values and conventions.
Measurement, Uncertainty & Significant Figures v1.0 · © 2026 Dr. David Meier · CC BY-NC-SA 4.0 · glidepathscience.org