MDM Reference  •  Variance notes

Variance Notes and Working Remarks

These notes collect, in one place, the small facts about variance and dispersion that recur in ordinary work. They are remarks rather than a treatise, and they are meant to be consulted quickly.

Variance of a sum

The variance of a sum of two quantities is the sum of their variances plus twice their covariance. When the quantities are independent the covariance vanishes and the rule reduces to simple addition. This one relation answers most questions about how dispersions combine, and it is worth committing to memory.

Variance of a mean

The mean of n independent readings from one population has a variance equal to the population variance divided by n. The standard deviation of the mean therefore falls as the square root of n, which is the reason precision improves ever more slowly as a sample grows. Doubling the precision calls for four times the readings.

Standardisation

Subtracting the mean and dividing by the standard deviation carries any variable to a standardised form with mean zero and variance one. Standardised values are comparable across variables measured in different units, and this is the reason standardisation precedes so many multivariate summaries. The operation is reversible, so nothing is lost.

The pull of outliers

Because departures are squared, a single extreme reading can dominate the variance. A robust alternative, built from the ordering of the readings rather than from their squares, resists this pull. The reader should decide in advance which measure of width suits the question, since the two may tell different stories on the same data.

Pooled variance

When several groups share a common dispersion, their separate estimates may be pooled into one, weighted by the shares of the sample they hold. The pooled value is more precise than any single estimate, and it rests on the assumption that the dispersions really are common. When that assumption fails the pooling is misleading, and the separate estimates should stand.

Variance and information

A small variance in an estimator is a large amount of information about the parameter, and the least attainable variance sets the limit of what any regular rule can achieve. This limit is the practical reason variance is studied so closely, and it explains why a rule that reaches the limit is prized. The limit holds only under stated conditions, and those conditions should be checked before the claim is made.

A worked illustration in words

Suppose ten readings are drawn from one population, and a second set of ten is drawn afterwards. Each set yields a mean of its own, and the two means differ. The size of that difference is governed by the variance of a mean, not by the variance of a single reading, and the relation between the two is the division by the number of readings. This small example is the everyday face of the rule stated above.

When to prefer a range

A range is quick to read and easy to explain, and for a small, tidy set of readings it may say more than a variance. For a large or uneven set it says less, since it rests on two readings alone and ignores the rest. The choice between the two is a choice of what the reader needs to see, and neither is wrong in itself.