MDM Reference • The Gaussian family
The Gaussian Family of Densities
No family of distributions is used more widely, or carries more weight in the theory of inference, than the Gaussian. Its shape is familiar, but the reasons for its central place are algebraic and deserve to be stated.
Definition and standardisation
The density of a Gaussian variable is a symmetric bell whose centre is the mean and whose width is fixed by the standard deviation. By subtracting the mean and dividing by the standard deviation, every member of the family is carried to a single standard member. Standardised values are therefore comparable across variables that share the family but differ in scale, and tables of a single curve serve the whole family.
Why addition favours the bell
When many small independent contributions are added, the sum tends toward a Gaussian shape whatever the shapes of the contributions, provided their variances are finite and no single term dominates. This central tendency is the reason measurement noise, aggregate scores and combined perturbations so often settle into the bell. The result is a statement about the limit, and it says nothing about any particular finite sample beyond the promise of an estimate.
Geometry of the curve
The curve turns at one standard deviation from the centre, and those inflection points mark where the density changes from concave to convex. Roughly two thirds of the density lies within one standard deviation and about nineteen twentieths within two. These proportions, fixed for every member of the family, make the standardised scale an efficient summary of spread.
Members derived from the bell
Squares of Gaussian variables follow the chi-square family; ratios of such squares follow the F family; and a Gaussian divided by the square root of an independent chi-square follows the Student family. These derived members carry the heavy tails that the bell itself lacks, and they are the ones used when the variance of a population is estimated rather than known.
The multivariate extension
Several Gaussian variables taken together form a multivariate Gaussian, fully described by a vector of means and a square arrangement of covariances. The conditional distributions within such a joint are again Gaussian, and this closure under conditioning is the property that makes the family so tractable in modelling. The next cell treats the density matrix that records those covariances.
Caution
The bell is a convenient ideal, not a law of nature. Data that are skewed, bounded, or heavy in the tails will not be well served by a Gaussian account, and the honest response is to choose a family that matches what is seen. The centrality of the bell in this reference reflects its role in the theory, not a claim that it describes every observable.
A reading of the tails
The bell assigns a very small weight to readings far from the centre, and it is this lightness of the tails that makes it a poor fit for records with occasional extreme values. A record that is mostly well-behaved but punctuated by rare, large departures is better served by a family with heavier tails, of which several are built from the bell itself. The reader should look at the tails before adopting a Gaussian account.