Summary some fifty years before the least sum of squared residuals fitting procedure was published in 1805, Boscovich (or Boškovic) proposed an alternative which minimises the (constrained) sum of the absolute residuals.

For , let represent the th observation on a set of variables and suppose that we wish to fit a linear model of the form

to these observations. Then, for , the -norm fitting procedure chooses values for to minimise the -norm of the residuals where, for , the th residual is defined by

The most familiar -norm fitting procedure, known as the least squares procedure, sets and chooses values for to minimise the sum of the squared residuals .

A second choice, to be discussed in the present article, sets and chooses to minimise the sum of the absolute residuals

A third choice sets and chooses to minimise the largest absolute residual .

Setting and if and and if , we find that so that the least absolute residuals ( ) fitting problem chooses to minimise the sum of the absolute residuals

for

and   for

The fitting problem thus takes the form of a linear programming problem and is often solved by means of a variant of the dual simplex procedure.

Gauss has noted (when ) that solutions of this problem are characterised by the presence of a set of zero residuals. Such solutions are robust to the presence of outlying observations. Indeed, they remain constant under variations in the other observations provided that these variations do not cause any of the residuals to change their signs.

The fitting procedure corresponds to the maximum likelihood estimator when the -disturbances follow a double exponential (Laplacian) distribution. This estimator is more robust to the presence of outlying observations than is the standard least squares estimator which maximises the likelihood function when the -disturbances are normal (Gaussian). Nevertheless, the estimator has an asymptotic normal distribution as it is a member of Huber's class of -estimators.

There are many variants of the basic procedure but the one of greatest historical interest is that proposed in 1760 by the Croatian Jesuit scientist Rugjer (or Rudjer) Josip Boškovic (1711-1787) (Latin: Rogerius Josephus Boscovich; Italian: Ruggiero Giuseppe Boscovich). In his variant of the standard procedure, there are two explanatory variables of which the first is constant and the values of and are constrained to satisfy the adding-up condition usually associated with the least squares procedure developed by Gauss in 1795 and published by Legendre in 1805. Computer algorithms implementing this variant of the procedure with variables are still to be found in the literature.

For an account of recent developments in this area, see the series of volumes edited by Dodge (1987, 1992, 1997, 2002). For a detailed history of the procedure, analysing the contributions of Boškovic, Laplace, Gauss, Edgeworth, Turner, Bowley and Rhodes, see Farebrother (1999). And, for a discussion of the geometrical and mechanical representation of the least squares and fitting procedures, see Farebrother (2002).

References

Yadolah Dodge (Ed.) (1987), Statistical Data Analysis Based on the -Norm and Related Methods, North-Holland Publishing Company, Amsterdam, The Netherlands.

Yadolah Dodge (Ed.) (1992), -Statistical Analysis and Related Methods, North-Holland Publishing Company, Amsterdam, The Netherlands.

Yadolah Dodge (Ed.) (1997), -Statistical Procedures and Related Topics, Institute of Mathematical Statistics, Hayward, California, USA.

Yadolah Dodge (Ed.) (2002), Statistical Data Analysis based on the -Norm and Related Methods, Birkhäuser Publishing, Basel, Switzerland.

Richard William Farebrother (1999), Fitting Linear Relationships: A History of the Calculus of Observations 1750-1900, Springer-Verlag, New York, USA.

Richard William Farebrother (2002), Visualizing Statistical Models and Concepts, Marcel Dekker, New York, USA.

Reprinted with permission from Lovric, Miodrag (2011), International Encyclopedia of Statistical Science. Heidelberg: Springer Science +Business Media, LLC

Richard William Farebrother