165.] One remarkable example, wherein the minimum value of a function of many variables is to be discovered, deserves insertion. The problem occurs in the combination of observations, all of which are subject to, and are supposed to be affected with, accidental errors; and the object is to determine the most probable conclusion from the series of given results which are affected with these errors. The process, of which the following is an outline, is generally called the Method of Least Squares. Suppose that there are n unknown quantities and let uu v^,,... um be m other quantities connected with them by m given equations, so that each of the latter is a given function of some or all of the former. Suppose also that the values of uu u^,... um are capable of being observed; from these observations the values of x\, xi} ...xn are to be deduced. Let the observed values of ux, «2,... um be oj, o2, ...o„; the observations then give the m equations, «i-Oi = 0, u2 — o2 — 0,...um — om = 0, (44) for the determination of the n unknown quantities. If m is less than n, these equations are insufficient. If m = n they are generally sufficient, and the solution of the problem is determinate and unique. But if, as is usually the case in practice, m is greater than n, the equations are more than sufficient. Still, if the observations were absolutely accurate, the equations would not be inconsistent, and every sufficient combination of them would give the same values for the unknown quantities. As however the observations are actually liable to error, the equations (44) will in general be inconsistent, and no one set of values of X\, x2, ... xn can satisfy them all at once. The question is, What set of values are we to adopt? At the outset it may be observed, that the simplest way of expressing that all the equations (44) subsist at once, would be by the single equation, (wi-o1)2 + (M2-o2)2+ ... +(Km-Om)* = 0. (45) In the actual case it is impossible to satisfy this equation; but the idea obviously suggests itself of satisfying it as nearly as possible, by choosing the unknown quantities so as to make the expression in the left-hand member of the equation as small as possible. The question whether this plan really gives the most probable values of the unknown quantities belongs to the Theory of Probabilities, and it would be out of place to discuss it bere. The same may be said of the modifications to be introduced when the observations are not all equally liable to error; and of the method of estimating the precision of the results. It may be observed however, that an obvious way of giving greater influence to the better observations is, to multiply each term in the left-hand member of (45) by a positive number representing the goodness or, as it is called, the weight of the corresponding observation; and this is in fact the method indicated by theory; so that the function of which the minimum is to be determined is ffi («i-Oi)t + g»(ut-o2)2+ ... + gm («„,-om)»; (46) where gu gt, ...gm are the weights of the several observations; and are proportional respectively to the number of times an observation, of arbitrary fixed liability to error, is to be repeated, in order that the arithmetical mean of its results may be entitled to the same degree of confidence as the single result of the observation in question. The estimation of these weights is the business of the observer; and for our purpose they are to be considered as given constants. Now if we can find the values of which make the expression (46) a minimum, we may substitute them in the functions uu v^,... um, and the results may be called the calculated values of these functions; and the differences between these calculated values and the observed values 0\, Oj>,...om may be called the apparent errors of the observations: they would be the true errors if the calculated values Wi, ... um were absolutely correct. Putting Bj, E2, ... Em for these apparent errors, we have «i — Oi = Et,... Um—om = Em, and the expression (46) becomes « , „ , , and this is to be a minimum. In the case in which the weights of the observations are equal, representing their common value by unity, we have simply E!2 + E22 + ... + Em2, which is to be a minimum; and thus in this case the method consists in determining the unknown quantities so that the sum of the squares of the apparent errors of the observations may be a minimum. Let us symbolize (46) by n; then as xit xt,... xn are independent variables, and as n is to be a minimum, (£)-«- (£) and from these n equations the values of the n unknown quantities x\, x2,... xn are to be found. The algebraical solution of these equations is in general impracticable, unless the functions «1} Ui,... wm are all linear; but as the problems which occur in practice may be reduced to this form, the difficulty does not actually arise. This simplification is effected as follows: A set of approximate values of the unknown quantities is first obtained in any way that is practicable, or is previously known. Let these be called f2,... f„; and let their unknown errors be d(u d(2,... d(„, so that the true values are + fa + rffs, ...£„ + d£n; and let d£u rff2,... df„ be treated as small quantities, of which the squares, higher powers, and products are to be neglected. Suppose the equation expressing «i in terms of X\, xit... xK to be «, = <p(xi,Xi, ...xn); then we have «i = </>(fi + rffi,fi + rffi, ... f* + d(u); and expanding this by (56) of Art. 142, and omitting all terms involving higher powers than the first of the errors, we have «, = *<£u *. - f.) + (%) *6 + (^|) rf6 + ... + (g) d(n; . in this equation <f> (fh ••• f„), (jf)' ••• (^") are ^ giveu quantities, so that % is reduced to a linear function of the new unknown quantities dfa, d£2* ■■■ d£n. A similar reduction may be effected with u^, ... um; so that finally the equations «i—d = 0, u-i — o-, — 0, ... um—om - 0, which are given by the observations, are all reduced to the linear form. If then, as heretofore, we substitute xu x2,... xH for dfo, d(2, ... d{„, and a, au a2,... k„ for the constants, we may assume Hi = a 4 flj^H a2x2-\ ... +anxHl ''2 = *4 f>i.r1 + bix2+ ... +b„xn I (rt„ai4-*„*i4-... 4 kl,ki)xi-t(ana2 + bnb2+ ... 4 knk2)x2 4-... ■■■ + (a2 + bn2+...+kn2)xn 4-«„(a-o,)4 b„(b-o2) + ... 4- kn(k-om) = 0; whereby we have n linear equations containing n unknown quantities; these may therefore be determined; and the values of them thus found will, according to the method of Least Squares, be affected with the least possible risk of errors. Now without going farther into the subject, and without introducing the convenient symbols which Gauss, to whom we are in great measure indebted for the method, has introduced, I may remark that the practical rules for forming the final simultaneous equations, as appears from the preceding system of equations, are the following. Multiply each equation by the coefficient of X\ in itself, then the sum of all thus multiplied is the first final equation. Again, multiply each by the coefficient of a?., and the sum of all thus multiplied is the second final equation: and so on for all the equations. It is evident that we thus obtain n final equations from which the n unknown quantities may be determined. Two examples are subjoined for the purpose of illustrating the process. 166.] Examples of the method of least squares. Ex. 1. Let there be four linear equations involving three unknown quantities, and of the following form: «i = x—y+2z, and let us suppose that observations are made, and that by them «i, iii, Ms, ut are found severally to be 3, 5, 21, 14; then the errors of the several equations will be u1—3, u^ — 5, u$—21, w4—14: and if M2 is the sum of their squares we have and as u2 is to be a minimum, u D « = 0 = («! - 3) rfwj + (w.2 - 5) dui + (us—21) £?Ms + (u4 -14) du4; in which substituting from the preceding equations, and equating to zero the coefficients of dx, dy, and dz, we have «i + 3w2 + 4«3— 88 = 0 -i — Mi + 2m-3+ W3 + 3tt4— 70 = 0 L; and substituting in terms of x, y, z, we have whence x = 2.470, y = 3.551, z = 1.916; and these are the most probable values of the variables. |