165.] One remarkable example, wherein the minimum value of a function of many variables is to be discovered, deserves insertion. The problem occurs in the combination of observations, all of which are subject to, and are supposed to be affected with, accidental errors; and the object is to determine the most probable conclusion from the series of given results which are affected with these errors. The process, of which the following is an outline, is generally called the Method of Least Squares. Suppose that there are n unknown quantities x1, x2, ... X; and let u1, 2, ... um be m other quantities connected with them by m given equations, so that each of the latter is a given function of some or all of the former. Suppose also that the values of u1, u2, ... um are capable of being observed; from these observations the values of x1, X2, In are to be deduced. U2, ... ... Let the observed values of u1, u2, Um be 01, 02, observations then give the m equations, U1—01 = 0, U2 — 02 = 0, ... Um—0m = for the determination of the n unknown quantities. If m is less than n, these equations are insufficient. If m = n they are generally sufficient, and the solution of the problem is determinate and unique. But if, as is usually the case in practice, m is greater than n, the equations are more than sufficient. Still, if the observations were absolutely accurate, the equations would not be inconsistent, and every sufficient combination of them would give the same values for the unknown quantities. As however the observations are actually liable to error, the equations (44) will in general be inconsistent, and no one set of values of x1, x2, ... x, can satisfy them all at once. The question is, What set of values are we to adopt? At the outset it may be observed, that the simplest way of expressing that all the equations (44) subsist at once, would be by the single equation, (U1-01)2 + (U2-02)2 + ... + (Um―0m)2 = 0. (45) In the actual case it is impossible to satisfy this equation; but the idea obviously suggests itself of satisfying it as nearly as possible, by choosing the unknown quantities so as to make the expression in the left-hand member of the equation as small as possible. The question whether this plan really gives the most probable values of the unknown quantities belongs to the Theory of Pro babilities, and it would be out of place to discuss it here. The same may be said of the modifications to be introduced when the observations are not all equally liable to error; and of the method of estimating the precision of the results. It may be observed however, that an obvious way of giving greater influence to the better observations is, to multiply each term in the left-hand member of (45) by a positive number representing the goodness or, as it is called, the weight of the corresponding observation; and this is in fact the method indicated by theory; so that the function of which the minimum is to be determined is J1 (U1 — 01)2 + J2 (U2−02)2 + ... + Im (Um−0m)2 ; (46) where 91, 92, ... 9m are the weights of the several observations; and are proportional respectively to the number of times an observation, of arbitrary fixed liability to error, is to be repeated, in order that the arithmetical mean of its results may be entitled to the same degree of confidence as the single result of the observation in question. The estimation of these weights is the business of the observer; and for our purpose they are to be considered as given constants. ... Now if we can find the values of x1, x2, xn which make the expression (46) a minimum, we may substitute them in the functions U1, U2, um, and the results may be called the calculated values of these functions; and the differences between these calculated values and the observed values 01, 02, ... Om may be called the apparent errors of the observations: they would be the true errors if the calculated values u1, U2, Um were absolutely correct. Putting E1, E2, ... Em for these apparent errors, we have u1-01 = E1, Um―0m = Em, and the expression (46) becomes ... 2 2 ... ... and this is to be a minimum. In the case in which the weights of the observations are equal, representing their common value by unity, we have simply E2+ Eq2+ +Em2, which is to be a minimum; and thus in this case the method consists in determining the unknown quantities so that the sum of the squares of the apparent errors of the observations may be a minimum. Let us symbolize (46) by ; then as x1, x2, pendent variables, and as n is to be a minimum, ... Xn are inde ... and from these n equations the values of the n unknown quantities x1, x2, an are to be found. The algebraical solution of these equations is in general impracticable, unless the functions U1, U2, ... Um are all linear; but as the problems which occur in practice may be reduced to this form, the difficulty does not actually arise. This simplification is effected as follows: A set of approximate values of the unknown quantities is first obtained in any way that is practicable, or is previously known. Let these be called 1, 2, ... En; and let their unknown errors be dέ1, dɛ2, ... den, so that the true values are §1+d§1, §2+d§2, ... En+den; and let d1, d2, ... dgn be treated as small quantities, of which the squares, higher powers, and products are to be neglected. Suppose the equation expressing u1 in terms of Xn to be X1, X2, ... then we have U1 = (X1, X2, ... Xn); u1 = $(§1 + d§1, §2 + d§2, ... En + dεn) ; and expanding this by (56) of Art. 142, and omitting all terms involving higher powers than the first of the errors, we have quantities, so that u is reduced to a linear function of the new unknown quantities d1, d2, ... den. A similar reduction may be effected with us, us, ... Um; so that finally the equations which are given by the observations, are all reduced to the linear form. If then, as heretofore, we substitute x1, x2, ... x, for d§1, d2, ..... dɛn, and a, a1, az, .k, for the constants, we may assume ... ... Here however we may remark that if each of the equations u1-01 = 0, Um―0m = 0, is multiplied by the square root of the weight of the observation by which it was obtained, and if u1, 01, ... are then written instead of {g}, {g}* 01, and a, a1, instead of {g}a, {g} a, ... the problem will be reduced to the form in which g1 = 92 ... = ... is the only case which need be considered. ... 9m 1, so that this Let u2 be the sum of the squares of the errors thus modified, which is to be a minimum; then we have u Du = 0) = E1 dE1 + E2 dE2 + + Em dEm; ... and replacing dE1, dE2,... dem by their values from (47) and (48), and arranging the terms as coefficients of dæ1, dx2, ... dxn, we have 0 = (α1 E1+b1 E2+ ... + k1 E...) dx1 ... +(α E1+b2 E2+ ... + k2 Em) dxg + +(an E1+b1 E2 + ... + kn Em) dxn; and as x1, x2, x are assumed to be independent of each other, the coefficients of each of the differentials in the right-hand member of the equation is equal to zero; so that replacing E1, E2,... Em by their values given in (47) and (48) we have (a12 + b2 + ... + k12) x1 + (α1 α2 + b1b2+ ... + k1 k2) x2 + ... X2 +a1(a—o1) + b1 (b −02) + ... + k1 (k−0m) = 0, 2 (ɑ2α1+b2b1 ... + (az a + b2bn +...+k2 kn) Xn + a2 (a−01) + b2 (b −02) + ... + k2 (k—0m) = 0, (a, a1 + b„b1 + ... + k„k1) x1 + (α, a2 + bn b2 + ... + kn k2) X2 + ... ... + (a,2 + b2 + ... + km2) xn n +a, (a-o1) + b1 (b−02) + ... + kn (kom) = 0; whereby we have n linear equations containing ʼn unknown quantities; these may therefore be determined; and the values of them thus found will, according to the method of Least Squares, be affected with the least possible risk of errors. Now without going farther into the subject, and without introducing the convenient symbols which Gauss, to whom we are in great measure indebted for the method, has introduced, I may remark that the practical rules for forming the final simultaneous equations, as appears from the preceding system of equations, are the following. Multiply each equation by the coefficient of x1 in itself, then the sum of all thus multiplied is the first final equation. Again, multiply each by the coefficient of x2, and the sum of all thus multiplied is the second final equation: and so on for all the equations. It is evident that we thus obtain n final equations from which the n unknown quantities may be determined. Two examples are subjoined for the purpose of illustrating the process. 166.] Examples of the method of least squares. Ex. 1. Let there be four linear equations involving three unknown quantities, and of the following form: and let us suppose that observations are made, and that by them u1, uz, uз, u are found severally to be 3, 5, 21, 14; then the errors of the several equations will be u1-3, Uz-5, Uз-21, u-14: and if u2 is the sum of their squares we have u2 = (u1-3)2+(U2-5)2 + (U3-21)2+(Us-14)2; and as u2 is to be a minimum, UDU=0= (u1-3) du1+ (u2-5) du2+ (u3-21) dus+ (us-14) dus; in which substituting from the preceding equations, and equating to zero the coefficients of dx, dy, and dz, we have 2u1-5uz+4u3+3u4-107 = 0 and substituting in terms of x, y, z, we have whence a 2.470, y = 3.551, z= 1.916; and these are the most probable values of the variables. |