EMPIRICAL STUDIES IN THE THEORY OF

MEASUREMENT

IN the present condition of psychology, sociology and education, convenience, economy and directness are as important desiderata in methods of measurement as refinement with respect to precision. The results of these studies justify certain methods which have the decided advantage of giving measures which are direct functions of the data, independent of any hypothesis about the prevalence of the so-called 'normal' distribution, but which have been somewhat discountenanced or at least neglected in both the theory and the practice of statistics.

The section on correlation attempts also to make clear just what is measured by a coefficient of correlation and what the dangers are in the application of correlation formulas without constant supervision by an adequate sense for the concrete individual facts to be related.

MEASUREMENTS OF TYPE AND VARIABILITY

§1. The Comparative Accuracy of the Average and the Median

The median as a measure of the central tendency of a series of measures has the advantages of greater quickness of calculation, freedom from the influence of erroneous measurements, ease of interpretation and often greater practical significance. It is, therefore, important to know whether the accuracy, with which the median actually obtained from a small sampling of a series conforms to the true median of the total series, is much less than the similar accuracy in the case of the more commonly used measure, the average.

It is possible with any given form of distribution to calculate on the basis of the theory of probability the accuracy in either case. Trusting that some one will soon do this for typical forms of distribution other than the so-called 'normal' I have chosen to get empirical data on the same question from actual experiments with random samplings from certain large series of measures.

The median was calculated for each sampling by regarding the total series as measures of a continuous variable, quantity 61, for instance, equalling from 60.0 up to 62.0, quantity 63 equalling from 62.0 up to 64.0, etc. Where the median fell within a unit of the scale, as of course it usually did, the fractional part was taken

which would be correct, supposing the cases within that unit of the scale to be equally frequent in all equal subdivisions of that unit of scale.

The series used were the four presented in Table I. A is an almost perfect representative of the so-called 'normal' surface of frequency, limited at about +3.2 and 3.2o. B is also a symmetrical distribution following, but not so closely, the so-called 'normal' type. C is a skewed distribution of the kind so frequently found in mental and social measurements. D is a flattened and rather sharply cut-off type of distribution, such as occurs often in facts subject to conventional regulation. The number of cases was for A 1,000, for B 1,307, for C 1,250 and for D 600. The mechanical arrangement of each series was simply so many small cards or slips of paper each with a number written on it. In each series these cards were approximately of the same size, shape and weight. From such a series, properly shuffled in a large bowl, drawings were made.

The total number of cases in any series is of course of no significance. Whether a series contains 1,000, 1,100, 1,426, 13,982 or 160,000 cases makes no appreciable difference to any of the matters to be investigated here, and in the case of a distribution of the type of D, drawings of 100 from 6,000 cases would not differ appreciably from drawings from 600. The reason for the particular sizes of the total series was economy of time.

It is most convenient to arrange series for such experiments with measures and from the central tendency, as in B and D; the time of recording the results of draws is lessened and also the likelihood of errors. Thus in A-31, — 37, - 35, etc., would be better than 61, 63, 65, etc. I give the series, however, in just the way they were made and used.

Every drawing of 10 or 50 or 55 or whatever number of cases was made from the full series. However, a draw of 10 having been made and recorded, a draw of 50 was obtained by adding 40 to the 10 and one of 100 by adding 50 to the 50. The 100 is thus from the full series, but is obtained with a saving of time.

As a rule drawings of 10 or 11, 50 or 55, 100 or 110 and 275 were made, but, with the larger drawings, if not exactly 50 or 100 were drawn, the drawing was still utilized. Of course exact similarity in the size of the drawings is of no consequence whatever to any of the conclusions drawn.

A.D.

σ

Q.

the average deviation from the average.

the mean square deviation from the average.

one half the difference between the 25 percentile and 75 percentile

measures.

=

=

The results of these drawings are summarized in Table II. In Table II., Nt the number of sets drawn; Nc the number of cases in each set; Av.: the average divergence of the obtained1 from the true2 average; Med. = the average divergence of the obtained from the true median; A.D. the average divergence of the obtained from the true average deviation; the average divergence of the obtained from the true mean square deviation; Q.the average divergence of the obtained from the true (75 percentile 25 percentile)/2.

The figures for the last three divergences under 'Actual' are the direct results; the figures under 'Percentile' are these divergences in percents of the true fact.

The table shows that there is not enough superiority in accuracy in any case to outweigh the practical advantages which the median has as a measure of such quantities as prevail in the mental sciences.3 The divergences of the medians are on the whole only about 22 per cent. greater than those of the averages.

1 Obtained, that is, from the limited number of cases in the drawing in question.

26

2'True' meaning here that obtainable if all the measures of the total series are taken.

The general merits of the median are not discussed in this report; so also in the case of the general merits of the average deviation and of percentile measures of variability. To thoughtful students of mental measurements they will be obvious. The matter is briefly discussed in my 'Mental and Social Measurements' ('04) pp. 37 ff. and passim.

§2. The Comparative Accuracy of the Mean Square Deviation (called by various authors o, μ, e, S.D., or Standard

Deviation) and the Average Deviation

The most burdensome of the ordinary statistical operations is the calculation of the square root of the average of the squares of the deviations of a series of measures from their central tendency, that is, of the mean square deviation. In the case of the author, at least, practical judgment has long rebelled against the imposition of this measure upon workers with mental measurements by the experts in the theory of measurement of variable facts. To call it the standard deviation has seemed to him objectionable. There is apparently no reason whatever for its use except its supposedly greater accuracy. Perhaps because of lack of knowledge of the purely mathematical side of statistics, I am not aware that this greater amount of accuracy has been calculated from theory in the case of typical forms of distribution other than the so-called 'normal.' At all events it will be useful to the non-mathematical student to learn the facts in the case of empirical samplings from known series.

The series were A, B, C and D of Table I. The facts concerning the divergences from the true average deviation of the total series of average deviations obtained from random samplings, and similarly for mean square deviations, are given in Table II.

The average deviation and the mean square deviation were calculated from an approximate average never over a half of the unit of the scale from the actual average and as a rule from an approximate average less than a fourth of the unit of the scale from the actual average. The Q. was calculated on the basis of the same suppositions as the median.

So far as these samplings go, the average deviation is nearly as accurate as the mean square deviation. The latter is on the whole 5 per cent. more accurate, with about one chance in eighteen that an infinite number of drawings from these series would raise this superiority to 15 per cent. There surely can not be enough superiority of the latter to recommend its use in even 10 per cent. of the operations involved in present researches in psychology, sociology or education. Indeed it is a question whether the mathematical statisticians ought not to recognize the average deviation as approximately equal in accuracy and vastly superior in practical serviceableness, and hence as the measure to be recommended to students.

There is something to be said in favor of a still simpler measure of variability, the percentile. Galton's quartile (Q.), for instance (one half the distance between the 25th percentile and the 75th