III. by writing for every item 1,000 times the decimal fraction that the time is of the total time spent by the five subjects on the item in question. These values we have called the 'weighted times'; they weight every item equally and are proportional to the ratio of a particular subject's time for a given item to the average time for that item.

We can not afford space for these tables. In Table IV., however, we give the averages of these weighted times for each subject and the mean variations of these averages. The two forms of Examination Alpha are treated together, as are also the two forms of the Otis Test. It will be seen that the rank order of the subjects is the same as the order stated above: B, A, C, D, E for Alpha, and A, B, C, D, E for the Otis Tests.

TABLE IV

AVERAGE WEIGhted Times and THEIR MEAN VARIATIONS FOR SUBJECTS A, B, C,
D, AND E IN EXAMINATION ALPHA, FORMS 5 and 6, and in THE OTIS

TEST, FORMS A AND B

To render the m.v.'s comparable for a comparison of subjects, the times in Tables II. and III. were written as 1000 X the ratio of the time of one subject to the total time of the five subjects in any item. The averages and m.v.'s of the table are averages and m.v.'s of these weighted times.

Table V. shows the significance of the individual differences of Table IV. The probable errors of the means have been found, the differences between every pair of averages, and the probable errors of these differences. The table gives the ratio of every difference to its probable error.

It is not possible to say dogmatically whether or not these differences are significant. Significance is relative and not absolute. Every difference must be thought of as having

SUBJECTS

TABLE V

SIGNIFICANCE OF INDIVIDUAL DIFFERENCES

The figures of the table are ratios of the difference to its probable error for the average weighted times for every pair of subjects. Examination Alpha is treated in the upper right half of the table, and the Otis Test in the lower left half. All the ratios are computed directly from Table IV.

Probabilities that the differences are not due to chance' are as follows:

* Only 0.34.

† 0.85.

Between 0.90 and 0.99.

§ Between 0.99 and unity.

In all other cases they are unity (with four-place tables).

=

some significance, and significance increases as this ratio of the difference to its probable error increases. Sometimes we are told that a difference is significant when this ratio is as great as 3; sometimes that it should be 5 or 6. It is between 5 and 6 that four-place probability tables give the probability 'that the difference is not due to chance' as 'certainty' (probability 1.0000).10 At any rate the table shows that, of the 20 differences shown, 14 are 'certainly not due to chance' in this mathematical meaning of the phrase. For Alpha all the differences except two are 'certainly not due to chance.' One of these two (A and C) is highly significant, however. The other is the difference between A and B, which we have already had occasion to doubt since it is reversed in the times of the Otis Tests, in intelligence scores of both Alpha and the Otis Tests, and, as we shall see presently, in reaction times. In the Otis Tests the differences are all less significant because

10 Cf. E. G. Boring, Amer. J. Psychol., 1916, 27, 317; 1917, 28, 456 f.

ALPHA

they are based on only two-fifths as many cases as are the ratios for Alpha. On the face value of the ratios, some very slight doubt attaches to the difference between D and E, a little more doubt to the difference between A and C, still more (naturally) to the difference between B and C, and a very great deal to the difference between A and B, which was the least significant difference in Alpha.

In general, then, we may conclude that our analysis has demonstrated significant individual differences in time of reaction when accuracy is approximately constant. Among our subjects, A and B are the fastest and very nearly equal in speed; C is a little slower than A and B, but not much slower; D, however, is much slower than C, and E is much slower than D. Our first task of analysis is performed: we have localized the difference of speed as inhering within the item. We have no need then to look for gross or occasional distractions or irrelevancies of thought, nor to appeal to gross introspection. The tendency for a differentiation as to speed appears in the simple intelligent act represented by a single item of Examination Alpha or of the Otis Test.

Reaction Times.-Having localized the speed factor as within the individual item, our next step was to discover whether it would still inhere in some simpler act that might be supposed to be a constituent or a simplification of the intelligent act (the reaction to an item). Here, however, it must be remarked that we had not actually reduced the speed factor to the simplest item, the item of test 4 of Alpha, the 'same-oropposite' test. In this test the reactions were so rapid that we were obliged to content ourselves with the times for items in groups of five. Nevertheless the implication from the form of the function in this test and from the results of the other tests made it seem probable that we should have found no different result had we had times for the separate items. Now the items of test 4 are very little more involved than controlled associative reactions, and we might have chosen to advance our limit by studying free and controlled association. times. It seemed, however, more economical for us to attempt to bracket the locus of the time-difference by choosing

the simple reaction, which we supposed could not possibly be expected to show these individual time-differences.

We used a Sanford chronoscope with a visual stimulus, a black dot that appeared at a circular hole in a screen. All subjects reacted by pressing the key with the right hand, except D, who was left-handed and used the left hand. The subjects were given a brief practice in the natural, sensorial, and muscular reactions, and, when it was found that they gave the chacteristic differences in times between these types of reaction, they were given 110 reactions of the muscular (abbreviated) type. We used the muscular type because we wished to take the simplest type of reaction possible in order to be sure of bracketing the time-difference. The first ten trials were rejected, and the remaining one hundred were averaged. The results were as follows:

To our great surprise even inspection shows that these reaction times give individual differences similar to the timedifferences obtained in the items of Alpha and of the Otis Tests. The three rank orders are:

Alpha..
Otis Test..

B, A, C, D, E.

A, B, C, D, E.

Reaction Time.

A, C, B, D, E.

There is a significant correlation between reaction time and the time of the test items (with Alpha, .70; with Otis, .90) even when only the rank-orders are considered. The relationship becomes more significant when one remembers that A, B, and C, among whom the inversions of rank occur, are grouped closely in all three measures, whereas D and E are more widely separated from them in every case.

If we compute the ratios of all the differences in reaction. time to their probable errors (vide supra), we get the following:

These ratios are not so high as the ratios noted in Table V. for the Otis Test, although both sets of ratios are based on 100 cases. Nevertheless five of them are over six, i.e., it is mathematically 'certain that the differences are not due to chance'; three more show a probability 'that the difference is not due to chance' greater than .98; and only two are small (B-C, .85; A-B, .60). If we are to trust common statistics,11 we must hold that the major differences here are as valid as the differences in times in the intelligence tests.

Some of the more significant data of this paper have been brought together in Table VI. and the rank correlations between the variables of Table VI. are shown in Table VII.

TABLE VI

COMPARATIVE AVERAGE PERFORMANCE OF SUBJECTS IN TIMES AND SCORES IN EXAMINATION ALPHA, IN THE OTIS TEST, AND IN SIMPLE REACTION TIMES

In Table VI. one can compare the reaction times with the other measures of time and of time-and-accuracy. Table VII. shows that the reaction times have a high correlation with the

11 That we may not always trust statistics, especially in the determination of the significance of differences, has already been argued by one of us: Boring, Psychol. Bull., 1919, 16, 335-338. T. L. Kelley has taken the author to task for these remarks in Amer. J. Psychol., 1923, 34, esp. 409-411; but the author sees still no reason to revise his original remarks.