Inasmuch as some portions of the studies reported here were based on the intercorrelations of tests, and the tests were administered in numerous examination centers, it is necessary to indicate the efforts made to maintain uniform conditions of administration in the various centers. The unit of administration was the sub-center or single room in which the candidates were seated under the control of a supervisor who read instructions in accordance with a printed schedule. In the six years of examining reported here there were the following numbers of sub- centers:
At one of the informal conferences on the two-factor theory conducted by Prof. Spearman at Columbia University during the fall and winter of 1931-32 the statement was made that the conditions under which our tests were administered were such that no reliance could be placed on the data. The point was made that statistical data based on examinations given by many different supervisors examining in many different centers were unreliable.
A general charge of faulty administration cannot be made, but specific errors made by certain supervisors at certain centers should be brought to our attention. All supervisors were carefully selected and many had years of experience in conducting examinations. It was realized at the outset that great care should be exercised in preparing instructions for the supervisors, and the care shown is illustrated in the following pages which reproduce the Supervisor's Manual of Instructions for 1929. All other manuals were closely similar to this sample.
Table 70 has been consolidated from the six annual reports and lists the numbers of candidates examined at each sub-center in each year. This table has been published annually in order that criticism of our administrative procedures might be made specific. All errors discovered have been described in our annual reports. We cannot, of course, maintain that our administration is entirely free from error. We do insist, however, that charges of faulty administration be specific.
To one who has had experience in the treatment of statistical data there are certain evidences of the stability of our data which are very impressive. If errors of administration had been frequent, the intercorrelations of the tests obtained in successive years would have varied. Table IX of the 1931 Report1 presents a series of coefficients obtained over a period of years. The improvements in test construction resulting from our item studies first became felt in 1928 and became more effective in the later years. Correlations as stable cannot be derived from data secured under lax conditions of test administration.____________________