Academic journal article The Journal of Negro Education

The Prospects Study and Desegregation

Academic journal article The Journal of Negro Education

The Prospects Study and Desegregation

Article excerpt

This report consists of two parts, first Puma's comments on David J. Armor's evaluation of the St. Louis desegregation plan and, second, a brief description of relevant portions of the Prospects (1996) study, for which Puma was the principal investigator.' The report he submitted to the court, dated December 26, 1995, has been revised to incorporate a few points raised in his oral testimony to the court on March 19, 1996, and to take advantage of findings from the Prospects study that were unavailable at the time of his testimony.

COMMENTS ON THE ARMOR REPORT

Dr. Armor's report, entitled "Evaluation of the St. Louis Desegregation Plan," was prepared for the Attorney General of Missouri. I was qualified by the court as an expert witness in education research and analysis, and my comments were on the author's analysis of student achievement data. The first part of his analysis presented time trends in reading and math norm-referenced test scores, measured as Normal Curve Equivalents (NCEs), based on a national norm of 50. These results were used to draw conclusions about the relationship between white and Black students' academic achievement. I had three main concerns with this part of the report

(1) As Armor noted, the time trends are somewhat difficult to interpret because of the city school district's decision in 1989 to switch from the California Achievement Test (CAT) to the Stanford Achievement Test (SAT). The downward shift in scores after the change is probably a result of a myriad of factors such as differences in test content and structure, differences in the norming samples and procedures, and the noted possibility that teachers were "teaching to the test" in the earlier time period. Consequently, I do not think there is much valid or useful information in the comparison of the two time trends, and suggest limiting the analysis to the post-1988 period. (2) If the analysis is to focus on SAT scores, I suggest use of scale scores for this assessment rather than NCEs. Scale scores are the correct measure of how students perform on the SAT and, as a consequence, are generally the preferred way to assess longitudinal changes in achievement.2 NCEs, by contrast, are essentially rankings against a national sample of students. (3) The conclusions that Armor derived from simple time trend analysis are not supported by the data presented in his report. From a statistical perspective, one cannot determine whether there are true differences between any two years of data on his accompanying graphs (or for the overall time trend) without information on the variation of student scores and how it changes from year to year. Consequently, Armor should have performed appropriate tests of statistical significance to support his conclusions about relative differences between the performance of Black and White students. Such tests are the commonly accepted practice in the research literature.

Moreover, there are many reasons why year-to-year differences in average test scores for a particular grade cohort should be observed apart from students' race, including (a) changes in the composition of the students who make up a particular grade cohort-that is, the presented data are cohort means measured over time and not differences over time for individual students; (b) changes in district and school decisions about who gets tested in any given year-for example, the extent to which limited-English-proficient and disabled students are included in the testing; (c) the declining appropriateness of the original norm sample for the test; (d) general temporal changes in classroom instruction and the extent to which the curriculum is more or less aligned with the test content; and (e) various school efforts to improve scores such as teaching of test-taking skills or offering incentives for teachers and administrators to coach students prior to the testing. The second part of Armor's analysis attempts to explain the observed test score differences between Black and White students. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.