The National Assessment of Educational Progress provides critical information regarding the academic achievement of our nation's students. Previous research using NAEP data showed substantial gaps in science achievement between several policy-relevant groups. Since the NAEP science assessment began in 1990, males have scored higher than females, whites and Asians have outperformed African Americans and Latinos, and higher socioeconomic groups have done better than lower SES groups (National Center for Educational Statistics, 2005).
Finding a conclusive explanation for these persistent gaps has proven to be elusive, however. While this study emphasized the associations between such socio-demographic factors as SES, ethnic/racial backgrounds and gender, it was not meant to point the finger at membership in "at-risk" groups as predictors of academic failure. Rather, membership in low-SES or minority groups is viewed as part of the broader explanation of interrelated factors that contribute to disappointing performances in science achievement by non-privileged groups (Van Secker, 2004; Van Secker& Lissitz, 1999).
The purpose of this study was to explore possible explanations for the enduring science achievement gap in California public schools between privileged and non-privileged students. I asked the following questions to guide the study:
1. Does science achievement vary systematically among schools?
2. To what extent do gender, race and SES account for differences in science achievement?
3. Do instruction and school environment affect the average achievement of students within the same school?
Exploring an ecological perspective
While previous research cited a preponderance of evidence regarding family background and school characteristics and their influences on science achievement, what is lacking in the literature is an integrated approach for arriving at explanations for the science achievement gap between privileged students and those labeled "at-risk." An ecological perspective opens the possibility for arriving at such an explanation. This line of inquiry specifies that when examining a student's academic outcomes, it is important to consider the influence of various social spheres, including families and schools (Bronfenbrenner, 1979; Elder, 1974; Rutter; 1988; Steinberg, 1996).
Simply put, this perspective provides an integrative approach for understanding how social background affects students' experiences during their schooling. My analysis builds on Bronfenbrenner's (1979) suggestion that an essential task for researchers should be to penetrate the label of socio-demographic characteristics (race/ethnicity, gender and SES) to identify the specific elements of social structure, family background and school environment that shape the course of student achievement.
To develop an ecological framework I selected key family and school characteristics available in the NAEP dataset.
Family, home and school environment
Evidence from the literature shows the ability of parents to foster positive attitudes about science is one of the most important predictors of science achievement (Carey & Shavelson, 1988). I selected variables that modeled the interrelated sphere of family and community influences: family background variables (race, gender, parental educational attainment), home environment measures (number of books in the home), and student attitudes toward science ("I like science"). I also selected the variable "hands-on learning" to model the student's instructional experience in the classroom.
As Bronfenbrenner suggested, the interaction between factors in the child's immediate family background, school experiences and the larger social context shapes his or her development. Changes or conflict at any level will ripple throughout the other dimensions. Therefore, to study a child's development it is critical to look at the child's immediate environment (family), and his or her interaction with society (school-level variables).
I chose two school-level variables: the percentage of minorities in the school (school minority status) and a measure of the percentage of students eligible for Title I funds (school SES) to model the broader social context of the youngster's schooling. I approached the analysis in two phases. I first conducted descriptive analyses for the individual and school-level variables in the study. I followed with HLM modeling (see below).
The full details of this study are reported elsewhere (Valadez, 2010). For the purposes of this article I have summarized the results. I reported the mean science achievement scores for a random sample of California fourth graders and eighth graders. I selected my sample by restricting my study in the NAEP national database to California public schools. After deleting missing responses, I had a final sample size of 2,084 fourth graders and 2,529 eighth graders.
The fourth-grade results followed familiar patterns found in the literature: males (144.03) outperforming females (141.13); whites (159.13) scoring higher than blacks (124.58) and Hispanics (126.99); and high-SES groups (149.13) outperforming low-SES groups (126.61).
Eighth grade results were similar: boys (151.75) outperforming girls (144.27);whites (156.39) doing better than blacks (123.29) or Hispanics (123.12); and high-SES groups (148.75) outperforming low-SES groups (124.04). Although these data are cross-sectional and not longitudinal, they still indicate a disturbing trend: the gap between boys and girls, whites and minority groups, and high- and low-SES groups appears to widen when students reach the eighth grade.
The next series of variables show differences in home environment among the various groups. Fourth-grade children who come from homes with 100 books or more (152.14) have much higher scores than children who come from homes with few or no books (123.39). Eighth-grade children are similar, 148.75 compared to 124.04.
The results for student attitudes deliver few surprises. Fourth-grade students who declare they "like science" (146.30) have higher scores than students who do not (131.39), while eighth-grade students who like science (143.51) outscore those who do not (126.61). Interestingly, the difference among eighth graders (16.9 points) is larger than fourth graders (14.91 points). The variable chosen to model reform-era practices for teaching science was a measure of the amount of hands-on learning (instructional experience) that teachers used in their classrooms. Students who participated in fourth-grade classrooms in which teachers used hands-on practices nearly every day (148.90) did better than students who never or rarely used hands-on learning (138.47). The mean differences associated with this type of learning were large for eighth graders as well, 141.94 compared to 121.19.
The school-level variables I used in the study were intended to model the social context of the children's school experience. I created two variables: school SES (eighth grade), the percentage of students attending a particular school who were eligible to receive Title I funds (fourth grade); and school minority status (eighth grade), the percentage of students in the school who were either black or Hispanic. Unsurprisingly, the differences were consistent and large between low-SES (127.91) and high-SES schools (162.22), and low-minority (145.61) and high-minority enrollment schools (121.43).
Analyzing the data at multiple levels
Hierarchical linear modeling (HLM) is a regression technique that is performed in two steps. In the first step, analyses are conducted separately for every school in the system using student-level data. In the second step, the regression parameters (intercepts and slopes) from the first step of the analyses become the outcome variable of interest. These are regressed on school-level data (school SES and school minority status), describing school contexts.
The basic concept is that it allows the researcher to analyze the data at multiple levels in order to find the influence on student achievement at the individual student level as well as at the school level. Below I describe the steps, beginning with the unconditional models.
Fourth-Grade Unconditional Model. The results here show a large effect associated with attending a low-SES school. In effect, attendance at low-SES schools indicates an average of a 24-point drop in science achievement.
Eighth-Grade Unconditional Model. Similarly to fourth graders, there is a significant cost attributed to attending a high-minority enrollment school--a drop of 24.23 achievement points.
In summary, these analyses produced a test of the first research question that all schools have the same mean science achievement. The findings show that low-SES and high-minority schools had different rates of achievement and provided a rationale for conducting further analysis. In the next step I sought an explanation for why some schools had higher means than others and why in some schools, the association between SES/minority status and achievement was higher than in others.
Fourth-Grade HLM Analyses. This analysis found that where differences in home environment or student attitudes exist, they had a stronger impact on children in high-SES schools than in low-SES schools.
Eighth-Grade HLM Analyses. The results for eighth grade are equally troubling. Certain students seem to get a boost when attending schools with low-minority enrollment. Students who come from high-SES families do better in schools with low-minority enrollment as opposed to high-minority enrollment schools.
Positive attitudes are also significantly different at low- and high-minority schools. Students with positive attitudes score higher in achievement tests than students at high-minority schools. In addition, students who engage in hands-on learning at low-minority schools show higher achievement than students at high-minority schools.
Concerns about low achievement
While the findings in this study are not conclusive, claims of causation were not the intention of this article. However, the article does highlight certain policies, practices and effects that raise concern regarding the education of students from low socioeconomic and minority backgrounds. The descriptive evidence shows convincing findings that Latinos, blacks and poor students lag behind their white peers in science achievement.
There are numerous reasons to be concerned with this low achievement. From a basic human capital perspective, non-privileged students are at a disadvantage for competing for places in higher education, high-tech jobs, or absorbing critical scientific knowledge for participation in an increasingly technological society. From a social justice perspective, a societal concern is the impact of schools on the education of California's diverse student population.
The fourth-grade data show that youngsters who attend low-SES schools, have supportive home environments and positive attitudes toward science do not perform as well as students with similar backgrounds attending high-SES schools. The mechanism for understanding this lag cannot be explained here, but there appears to be a benefit that is attributed to high-SES schools that may be absent in low-SES schools.
The case for eighth graders is also troubling. Students with high-SES backgrounds in high minority schools do not perform as well as high-SES students in low-minority enrollment schools. This effect extends to students who have positive attitudes toward science. Clearly, the evidence shows that positive attitudes are linked to better performance on the science achievement tests, but students at high-minority schools perform less well than their low-minority-enrollment counterparts.
Another curious finding is the impact of instruction on eighth graders. Engaging in hands-on instruction in science class seems to benefit students at low-minority enrollment schools more than it does students at high-minority schools.
These findings open up some important policy discussions. There is evidence that promoting strong student interest in science produces students who seem more engaged and willing to learn. Simply put, students who like science do better in science. Another issue, particularly for younger students, is the importance of home environment for promoting learning. "Books in the home" is a rather oversimplified proxy for home support, but signifies that a supportive home environment is critical for student achievement.
Two other issues are evident but may not be easily addressed without further study. Students with resources at their disposal (high-SES students at high-minority schools) do not do as well as students at low-minority schools. Internal mechanisms for discouraging achievement may be at work, or some other undiscovered explanation may be present.
The same is true of instructional experiences. Active learning (frequent hands-on instruction) is associated with improved achievement. This is true of fourth graders and eighth graders. For eighth graders, however, there appears to be a dampening of their achievement in high-minority schools.
School reforms--particularly improved instructional practices--are in order, but policymakers must address why certain groups (white, high-SES) gain more from reforms than other groups.
Explaining differential achievement
Earlier I presented confirming evidence that achievement varies among schools. In addition, the data in this article assert the widely documented fact that privileged students do better than non-privileged students. This, of course, is not surprising news. The point here is to dig deeper into the data to build an explanatory model for investigating the persistent differences in achievement between privileged and non-privileged students. I proposed an ecological framework to offer a conceptual model for understanding these differences.
What is clear is that individual-level variables such as student attitudes, home environment and instructional experiences contribute to a partial explanation for the differential achievement of students. The school-level variables in the model--that is the larger school context--also contribute toward building an understanding for differential achievement of students.
Students enter schools with certain resources, abilities and aspirations. Although the evidence presented here is preliminary, it does appear that schools can either activate or depress those resources to influence achievement. It appears that high-SES schools and low-minority-enrollment schools are most effective in converting these resources into high academic achievement. The irony perhaps is that students with advantages and privileges are those students who gain the most from individual attitudes, familial support and school reform practices.
Bronfenbrenner, U. (1979). Nature and design. Cambridge, MA: Harvard University Press.
Carey, N. & Shavelson, R. (1988). "Outcomes, achievement, participation, and attitudes." In R. J. Shavelson, L. M. McDonnell, & J. Oakes (Eds.). Indicators for monitoring mathematics and science education. Los Angeles: Rand Corporation.
Elder, G. H. (1974). Children of the Great Depression. Chicago: The University of Chicago Press.
National Center for Educational Statistics. (2005). The Nation's Report Card. Washington, D.C.: United States Department of Education. http://nationsreportcard. gov/science_2005/
Rutter, M. (1988). Studies of psychosocial risk: The power of longitudinal data. Cambridge, UK: Cambridge University Press.
Steinberg, L. (1996). Beyond the classroom: Why school reform has failed and what parents need to do. New York: Simon and Schuster.
Valadez, J. R. (2010). Estimating the Influence of Ecological Factors on Low-income Student Science Achievement. Paper presented at the Conclave--UCLA, California Association of Latino Superintendents and Administrators.
Van Secker, C. (2004). Science achievement in social contexts: Analysis of data from the National Assessment of Educational Progress. Paper presented at the annual meeting of the American Educational Research Association: San Diego.
Von Secker, C. & Lissitz, R. W. (1999). "Estimating the impact of instructional practices on student achievement in science." Journal of Research in Science Teaching, 36(10).
James R. Valadez is professor of education at California Lutheran University. This research was presented at the California Association of Latino Superintendents and Administrators Higher Education Conclave in March 2010…