National educational policy analysis and evaluation are complex endeavours that demand empirical data-gathering efforts that are of appropriate scale and high quality but mounting such data-gathering efforts can be resource- and time-intensive. As an alternative, perhaps under-utilised, strategy, this paper describes a retrospective secondary analysis of an existing large-scale data set that potentially adds value to educational policy evaluation. Specifically, as a member of the Organisation for Economic Co-operation and Development (OECD), Australia participates in the Programme for International Student Assessment (PISA) that assesses the literacy of 15-year-old students in reading, mathematics and science. PISA is administered on a cyclical three-year schedule that began in 2000 with a focus on reading, followed in 2003 with a focus on mathematics and 2006 with a focus on science. The PISA surveys have made an important departure from other international assessments by decoupling the instruments from school curricula; rather, the assessment instruments are based on holistic definitions of discipline-specific literacies--the skills and knowledge deemed necessary for personal and working life in industrialised countries with 21st-century economies--in the core learning areas of reading, mathematics and science (OECD, 2004). PISA data sets are housed and managed by the Australian Council for Educational Research (ACER) and it is the 2003 data set that is the subject of our secondary analysis here.
Australia's Commonwealth government has begun consideration of applying a so-called 'socio-economic status (SES) model' within its policies guiding school funding. For the current study, we suggest that the secondary analysis of extant large-scale data sets can provide important input to the discussion of Commonwealth school funding policy by shedding light on previously obscured or possibly unexamined relationships. In particular, it is already well established in the educational research literature that the socio-economic status of individual students is strongly associated with educational achievement as measured by standardised assessment systems, whether local, national or international. In addition, various international studies have shown that the aggregated socio-economic profile of a school is also positively associated with students' academic achievement (OECD, 2004; Rumberger & Palardy, 2005; Sinn, 2005).
On the other hand, less is known about the nature of these relationships when both individual student and school socio-economic status are disaggregated. To uncover these finer-grained associations, we subjected Australia's 2003 PISA data set to retrospective secondary analysis to better understand the reading and mathematics literacy performance of secondary school students from different SES backgrounds, across a variety of school SES strata. This analysis therefore contributes to our understanding in two important ways. First, from a methodological perspective, the study demonstrates the process and potential usefulness of a secondary analysis approach using a large-scale dataset as a contributor to national policy evaluation. Secondly, the study adds value from a substantive perspective in shedding light on a key policy question currently facing the Commonwealth: specifically, the findings presented will add to data-informed decision-making around the appropriate federal funding of public education, as well as the use of public funds in the support of independent and Catholic systems of schooling across Australia. In these two ways, this secondary analysis demonstrates a strategy that holds potential for optimising the value of public policy evaluation through the enhanced use of extant large-scale, high-quality data sets in the consideration of important national policy questions.
Socio-economic status and student outcomes
School socio-economic composition is a strong predictor of student academic achievement in many countries (OECD, 2004; Rumberger & Palardy, 2005; Sirin, 2005). …