Academic journal article Perspectives in Education

A Rasch Analysis to Determine the Difficulty of the National Senior Certificate Mathematics Examination

Academic journal article Perspectives in Education

A Rasch Analysis to Determine the Difficulty of the National Senior Certificate Mathematics Examination

Article excerpt

Introduction

Large-scale studies, including examinations, tests and questionnaires have been used for data collection for research and, in the case of examinations, teachers use the results of the analysis to guide their teaching (Edwards & Alcock, 2010). A number of studies have also been undertaken to determine levels of mathematical ability at different stages of schooling (Wendt, Bos & Goy, 2011; Wilson & Macgillivray, 2007).

The National Senior Certificate (NSC) was written for the second time in 2009, followed by much criticism when the results were released (Association for Mathematics Education of South Africa [AMESA], 2009; Keeton, 2010). AMESA, which reported on the 2009 and 2010 Mathematics examinations, stated that the 2009 paper 1 was at too high a level, while the standard of the 2010 paper was fairer, despite the fact that there were not many questions at the lower level. Furthermore, Mathematics paper 2 of 2009 was a fair paper, and that of 2010 was at an appropriate level (AMESA, 2009; 2010).

The NSC examinations are high-stakes examinations in the South African schooling system, because they are school-leaving examinations. Also, they are used to select candidates for higher education programmes, hence, the maintenance of high standards of these examinations. It is, therefore, important that examination papers be analysed, paying particular attention to the quality of the questions (Grussendorf, Booyse & Burroughs, 2010).

Mathematics, Physical Sciences and Accounting are seen as 'gateway' subjects that facilitate entry into tertiary education for school leavers. Passing these subjects is critical because university study has the potential to address the lack of skills in South Africa (Grussendorf et al., 2010). Based on this issue, more emphasis has been placed on the analysis of the examinations of different learning areas such as Mathematics (Umalusi, 2009). Results in the NSC examinations for Mathematics, in particular, have been poor for a number of years. An illustration of this is given in table 1 which shows, for instance, that in 2009, 29% of the candidates obtained a mark of 40% or more nationally, while 31% achieved this in 2010. These results point to the need for perusing candidates' responses in the examinations. In this regard, the Rasch model was used to analyse the 2009 Mathematics scripts.

The Rasch model

Item response theory (IRT) is based on two postulates: the performance of an examinee on an item (test question) is related to the examinee's ability or latent trait; and the relationship between the examinee's performance and the difficulty of an item can be related by an item characteristic curve (ICC).

Examinees' abilities are scaled such that an 'average' person has a latent trait of zero and an 'average' examinee will have a 50% probability of answering correctly a question of 'average' difficulty. Also, IRT has the property of invariance, in other words, the characteristics of difficulty of an item are not dependent on the ability distribution of the examinees, and the ability of an examinee is not dependent on the item characteristics (Baker, 2001; Hambleton, Swaminathan & Rogers, 1991). So, if a question is asked in a different test with a different set of examinees, it should have a similar level of difficulty. The item parameters (item difficulty and discrimination index) are independent of the test takers' characteristics, and the test takers' parameter (ability level) is independent of the item characteristics.

Rasch analysis is a specific application of IRT. In Rasch analysis a distinction is made between dichotomous and polytomous analysis. The dichotomous model is used in simple questions where an answer is either right or wrong, such as multiplechoice questions. On the other hand, the polytomous Rasch model is used when a variety of marks can be awarded for a question (Wu & Adams, 2007), as was the case with questions in the NSC Mathematics papers. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.