Academic journal article Journal for Educational Research Online

Domain-Specificity of Research Competencies in the Social Sciences: Evidence from Differential Item Functioning

Academic journal article Journal for Educational Research Online

Domain-Specificity of Research Competencies in the Social Sciences: Evidence from Differential Item Functioning

Article excerpt


Acquiring research competencies (RC) is an important goal of higher education (British Academy, 2012; Wissenschaftsrat, 2006). Yet, in the social sciences, there is a lack of discussion about specific learning objectives, and no comprehensive tools to assess the attainment of these objectives are available (Earley, 2014).

Existing competency measures focus on parts of the research process only (information literacy, see Katz, 2007; statistical literacy, see Stone, 2006) or on understanding and applying research results (Groß Ophoff, Schladitz, Leuders, Leude rs, & Wirtz, 2015). Additionally, none of the measures available incorporates the assessment of competencies in both quantitative and qualitative social research. While in the natural sciences, recent efforts yielded a measure applicable across disciplines (in physics, chemistry and biology, see Hartmann, Upmeier zu Beizen, Krüger, & Pant, 2015), no such measure is available across the social sciences.

Against this backdrop, for the present paper, a newly developed measure (Gess, Geiger, & Ziegler, in press) assessing RC across social research paradigms and social-scientific disciplines was applied to examine the domain-specificity of RC. Students' RC is typically taught in discipline-specific learning environments, such as specific research methods courses for students enrolled in psychology or educational studies (Wagner, Garner, & Kawulich, 2011). The opportunities to learn (OTL) may vary between disciplines, presumably resulting in distinct response behaviors to the measure's items.

The study follows, thus, a recommendation initially made for research on scientific reasoning to focus on differences between disciplines instead of their commonalities (Fischer et al., 2014). Hence, differences in OTL between the disciplines were analyzed and related to between-discipline variance found in the measure's items. This analysis allows for a closer investigation of the new measure but also provides tentative conclusions on the nature of social-scientific RC as such.

2.Conceptual framework and state of research

2.1Research competencies in social sciences

The term research competencies is used in different ways. First of all, it is important to differentiate between studies that focus on an engagement in research and on an engagement with research (Borg, 2010). The competencies to engage with research are often examined in higher education programs that train for a specific profession (e.g. teacher, nursing or medical education). In these studies, students are tested on their competencies to understand and use research results in professional decision making ("educational research literacy", see Groß Ophoff et al., 2015, p. 560). In the present paper, the competencies to engage in research - i.e. the competencies necessary to generate new knowledge based on scientific methods - are examined. In the following, this line of research is called research competency.

Our study built on a conceptual framework of research competency. The framework specified the content areas associated with RC in the social sciences - based on interviews and two surveys with methodological experts. Experts highlighted three research steps as particularly insightful: (a) finding and defining a research problem, (b) planning a research project and (c) analyzing and interpreting data. Three knowledge domains were identified as "cognitive dispositions" (Koeppen, Hartig, Klieme, & Leutner, 2008, p. 68) underlying successful social research: (a) research process knowledge, (b) knowledge of research methods and (c) knowledge of methodologies. By highlighting knowledge domains and research steps, the competency model is in line with the continuum of approaches to model competencies (Blömeke, Gustafsson, & Shavelson, 2015). For each combination of knowledge domain and research step, indicators were identified (see Figure 1). Note that content-specific knowledge pertaining to the research field is required to competently conduct a research project. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.