Response Style Differences in Cross-National Research: Dispositional and Situational Determinants

Beginning of article

Abstract:

* International Management researchers often rely on surveys to collect their data. However, responses to survey questions can be biased by response styles, a respondent's tendency to provide a systematic response to questions regardless of their content. Response styles vary across countries and individuals, but there is limited systematic research that investigates why they vary.

* Our study investigates middle (MRS) versus extreme response styles (ERS), the tendency to use the middle or extreme categories on rating scales. We examine the impact of culture, different types of scale anchors and the level of knowledge of the topic in question on MRS and ERS.

* We asked five groups of respondents (Chinese in China, Chinese in Australia, Anglo-Australians in Australia, and two groups of German students in Germany) to indicate on a 10-point scale whether certain employee attitudes or behaviour were more typically Australian (left-hand of the scale) or Chinese (right-hand of the scale). We then asked them how they would rate the performance (low to high on a 10-point scale) of an employee who displayed this attitude or behaviour.

* Asian respondents showed higher MRS than Western respondents. When scale anchors referred to naturally opposing and mutually exclusive constructs (Australian versus Chinese) respondents showed more ERS than when they referred to level or degree of a construct (low-high performance). Knowledge of cross national differences resulted in higher ERS on behavioural questions but not on performance questions.

Keywords: Response styles. Survey research * Culture * China * Australia * Germany

Introduction

Researchers in the field of International Management use a wide variety of research methods. Two of the most frequently used methods and data sources are survey research or the use of secondary data, in itself often based on surveys conducted in the past (see e.g. Buckley et al.'s. (2007) review of studies on FDI location choice). Researchers would normally assume that responses to questionnaire surveys are only based on the substantive meaning of the questions involved. This might be true for surveys that deal with issues that can be measured objectively, such as for instance the classification of a subsidiary as greenfield establishment or acquisition. However, there are many types of questions where respondents may display a bias in their responses.

These biases could include a consistency bias, i.e. the respondents' desire to be seen as consistent by the researcher, which may lead them to respond in accordance with a presumed underlying relationship, for instance the assumption that certain practices should lead to high performance. It could also refer to social desirability bias, the tendency to respond in a way that is seen as socially desirable. This can be expected to be especially relevant in questions that are for instance dealing with ethics or corporate social responsibility. Researchers will also find that questions relating to firm performance, when measured subjectively with Likert scales using low/high or below/above average anchors, often lead most respondents to score their performance above average.

However, there is another type of bias--response style bias--that can be invoked with any types of surveys that contain Likert-scale questions, which ask respondents to rate their opinions and attitudes. These questions could relate to individual cultural values or norms, but also to the respondent's assessment of the company's international strategy or the subsidiary's role. Response style bias is rather unique in that it is not dependent on the content of the question. The term 'response style' refers to a respondent's tendency to provide a systematic response to questionnaire items regardless of their content (Baumgartner and Steenkamp 2001). The most common response styles are acquiescence (ARS) or disacquiescence (DRS); that is, the tendency to agree or disagree with an item regardless of the content, and extreme response styles (ERS) versus middle response styles (MRS); that is, the tendency to use the extreme or middle response categories on ratings scales (Harzing 2006).

Response styles can create a range of problems for researchers. First, they contaminate observed responses, because they either inflate (ARS) or deflate (DRS) respondents' scores on the measurement instrument in question (Baumgartner and Steenkamp 2001). They can also increase (ERS) or constrict (MRS) the response range. These effects are particularly problematic in comparing groups of respondents from different countries, as these have been shown to differ substantially in typical response styles (see e.g. Harzing 2006 for a recent summary). An international management researcher might therefore draw the erroneous conclusion that particular groups of respondents are different on the phenomenon under investigation, while in reality the groups only differ in terms of their response styles. Second, response styles can also influence the conclusions drawn about the relationship between variables as they can inflate or deflate the correlation between respondents' scores on the various scales (Baumgartner and Steenkamp 2001).

Acquiescence and extreme responses styles are the most commonly studied type of response style and have been recognized for half a century (see e.g. Couch and Keniston 1960; Cronbach 1946). However, these studies have all focused on Likert-scale questions with response anchors reflecting an intensity dimension, such as levels of agreement or levels of importance. The first part of this study follows a different approach by using scale anchors that prevent an acquiescence bias by offering two opposing and mutually exclusive options, in our case Australian versus Chinese behaviour. We therefore do not consider acquiescence bias in our study. Instead, we study extreme response styles (ERS) versus middle response styles (MRS); that is, the tendency to use the extreme or middle response categories on ratings scales. Although previous studies have looked at ERS versus MRS (see below for a summary), they have all focused on Likert-scale questions with disagree/agree scales. As detailed below, in our study we contrast two different types of scale anchors. This allows us to significantly expand our knowledge of response style differences and hence our ability to get more meaningful answers in cross-national research.

Contributions of Our Study

In our study we asked five groups of respondents (Chinese in China, Chinese in Australia, Anglo-Australians in Australia, and two groups of German students in Germany) to read 90 statements about employee attitudes or behaviour and indicate whether they thought these attitudes or behaviours were more typically Australian (left hand side of the 10-point scale) or Chinese (right-hand side of the 10-point scale). We then presented them with the same 90 statements and asked them how they would rate the performance of an employee who displayed this type of behaviour or attitude on a 10-point scale from low performance (=1) to high performance (=10).

This approach has three advantages over previous studies. First, the use of countries as scale anchors in the first part of the study and performance implications of typical behaviour in these countries in the second part of the study, provides a more focused assessment of ERS versus MRS. In contrast to the use of disagree/agree Likert scales, these scale anchors do not conflate these response styles with ARS/DRS. When using Likert scales with disagree/agree formats it is difficult to distinguish positive ERS (the tendency to use the positive extremes of the scale) from ARS (the tendency to agree, i.e. use the positive extremes of the scale). The same is true for negative ERS and DRS.

Second, our scale anchors are less likely to be vulnerable to translation problems. As Harzing (2006) notes, differential interpretation in different languages of equivalent scale anchors might be an important explanation for response styles. Although scale anchors might translate into appropriate local equivalents, the intensity associated with these equivalents might be different from the original language. Since our scale anchors in the first part of the study refer to specific country names rather than abstract constructs such as trust or motivation, it is unlikely that our translations are misinterpreted. Although the scale anchors in the second part of the study referred to low/high performance, this was framed in a country context, with alternating statements of Chinese and Australian behaviour. Hence, any differences in intensity between languages would be cancelled out as they would apply to both "Australian" and "Chinese" statements.

Third, the use of a 10-point scale is relatively uncommon in studies dealing with response styles. Hui and Triandis (1989) recommend increasing the number of response categories as a technique to reduce ERS. The rationale behind this is that scales with finer gradations provide more opportunity for the respondent to reflect their true opinion, without having to rely on extreme answers. Hui and Triandis (1989) recommend scales with more categories as particularly appropriate for cross-cultural research.

Our study builds on earlier studies by providing a coherent rationale for differences in response styles. Most of the earlier studies on response styles provided either descriptive results or limited post-hoc rationales for differences. More important for international management researchers is why response styles differ between countries and groups. Reasons for differences in response style can be dispositional--that is, related to individual characteristics such as age, gender or personality--or situational--that is, related to situational characteristics such as the format of the response scale, the ambiguity of questions, or time pressure (Baumgartner and Steenkamp 2001).

In the context of cross-country differences in response styles, cultural differences would be a likely dispositional explanation. In this article, we mainly focus on collectivism versus individualism and argue that MRS is likely to be more common in collectivistic countries. We also include a situational variable that has not been studied before: the type of scale anchors used. We argue that if the scale anchors refer to naturally opposing and mutually exclusive constructs, respondents are more likely to provide extreme responses than when scale anchors refer to a level or degree of a construct, e.g. agreement, performance or importance. To the best of our knowledge there are no studies that have systematically varied the type of scale anchors with the same question set. Finally, we include a dispositional variable that has not been studied before: the level of knowledge a respondent has about …