Academic journal article Political Research Quarterly

A Reply to Miller et Al.: Replication Made Simple

Academic journal article Political Research Quarterly

A Reply to Miller et Al.: Replication Made Simple

Article excerpt

In the preceding argument, Miller and his colleagues leveled strong charges against our multiple values approach to the study of political intolerance. Distilled to its essence, their critique focused on three elements our study: (1) they claimed an inability to replicate our results; (2) they challenged the appropriateness of our measurement protocol; and (3) they raised more general problems with research on political tolerance. In this our rejoinder, we show, first, that any problems Miller et al. experienced in replicating our results are of a trivial nature: generally speaking, our findings are robust and may be replicated freely by interested readers by visiting our web-site where both the data for the analysis and our statistical program are posted. Second, Miller et al.'s evidence concerning the weakness of our value measures is based on a fundamentally flawed analysis. And third, the authors' more sweeping criticisms of political tolerance research, as well as their specific criticisms of our study, suffer from a limited understanding of existing theory and research in the area. We take up each of Miller et al.'s criticisms in turn.

Replication is surely a prerequisite to reliable knowledge in the social sciences. For this reason, we have posted the data and our statistical program on a publicly accessible web-site1 so that anyone can reproduce our results. A simple replication exercise will not only demonstrate the robustness of our findings but show that the alarm Miller et al. express at not being able to duplicate our results is seriously exaggerated, for a variety of reasons. In the first place, Miller et al.'s replication actually yields stronger support for our model than the original results we reported. Thus, in estimating baseline political tolerance, the unstandardized regression coefficient is larger in Miller et al.'s replication than in our original Table 1. Second, while our critics cannot replicate our results, we cannot replicate theirs. Try as we might, we are unable to determine why Miller et al. fail to uncover a significant coefficient for value conflict in the Cable Access model (the Replication Free Speech equation of their Table 2), although it should be noted that their coefficient is close to being significant at the .05 level. All our replications find a robust and significant effect for value conflict in the Cable Access model. In addition, we cannot account for the fact that Miller et al. end up with a different sample size for their analyses (i.e., the Replication Free Speech models in Tables 1-3).2

We conclude that the differences between our findings and Miller et al.'s repliccation are trivial and not demonstrative of any noteworthy flaws in our original analysis. This is particularly true of Miller et al.'s discussion of the direction of the effects of gender on political tolerance in our Table 1. Nowhere in our article do we state that "females are more tolerant than males," as Miller et al. claim we do. In fact, the effects of gender on tolerance is barely mentioned in our article aside from (correctly) stating, in passing, that "Finally, we note that-in keeping with prior studies-higher levels of political tolerance are significantly associated with greater political knowledge, less negative affect toward the group, more formal education, being younger, and being male" (p. 391, emphasis added). Miller et al. are apparently reacting to a clerical error made in the note at the bottom of Table 1, which should read, "Variables are coded so that higher values indicate: greater political tolerance ... and male," instead of female. Again, however, a sense of perspective is important: not only is the relationship stated correctly in the article, but gender differences are not even remotely central to the purpose of our analysis. Thus, we stand by our original findings, all of which can be easily replicated.

THE NOT SO SCIENTIFIC TEST

In our article, we go to great pains to demonstrate that values matter in affecting political tolerance judgments. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.