Proxies in the New Political Economy: Caveat Emptor
Hanson, John R., II, Economic Inquiry
Today proxies for cultural, political, and institutional variables abound in econometric studies, and strong policy recommendations are based on results obtained with them. Unlike the traditional data of economists, they are neither market-generated nor the fruit of large official or semi-official statistical projects. The main providers are ideological or public-interest organizations, such as Transparency International and Freedom House, supplemented by scholars, corporations, and miscellaneous organizations.
The diverse and perhaps suspect origins of these data are reasons to use them cautiously. Yet even the prudent Robert Barro (1999) takes Freedom House's ratings of international political and civil liberty on faith in his recent study of the determinants of democracy--the trust standard. What I shall call the correlation test is a common way to assess multiple proxies for an intangible variable which also breeds complacency because it often is used as a substitute for direct appraisal of them. Alberto Alesina and Beatrice Weder (1999, p. 10) use it in a study of corruption's relation to foreign aid, claiming that "these relatively high correlations [among proxies for corruption] provide some confidence in the measures of corruption since most of them were compiled by different institutions using very different ... methodologies." The same spirit infuses what I shall call the mutual corroboration test for regression results. It is considered probative that different proxies for the same intangible variable produce similar or at least mutually compatible regression estimates.
The correlation and corroboration tests also appear in a recent World Bank study (Kaufman et al., 2000, p. 11) utilizing measures of the quality of governance. The authors write that "if [the data were not informative], we would not expect to see the ... strong agreement across sources about the quality of governance. Particularly striking is the broad consensus that emerges [among many diverse raters]." They acknowledge the conceptual differences among the literally hundreds of measures used in their study (all qualitative) but omit none from their composite indicators.
Finally, a pioneer in the new data, Gerald Scully (1992), constructs eight economic freedom indexes plus an average of them to be the economic freedom variable in one of his projects. He cites high rank correlations among them approvingly.
These tests use mutual resemblance or consistency as substitutes for directly assessing the suitability of proxies for a particular project. One-by-one evaluation is, to be sure, a laborious and inconvenient task, but ease and convenience are not tests of the validity of evaluation procedures. Correlation among the proxies or results obtained with them must not be confused with aptness of the proxies or results. The correlation and corroboration tests assume, for instance, that only measurement error differentiates proxies from one another. Yet political-institutional variables normally are defined differently from one another. "Good governance" is already a subjective judgment; "broad consensus" allows differences of opinion. Identical policy conclusions cannot follow from regressions based on measures having different meanings, so the quotation from Alesina and Weder is an indictment of their study, not a defense of it.
The main purpose of this article is to show that the trust standard and the correlation and corroboration tests are pseudo-criteria for judging the quality of proxies and results based on them because they are capable of supporting fallacious statistical estimates. These tests, I show, endorse statistical nonsense in an interesting, familiar case. The nonsense is attributable in part to the inapposite use of proxy regressors in estimations of a simple and well-known model, specifically bivariate regressions often cited in support of the strict free-market school of thought. …