Using Response Latency to Increase Lead Time in Election Forecasting [Using the 1993 Federal Election]
Bors, Douglas A., Bassili, John N., Canadian Journal of Behavioural Science
This research explores ways of increasing lead time in forecasting in the context of the 1993 Canadian Federal Election. The aim was to test if the accessibility of voting intentions, as indexed by the time taken by a respondent to express a voting preference in a pre-election CATI study, can be used to identify respondents whose intentions are likely to change as election day nears. The paper focusses on three models: A Standard Model that allocates a full unit of support to the party the respondent intends to vote for, a Weighted Vote Model, that allocates decreasing support to the chosen party as response latency to expressing a voting intention for that party increases, and a Split Vote Model that allocates decreasing support to the chosen party as response latency increases and allocates the rest of the support to the respondent's second choice. An improvement in the accuracy of forecasting was made by using the Weighted Vote Model, but this improvement did not extend beyond two months prior to the election. Augmenting the Weighted-Vote model with information about respondents' second choices not only did not result in further improvement, but also resulted in a reversal of the previous improvements.
In his comprehensive discussion of variables that affect pre-election polling accuracy Crespi (1988) notes four major sources of error: (1) flawed sample designs, (2) failure to screen nonvoters out of the sample, (3) inadequate treatment of undecided responses, and (4) failure to measure late changes in voting preferences. Crespi's analysis, which was originally presented by the Social Science Research Council in its examination of factors that contributed to the poor performance of the polls in the 1948 presidential election, captures what continues to be important preoccupations among election pollsters.
Crespi (1988) reports the results of a multivariate analysis of the impact of a wide range of factors on poll accuracy. The factors that were examined fell into two categories - factors under the control of the researcher (sample size, importance of accuracy as a goal of the survey researcher, number of days until the election) and factors not under the researcher's control (primary or general election, whether an incumbent is running for reelection, turnout rates for the election, margin of error for the winning candidate, percent undecided in the general public; cf. Lau, 1994). Among the fascinating results emerging from this study was the surprising revelation that sample size was not significantly related to error rates, a result that has been replicated in a subsequent, though less ambitious, multivariate study by Lau (1994), and that is probably due to the importance of response rates even in large samples that are based on good sampling procedures. As for variables that were related to error rates, Crespi found that the number of days between the election and interviewing or "lead time" was the strongest predictor of error rates, polls conducted close to the election naturally enjoying an advantage over polls that were temporally more remote.
The issue of lead time is at the heart of any forecasting exercise, accurate predictions made barely moments prior to the criterial event being of limited utility, while inaccurate predictions made well in advance of it being even less useful. This familiar predicament was evident to the Social Science Research Council following the 1948 presidential election and is captured by the fourth source of error it identified, namely the failure to measure late changes in voting preferences. Much effort has been expended since then to track late changes in voting sentiments. One of the central challenges in this endeavour is the difficulty of collecting, compiling and reporting large amounts of survey data in a matter of a few days. Although telephone interviewing and the widespread use of computers in the survey process have reduced this challenge to manageable proportions in recent years, it is still often necessary to assess trends by com-paring the results of successive surveys or by using rol-ling samples designs. …