Several problems in polling lack a consensus solution. But in my opinion, only the decline in response rates presents a bigger unresolved issue than the identification of likely voters.

Most elections involve just a fraction of American adults. But how to accurately sample them?

Surveys overstate voting rates for the simple reason that many nonvoters claim to regularly vote. As a consequence, many pollsters ask one or more questions to indirectly gauge the likelihood of voting. Such questions can ask how closely the respondent follows politics or how much they care about a particular race. One study placed the accuracy of the most celebrated methodology in this vein—Gallup—at 73 percent. That is, it misclassifies about a quarter of actual voters as unlikely to vote, or nonvoters as likely to vote.

Another approach is to sample based on voting history. Analysis of voter registration files reveals that a good predictor of voting in non-presidential elections is previous voting in non-presidential elections. While this method approaches 90 percent accuracy in a static voter file, this approach is far from perfect. Among other problems, there is regular turnover in voter registrations. Whether it is young or old voters, or simple mobility and migration from one jurisdiction to another, there is always some degree of change. Another problem is logistical—sampling from voter files adds the further difficulty of finding valid household telephone numbers, typically with about 70 percent success. But that will have to be the subject of another post.

Our solution for Majority Watch was two-fold.

First, we sampled from the voter registration rolls those that have voted in at least one non-presidential general election in the last four years (2 of 4 voters in campaign parlance), supplemented by a selection of newly registered voters (in most districts this made up approximately ten percent of our sample).

Second, we began the interview by asking the respondent to estimate their own likelihood of voting. For the first round we used a nine interval scale with uncertain/undecided at the midpoint. We have recently tested a revised question based on Charles Manski’s work (self-reported probability of voting). In our past work we achieved very accurate results by assigning voter likelihood probabilities to each respondent based on past vote history. Our plan is to integrate a similar model into the second round of Majority Watch.

In combining both independent and self-reported information, we hope to arrive at the best estimate of likely voters. At the least, we’ve found internal validity. For example, almost every “perfect” voter reported that they were certain to vote, whereas we found a more diverse range of responses for more infrequent voters.

For this first round of the project, with interviews in 27 of 30 districts conducted before Labor Day, we did not think it appropriate to screen by voter likelihood beyond voting history as reflected in the initial sample. We’ve reported a “most likely voter” banner—respondents that indicated they were certain to vote. Expect further refinements the next time out.