Civil Society

This is also worthy of a close read. How could the pollsters get it so wrong?



--------------------------------------------------------------------------------

January 10, 2008
Op-Ed Contributor
Getting It Wrong
By ANDREW KOHUT
Washington

THE failure of the New Hampshire pre-election surveys to mirror the outcome of the Democratic race is one of the most significant miscues in modern polling history. All the published polls, including those that surveyed through Monday, had Senator Barack Obama comfortably ahead with an average margin of more than 8 percent. These same polls showed no signs that Senator Hillary Clinton might close that gap, let alone win.

While it will take time for those who conducted the New Hampshire tracking polls to undertake rigorous analyses of their surveys, a number of things are immediately apparent.

First, the problem was not a general failure of polling methodology. These same pollsters did a superb job on the Republican side. Senator John McCain won by 5.5 percent. The last wave of polls found a margin of 5.3 percent. So whatever the problem was, it was specific to Mrs. Clinton versus Mr. Obama.

Second, the inaccuracies don’t seem related to the subtleties of polling methods. The pollsters who overestimated Mr. Obama’s margin ranged from CBS and Gallup (who have the most rigorous voter screens and sampling designs, and have sterling records in presidential elections) to local and computerized polling operations, whose methods are a good deal less refined. Everyone got it wrong.

Third, the mistakes were not the result of a last-minute trend going Mrs. Clinton’s way. Yes, according to exit polls the 17 percent of voters who said they made their decision on Election Day chose Mrs. Clinton a little more than those who decided in the past two or three weeks. But the margin was very small — 39 percent of the late deciders went for Mrs. Clinton and 36 percent went for Mr. Obama. This gap is obviously too narrow to explain the wide lead for Mr. Obama that kept showing up in pre-election polls.

Fourth, some have argued that the unusually high turnout may have caused a problem for the pollsters. It’s possible, but unlikely. While participation was higher than in past New Hampshire primaries, the demographic and political profile of the vote remains largely unchanged. In particular, the mix of Democrats to independents — 54 percent to 44 percent respectively — is close to what it was in 2000, the most recent New Hampshire primary without an incumbent in the race.

To my mind all these factors deserve further study. But another possible explanation cannot be ignored — the longstanding pattern of pre-election polls overstating support for black candidates among white voters, particularly white voters who are poor.

In exploring this factor, it is useful to look closely at the nature of the constituencies for the two candidates in New Hampshire, which were divided along socio-economic lines.

Mrs. Clinton beat Mr. Obama by 12 points (47 percent to 35 percent) among those with family incomes below $50,000. By contrast, Mr. Obama beat Mrs. Clinton by five points (40 percent to 35 percent) among those earning more than $50,000.

There was an education gap, too. College graduates voted for Mr. Obama 39 percent to 34 percent; Mrs. Clinton won among those who had never attended college, 43 percent to 35 percent.

Of course these are not the only patterns in Mrs. Clinton’s support in New Hampshire. Women rallied to her (something they did not do in Iowa), while men leaned to Mr. Obama. Mrs. Clinton also got stronger support from older voters, while Mr. Obama pulled in more support among younger voters. But gender and age patterns tend not to be as confounding to pollsters as race, which to my mind was a key reason the polls got New Hampshire so wrong.

Poorer, less well-educated white people refuse surveys more often than affluent, better-educated whites. Polls generally adjust their samples for this tendency. But here’s the problem: these whites who do not respond to surveys tend to have more unfavorable views of blacks than respondents who do the interviews.

I’ve experienced this myself. In 1989, as a Gallup pollster, I overestimated the support for David Dinkins in his first race for New York City mayor against Rudolph Giuliani; Mr. Dinkins was elected, but with a two percentage point margin of victory, not the 15 I had predicted. I concluded, eventually, that I got it wrong not so much because respondents were lying to our interviewers but because poorer, less well-educated voters were less likely to agree to answer our questions. That was a decisive factor in my miscall.

Certainly, we live in a different world today. The Pew Research Center has conducted analyses of elections between candidates of different races in 2006 and found that polls now do a much better job estimating the support for black candidates than they did in the past. However, the difficulties in interviewing the poor and the less well-educated persist.

Why didn’t this problem come up in Iowa? My guess is that Mr. Obama may have posed less of a threat to white voters in Iowa because he wasn’t yet the front-runner. Caucuses are also plainly different from primaries.

In New Hampshire, the ballots are still warm, so it’s hard to pinpoint the exact cause for the primary poll flop. But given the dearth of obvious explanations, serious consideration has to be given to the difficulties that race and class present to survey methodology.

Andrew Kohut is the president of the Pew Research Center.