NEAL CONAN, host:
This is TALK OF THE NATION. I'm Neal Conan in Washington.
The op-ed page of today's New York Times features a stinging indictment of opinion polls in New Hampshire. All of which had Senator Barack Obama comfortably ahead of Senator Hillary Clinton in the Democratic primary. The indictment is particularly strong given that it comes from Andy Kohut, director of The Pew Research Center for the People & the Press. One of the pollsters who got it wrong. He joined us here in Studio 3A to answer your questions about what went wrong and why.
800-989-8255, if you'd like to join us. E-mail us, [email protected]. You can also weigh in on our blog. That's at npr.org/blogofthenation.
Andy Kohut's op-ed is called "Getting It Wrong."
And thanks very much for coming in, on what can't be your happiest day as a professional blogger.
Mr. ANDREW KOHUT (President, The Pew Research Center for the People & the Press): As it turns out, Neal, I didn't get it wrong because I didn't play. But I might well have gotten it wrong if I had played.
CONAN: Okay.
Mr. KOHUT: Because one of the things I thought that struck me was that all the polls got it wrong from the more sophisticated ones like Gallup and CBS to less rigorous ones - the computerized polls. They all seemed to be suffering the same problem in the Barack Obama-Hillary Clinton race - an overstatement of Obama.
Now, it's not a general breakdown of methodology because the same polls did a wonderful job in the Republican race of getting McCain's margin just about right.
CONAN: So, what happened on the Democratic side?
Mr. KOHUT: Well, you know, I don't know. But there are a couple of - I mean, what you have to do is think about what didn't happen before you can consider what happened. And there are some other things that didn't happen. There really wasn't that large a last-minute trend. 17 percent of the voters said that they decided that day, and there was a three-point margin for Hillary Clinton over Barack Obama, not enough to account for the difference.
The turnout levels were high, but the profile of the electorate was pretty much what it was expected. So it wasn't the conventional things that you would point to and say, oh, there's a survey method thing going on here. Now, detailed rigorous analysis may provide something, but there's nothing really obvious.
I think one of the things that got me thinking was the nature of Hillary Clinton's constituency, mostly poor voters, mostly less well-educated voters. And that, in combination, was what I know about survey sampling that says these are people we undersample. And the history of polls overstating support for black candidates suggested to me that we should really consider whether race was an issue or reluctance of some voters to support - white voters to support Barack Obama may not be definitely the case.
But given the profile of her support, the history of these things and the fact that polls under - don't get enough of these poor people, less well-educated people. We weighed them up, but the people we don't get are probably not like the people we do get in some respects, opinions about race might be one.
CONAN: Well, let's pick that apart. Why is it that, to begin with, that poor or less well-educated people are undersurveyed?
Mr. KOHUT: I think because they're suspicious of pollsters. They don't want to participate. They're not as interested in politics. It's a historical trend. It was the case back 15 years ago when I did get a biracial election wrong. The David Dinkins-Rudy Giuliani race in 1989.
CONAN: For mayor of New York.
Mr. KOHUT: For the mayor of New York. And, you know, this is a standard fact in surveys. You get fewer poorly educated people, fewer less affluent people. And…
CONAN: Some have pointed out that David Dinkins - there was a corruption charge that came out the day before the election that might have had some…
Mr. KOHUT: That might have had something to do with it. But the margin of error for me was 16 points. Sixteen points. Most elections - I do pretty well at. And, you know, I wasn't alone.
And, you know, biracial elections in the past have had this history. Now, recently, the polls have done a better job, which in measuring support for black candidates and black-white races, but - and it might not be the issue here. But we really have to give it some consideration given the nature of Hillary Clinton's constituency, and the fact that the affluent well-educated people are the Barack Obama supporters and the poor people support Hillary Clinton.
CONAN: In Iowa, there are poor people than less well-educated people. And the surveys - the opinion surveys in Iowa were more or less right.
Mr. KOHUT: They certainly were. But, yeah, Iowa is a caucus and not an election, and you're drilling down to a much smaller group of people. And you also have a process by which people come and state their things, they're not just voting.
And the other thing is that Barack Obama was not the clear frontrunner in Iowa that he was going into the New Hampshire primary where all the news was about how Hillary Clinton was going to lose, and he was going to win. And that might have been a motivating factor for some white voters who are reluctant to see a black candidate.
I want to emphasize, I mean might, Neal - but, you know, what concerns me is that pollsters are unwilling to consider - some pollsters are at least - are willing to consider the fact that half of the people we try to get, we don't get. And because they're generally not different from the people we do get, we do fine. But that's not always going to be the case and it could be what was happening here and it could be what happens in other cases where polls don't do so well.
CONAN: We're talking with Andy Kohut.
If you'd like to join the conversation, our number is 800-989-8255. E-mail is [email protected].
And let's start with Helen(ph). Helen calling us from Portland, Oregon.
Helen, are you there?
HELEN (Caller): Yes, speaking.
CONAN: Go ahead. You're on the air.
HELEN: Oh, great. Hello?
CONAN: Hi. You're on the air. Go ahead.
HELEN: Hi. It's funny, I'm listening to the radio, it sounds different. I…
CONAN: Because it's a little later. Just listen to me and not to the radio.
HELEN: I see. Okay. I'm calling from Portland, Oregon. And my comment is that even though Dennis Kucinich is my candidate of choice, I think he'd make the best president. But I think the polls got it wrong because a lot of people, women, I think, were keeping it secret maybe from their husbands and even at poll, that they just want to see a woman win, they just want to see feminism maybe take a step forward instead of the giant leap backwards that it's been taking. And what I consider would be Bush's America right now.
CONAN: Is that - could that have been a factor, Andy Kohut?
Mr. KOHUT: It could have been a factor in this small rally to Hillary Clinton. There was some issue among the voters who decided at the last minute. There was some issue about…
HELEN: Mm-hmm.
Mr. KOHUT: …the ways in which Barack Obama addressed Hillary Clinton. Yes, certainly she had stronger support from women in New Hampshire than she did in Iowa, much stronger support. And - but Barack Obama continued to have the big margin among men. So, that may be part of it.
HELEN: Yeah.
Mr. KOHUT: And I'm not denying that. But my concern is why were these polls so far off the margin? And I don't think that's part of it, to be honest with you.
HELEN: Hmm.
CONAN: All right. Thanks, Helen.
HELEN: Thank you.
CONAN: Bye-bye.
Let's see if we can go now to - this is Peter(ph). Peter with us from Berkeley, California.
PETER (Caller): Hi. My comment is this, that whatever the technical reason, I found this refreshing. I found it as a reminder that it's kind of a non-determinate, universally (unintelligible) in this case and exciting. It's more fun that - to think that, you know, the election can turn out a little bit different from what everybody is called scientifically predicting.
And in a way, it goes along with quantum physics, doesn't it, where, you know, there is a certain level of uncertainty that's just always there? And I find that fun.
CONAN: And so, is there a Heisenberg principle here operating here?
Mr. KOHUT: Well, I don't know if there's a Heisenberg principle operating. But the caller is right on. I mean, there is always a level of uncertainty in these things. And they're talked about with gospel, in part, because the polls have been so good.
I mean, we've come off of 2000, we've come off of 2004, two of the closest elections, that pre-election polls were better than the exit polls. And in 2000, they accurately mirrored one of the closest elections in modern history.
That isn't to say that in all cases, this methodology is going to work and that we shouldn't have a little pause and let the reality speak for itself and not only, you know, the strong assumption that something is going to happen, particularly when we're dealing with two unknown phenomena.
We've never had a woman running for president as a frontrunner. We've never had an African-American running at this level of standing in the polls. And look, this is, for me, this is hindsight. I wasn't up there screaming and saying, whoa, these polls are probably going to be wrong. But this poll - this experience, as the caller suggests, has taught us a lesson. It's not refreshing for the polling community. And you enjoy it, please. But we're not enjoying it, but hopefully we're going to learn from it.
CONAN: Thanks very much.
PETER: Thank you.
CONAN: Stay tuned for more excitement, maybe. I wanted to ask you a sort of technical question. We are talking about opinion surveys that were conducted, I think, up until Monday, at least in part.
Mr. KOHUT: Right.
CONAN: There are also then exit polls, which is a different method entirely. You're talking to people coming out of the voting booth. Did they make the same kind of mistake?
Mr. KOHUT: No. They seemed to do pretty well. There may have been a little bit of a bias there, but it was pretty good. And they had the election results to weight them too pretty soon. And so they - you know, we didn't have - we haven't had the miscues of these exit polls that we saw in 2004. And 2006 was pretty good. The folks who are running them now, Joe Lenski and Edison, market research - they're really doing quite a good job.
CONAN: Let's get Lee(ph) on the line. Lee with us from Silver Springs in Nevada.
LEE (Caller): Hi.
CONAN: Hi.
LEE: Well, I have a question which is - I'm wondering if the polls are media-driven or if the media is poll-driven? And the reason I'm saying this is because as a Democrat in Nevada, which is, you know, puts me in a minority, I've been following this race and wondering why Bill Richardson, who's the guy I like, hasn't been getting any media attention. I mean, I'm listening to NPR on the night about the Iowa caucus and after New Hampshire, listening for how he did. They don't even mention him. I - you know, it's like (unintelligible) of this man is spectacularly qualified to be president. Probably, the most qualified person in both camps. And he - you know, the first attention the guy is getting this when he's talking of dropping out of the race.
CONAN: Which he supposed to do about 10 minutes from now or at least…
LEE: Right.
CONAN: …at least they've reported it.
LEE: And have the sort of rock star mentality going on. It was like the press is like groupies following (unintelligible). And then black man and, you know, (unintelligible). I'm woman. I'm all for my minority. I think, you know, white men (unintelligible). I'm all for it at this point. But I'm wondering why, you know, why the other candidates aren't getting the attention from the media and if the polls aren't skewed by what the media is doing or vice versa.
CONAN: Andy?
Mr. KOHUT: Well…
LEE: I'll take my answer off the air.
CONAN: Okay. Thanks for the call, Lee.
Mr. KOHUT: Certainly, the media does follow the polls. The media often sponsors these polls, pays for them, and they recognize that polls have pretty good record, so they're going to pay attention to them.
Now, to what extent to people who are not doing well in the polls, never make it, many of them don't. But you have a candidate like Mike Huckabee, who was at about 3 percent in the polls in August or September, gradually come up in the polls, come into Iowa, and do very, very well. And the polls saw that. And the polls really do give voice to what's going on in terms of American public opinion that the media often doesn't recognize until they see it manifested itself in the polls.
I mean, the polls are sort of a - in many respects, a troop squad for the media because there's - the media is sometimes surprised. The biggest surprise, for example, years ago was the extent to which the public actually became more approving of Bill Clinton after the Lewinski scandal broke, not less approving. The media was ready to have him resigned by the third week in January. And there are many other examples of how - what the polls tell the people, challenges the media's assessment and the conventional wisdom. Now, which isn't to say that, you know, the notion that the media can get poll-driven isn't also a problem, but they're both problems.
CONAN: And the media also looks at the fundraising capabilities, which before people vote is taken as a measure of their organizational strength and their likelihood to be able to compete in the future.
In any case, we're talking about opinion polls right now in New Hampshire and why the pollsters got it wrong with Andy Kohut.
You're listening to TALK OF THE NATION from NPR News.
And let's get Clint(ph) on the line. Clint calling from Kansas City.
CLINT (Caller): Hi. Yes. You got to kind of trust (unintelligible) a little bit earlier. But I kind of see polls as the moment you start studying a poll, the moment things start changing. I think polls tend to have changed people's mind.
CONAN: Oh, back to Heisenberg principle…
CLINT: (Unintelligible) shows about how people (unintelligible) such as a high chance of winning. They voted for McCain.
CONAN: Yeah. So, well, let me ask about that, Clint. We're going to have to let you go because your line is so poor. But thanks very much for the call.
Mr. KOHUT: I don't think so. There is no real evidence of a bandwagon effect. In fact, we would never have candidates who come out of nowhere and either burst onto the scene and then get picked up by the polls and the polls show growing support for the constituency or candidates who are way behind and all of a sudden the polls begin to show that they're grainy gaining ground even at that time when they're being dismissed by political observers.
CONAN: Let's get Tim(ph) on the line. Tim with us from Roseville in California.
TIM (Caller): Hello. I want to say I'd listened to Mr. Kohut yesterday morning on MORNING EDITION. I read at the editorial today in New York Times, which parallels a lot of what he'd said so far in your show. And my feeling is I don't understand why you, Mr. Kohut, are apologizing to us for these mistakes in an explanatory way because we're not the - we're consumers, but we're not the one who's paid to do the poll. We end up with the poll and sometimes our votes are based on the poll, whether people go out and vote or not in the primaries because of a poll or whether they go in the general election because of a poll, I think disrupt the whole effect of our voting. People should just go out and vote not because they know where someone supposed to be. I understand the campaign works know where they candidate is. But the voter, you know, shouldn't be affected by that. And my opinion (unintelligible) published.
CONAN: Are you saying we shouldn't have opinion polls?
TIM: Well, no. We're going to have opinion polls because, as a marketer or a campaign leader, you want to know where your horse, so to speak, is in the race. So I understand that. But as a voter, you should just be going to vote.
People will vote base on, well, that person is doing well, so I go vote for them. That should not be a reason for people to vote for any candidate.
CONAN: Do…
TIM: So I just feel that these…
CONAN: And let Andy respond, okay?
TIM: Sure.
Mr. KOHUT: I'm not indicting the polls. I'm not apologizing for the polls. I'm a pollster and I have great empathy with what happened yesterday. But what we do - our job, the public pollsters, is to try to tell the story of public opinion. Voting is a - voting decision is a process. It doesn't happen overnight. People just don't go into the polls and pull the lever. The opinions about candidates and issues developed overtime and if polls are well done, they charts the process and tell the story of the way campaigns are won or lost, and it's a form of journalism. And I think it's perfectly legitimate now and important to make that case with data rather than to have the talking heads and the so-called experts who are basically expressing their opinions.
Now, as to citizens, the most important thing for citizens to do is to think about the issues, think about the candidates and, you know, look at the polls as news and the environment. But this is a personal decision, not a collective decision.
CONAN: Tim, thanks for the call. We appreciate it. And, Andy Kohut, again, I know you've been working really hard. We thank you for your time today.
Mr. KOHUT: Thank you, Neal.
CONAN: Andy Kohut joined us in Studio 3-A. He's director of The Pew Research Center for People & the Press. You can find a link to his op-ed on our blog at npr.org/talkofthenation.
And this is TALK OF THE NATION from NPR News. I'm Neal Conan in Washington. Transcript provided by NPR, Copyright NPR.