Why was the polling wrong in NH?

Matt Dabrowski, a pollster and a reader, sent in a letter about what happened with the broken polling in New Hampshire

Just yesterday, some major media polls showed Barack Obama with a 10-point lead over Hillary Clinton. But Clinton won the New Hampshire primary by at least 3%. At the same time, John McCain posted a much stronger lead over Mitt Romney than polling predicated.

The polls were dead wrong tonight. (And so was I. I told many of you that Barack Obama would sail through New Hampshire on his way to the presidency.)

Why? In effect, the pollsters double-counted New Hampshire independents.

How does this work? In order to survey likely voters, pollsters ask a series of "screener" questions. Here is how one public pollster, American Research Group (ARG), asks their screener:

"Would you say that you definitely plan to vote in the 2008 Republican presidential primary, that you might vote in the 2008 Republican presidential primary, or that you will probably not vote in the 2008 Republican presidential primary?"

Typically, pollsters treat each party separately when conducting primary polls. One survey will poll the Republican primary (and all non-Republicans will be screened out), and a separate survey will poll the Democrats (and vice versa). This is the case for two reasons. First, most states don’t have open primaries, so cross-party voting isn’t a concern. Second is cost, doubly true in internal campaign polling. It would be a waste of money for, e.g., Chris Dodd to poll Republican primary voters.

Several well-regarded campaign pollsters will travel to South Carolina tomorrow to find themselves in hot water with their campaign managers.

So let’s think this through. Pollster X conducts his survey in the Republican primary. Independents are allowed into the poll, while Democrats are not. Then Pollster X conducts his survey among Democrats. Independents are allowed into the poll a second time, while Republicans are not.

This is the polling equivalent of being allowed to vote twice. You create a situation where a voter would say this: "Well, if you were asking me about the Republican primary, I’d vote for McCain. But since you’re asking me about the Democratic primary, I’ll vote for Obama." In fact, it creates the bizarre possibility that the same individual New Hampshire voter could be literally polled twice.

It comes down to what we call "sampling error." Since the same Independents were allowed into both surveys, the poll’s sample didn’t look like the actual primary electorate. The 10-point Obama lead was only a paper lead — those were actually McCain voters who were erroneously allowed into the Democratic primary polls.

Should pollsters have known this would happen? I’d argue yes. We knew that both Obama and McCain both had wide support among Independents. At first glance, none of the pollsters realized this and changed their models accordingly.

One proper way to screen would have been something like this:

"Q1. Would you say that you definitely plan to vote in the primary, that you might vote in the primary, or that you will probably not vote in the primary?

Q2. Do you plan to vote in the Republican presidential primary, or the Democratic presidential primary?"

Notice that every likely voter is allowed into the poll, and then pushed into the appropriate primary. No segment of the population has the possibility of entering both primaries.

We desperately need to re-think our turnout models in the future, or we will continue to struggle with open primaries, early voting, etc., etc…

Rasmussen: Romney as unelectable as Clinton?

Core Favorability/Opposition Among All Voters


Def. FOR











































Rasmussen has released another set of polling that allows us to compare all the candidates. The summary table to the right captures the main facts. You will recall that there is a horrible environment for Republicans in 2008. On the generic ballot, Democrats crush Republicans. However, Clinton has such an awful public image that the generic GOP problems are counter-balanced. Well, the conclusion from Rasmussen is that Mitt Romney has a comparable problem:

Among the leading Presidential candidates, New York Senator Hillary Clinton and former Massachusetts Governor Mitt Romney have the highest level of core opposition among voters. Forty-seven percent (47%) say they will vote against each of these candidates no matter who else is on the ballot.

Republicans often argue that Hillary is beatable because of this opposition. What about Romney who has no environmental advantages? Back to Rasmussen, which notes that John McCain is doing the best:

At the opposite end of the spectrum is Arizona Senator John McCain. For the second straight month, McCain finds himself with the smallest level of core opposition–just 33% say they will definitely vote against him. That figure is unchanged from a month ago, down from 39% a two months ago and a peak of 42% in June. These results are just one part of the reason that it is a good time to be John McCain.

Assume for a second that swing-voters will be the key in a general election, consider the additional facts among unaffiliated voters:

McCain has the lowest level of core opposition among unaffiliated voters–just 26% are committed to voting against McCain.

On a net basis, McCain (-6) and Obama (-11) have the best numbers among unaffiliated voters. Clinton (-26) and Romney (-20) have the weakest showing among this group.

In other words, the Clinton/Romney unelectability numbers extend into unaffiliated voters. While McCain gives Republicans the best chance of picking up substantial independent voters. That means keeping the White House. 

Important polling corrective

The LA Times reminds us the most important number in the recent NH polls:

But that’s not the most important data in this survey.

What’s most important is the little-noticed numbers revealing that with slightly less than two months to go before the voting, 48% of Democrats and fully 60% of Republicans remain undecided.

Now that’s a volatile setting, just made for stunning surprises in the nation’s first primary vote.

It’s gonna be a wild ride.

Economy most important issue?

Last week at Blog World Expo, a fascinating discussion broke out. Jerome Armstrong from MyDD and Markos from DailyKos, among other lefties, argued that Iraq was going to be a driving issue. They, furthermore, argued that success wouldn’t matter, because the failure was the initial decision, and Americans will stop paying attention The righties, Hugh Hewitt, Rob Bluey, John Hinderaker, and others argued that success would matter. Dean Barnett seemed to argue that it should but wouldn’t.

My sense is that the lefties are wrong. Iraq will come out of the headlines if we really start to succeed. The historical evidence provides the scenario. In 1952, Eisenhower ran on, more or less, pulling out of Korea. Eventually, after much wrangling, the troops stayed. Over 50 years later, 30k+ troops remain. It is clear that a similar scenario arises for the Dems. They know that it is irresponsible to completely pull out, in spite of their base. Dem presidential staffers admit to numbers in the range of 100k. They are even saying so in debates.

One wonders if, on a certain level, the Dems are going to mirror Eisenhower in this respect. He wanted to keep the GOP from embracing isolationism. There is little risk that Clinton would embrace the sort of irresponsible isolationism that so much of the Democratic base would seem to like, and that the 50s Taftians so dearly wanted.

That’s the good news for America. The bad news for the GOP is that once Iraq goes from being the top headline, the economy is the next issue. And, indeed, with casualties falling, Americans are focusing on the economy. As the Economist’s Democracy in America blog notes:

A MILESTONE of sorts was reached this month when a Newsweek poll showed that for the first time in years Iraq was not the top issue influencing prospective American voters. The economy had surfaced as the major issue on voters’ minds.

Fareed Zakaria quotes the poll:

In the new NEWSWEEK Poll, the economy now tops Iraq as the issue that voters say will most influence their choice for president, 22 percent to 19 percent. For two years, Iraq dominated these kinds of surveys. Only a month ago, in a CBS News poll, 28 percent of respondents wanted Iraq to be the campaign’s most-discussed issue, while the economy came in second at 16 percent.

Now, it is not clear to me that this is a huge win for Republicans, given the housing problems.  But Iraq is clearly coming out of the headlines. Which might mean that the Democrats have some space to make responsible decisions, if actually elected.

ARG, Fred, and Mitt

I normally don’t write about polls, especially ARG polls. But I was struck by two things in the most recent batch.

The first one, which lots of people have talked about is Mitt Romney’s rise in South Carolina. That’s new. Is it real? Is it TV moving numbers?

But the second thing is Fred Thompson. His collapse. In every state. What happened? Is that why Romney is rising? Are early state voters starting to really make up their minds?

When reporters miss the story: Rudy and conservatives.

Alan Fram AP’s story about Rudy Giuliani and conservatives fundamentally misses the point. He says, "Giuliani’s Conservative Vote Tenuous." That would be true, if it turned out that his lead in polls was based on the vote of conservatives who don’t know what he thinks on issues. It seems that the important analysis of his story is:

Yet a close look suggests his support from the GOP’s potent right wing is less than meets the eye, according to recent Associated Press-Ipsos polls.

But the important point is that, Rudy Giuliani holds the lead in-spite of weak support from conservatives. Rudy has found a possibly winning coalition does not involve the most conservative elements of the party.  That, dear reader, is a story. That shows that his path to winning the nomination is less-than-tenuous. But the reporter doesn’t seem to understand that the goal in a primary is to build coalitions within the party.

Let’s look at the facts from the poll:

Conservatives, evangelical and born-again voters, and strongly loyal Republicans who back Giuliani tend to be less conservative, less religiously active and less supportive of President Bush than those favoring Fred Thompson, Giuliani’s chief rival so far, the surveys show.

Is that news? That Rudy’s conservatives are less conservative?


  • Just 37 percent of Giuliani’s conservatives call themselves strongly Republican, compared to 52 percent of Thompson’s.
  • While 22 percent of Giuliani’s evangelical or born-again Christian supporters say they are very conservative, 47 percent of Thompson’s do.
  • Sixty-four percent of Giuliani’s supporters approve of Bush’s performance, compared to 78 percent of Thompson’s.

Isn’t this fantastic news for the Giuliani campaign? Doesn’t this tell us that his lead is based on people who aren’t going to go fleeing when someone (who?) puts up ads saying that he’s a liberal?

Isn’t this a reason for confidence? They know he is pro-choice, gay-friendly, etc. And they still support him. What additional information is going to make Rudy’s numbers fall? Probably not information about abortion, etc.

At least the reporters aren’t alone in their ignorance. The conservative interest groups don’t get it either.

New Des Moines Register poll: Strategy

Candidate %
Romney 29
Thompson 18
Huckabee 12
Giuliani 11
McCain 7

I normally don’t like to write on polls, but I think that this one tells us a lot about the future of the GOP race in Iowa. David Yepsen comments here. I am going to write this as a two-parter. The first part is about the numbers. The second part is about the strategy that I think needs to follow from the numbers, for some of the candidates.

I think that the lesson here is that Fred Thompson, Mike Huckabee, and, to a lesser extent, Mitt Romney have a lot of room for upwards progress. John McCain and Rudy Giuliani do not. If these trajectories continue, they should both consider dropping out of Iowa. New Hampshire and Michigan are both more permissive — open primaries — and less conservative. It seems clear that both candidates are, more or less, on this path. They should even consider telling their grassroots organizations to, on caucus day, support either Huckabee or Thompson. This would increase the likelihood of the story coming out of Iowa being about Romney.  As Yepsen says:

With everyone expecting [Romney] to win, when he does, he won’t get that much bounce out of it and he needs that to build his national poll numbers, which aren’t so hot.

On the GOP side, the focus is now starting to center on who finishes second and third in Iowa since the “coach” and “standby” tickets into New Hampshire are still good ones to have. Fred Thompson and Mike Huckabee have grabbed those positions. That’s especially good news for Huckabee who had been languishing back in single digits.

Rudy and McCain probably don’t have the power to influence the story about themselves coming out of Iowa. But they can probably do a lot to make sure that the right story about the other candidates gets told.

New Des Moines Register poll: Numbers

Candidate %
Romney 29
Thompson 18
Huckabee 12
Giuliani 11
McCain 7

I normally don’t like to write on polls, but I think that this one tells us a lot about the future of the GOP race in Iowa. David Yepsen comments here. I am going to write this as a two-parter. The first part is about the numbers. The second part is about the strategy that I think needs to follow from the numbers, for some of the candidates.

The most interesting bit is this: Mike Huckabee is, effectively, tied with Rudy Giuliani for third. And the trends are good for Huckabee and bad for Giuliani.

Some of the subgroup questions are interesting.

For McCain immigration is the problem:

Among those polled whose first choice was someone other than McCain, 58 percent say his support for a plan that would have offered illegal immigrants a path to citizenship is a major factor in not getting behind his candidacy.

For Romney, it seems to be flip-floppery:

But criticism of Romney as a politician who changes positions on key issues, rather than sticking to his convictions, appears to have had an impact. Among likely caucus participants whose first choice is someone other than Romney, 51 percent say his shifting of positions on issues like abortion is a major factor in not supporting him.

Giuliani, it is abortion:

Giuliani appears to be paying a political price, however, for his support for abortion rights. Among likely caucus participants whose first choice was someone other than the former New York mayor, 55 percent say his abortion stance is a major factor in not backing him.

Thompson seems to have the most potential:

Another plus for Thompson is that 10 percent of the supporters of other candidates would seriously consider him and an additional 53 percent might consider him if they knew more about him. Those are better marks than any other candidate received.