If you are a progressive political junkie, odds are that one of the most depressing features of your week is the release of new polls from Scott Rasmussen. By and large, the ubiquitous robo-calling firm yields results that are more encouraging to Republicans than others (e.g., the big advantage it shows for the GOP in the generic congressional ballot), and the sheer weight of its state polling can be mind-numbing and spirit-sapping.
It’s generally been thought that this “house effect” of Rasmussen polls is the result of the early and stringent use of “likely voter” screens, which tend to produce a more conservative electorate. According to that theory, the “house effect” would be reduced as we get closer to election day and people make up their minds whether or not they are going to vote (this also accords with Rasmussen’s good record of final-days accuracy in recent elections).
But Nate Silver, as is his habit, takes a closer look at Rasmussen’s operations, and reaches a different conclusion: the raw sample Rasmussen uses before applying a “likely voter” screen seems to bear a “house effect” as well:
Although Rasmussen rarely reveals results for its entire adult sample, rather than that of likely voters, there is one notable exception: its monthly tracking of partisan identification, for which it publishes its results among all adults. Since Labor Day, Rasmussen polls have shown Democrats with a 3.7-point identification advantage among all adults, on average. This is the smallest margin for the Democrats among any of 16 pollsters who have published results on this question, who instead show a Democratic advantage ranging from 5.2 to 13.0 points, with an average of 9.6.
Why would that happen? Nate doesn’t suggest any deliberate bias by Rasmussen; but the firm does use polling techniques that tend to skew the sample:
Raw polling data is pretty dirty. If you just call people up and see who answers the phone, you will tend to get too many women, too many old people, and too many white people. This is especially the case if you rely on a landline sample without a supplement of cellphone voters.Pollsters try to correct for these deficiencies in a variety of ways. They may use household selection procedures (for instance, asking to speak with the person who has the next birthday). They may leave their poll in the field for several days, calling back when they do not contact their desired respondent. An increasing number may call cellphones in addition to landlines.
Rasmussen does not appear to do any of these things. Their polls are in the field for only one night, leaving little or no time for callbacks. They do not call cellphones. They do not appear to use within-household selection procedures. In addition, their polls use an automated script rather than a live interviewer, which tends to be associated with a lower response rate and which might exacerbate these problems. So Rasmussen’s raw data is likely dirtier than most.
Add in the likelihood that Republican voters are a bit more enthusiastic about reporting their views to pollsters at present, and you can see how Rasmussen’s “house effect” could be baked right into the cake. But if that’s true, the assumption that Rasmussen’s numbers will get more reliable as we approach election day may be questionable as well. Silver thinks the Rasmussen “house effect” is a new development that has emerged during this election cycle. So, too, may be a pattern of inaccuracy unless the firm takes corrective action.
Thus, Democrats are not necessarily exhibiting their own biases by taking Rasmussen’s results with a large grain of salt or mentally shifting the numbers leftward a few points. That’s an inexact science, but so, too, is polling.
This item is cross-posted at The Democratic Strategist.
