PHOTO ILLUSTRATION BY DAN DAO / GETTY IMAGES
“The polls are in crisis. Polling is dead.” We heard variations on this theme after somewhat larger-than-average polling errors in the 2016 and 2020 presidential elections. The reality, though, is that polls are still quite important to understanding what the public thinks about different issues and remain useful for forecasting elections, as FiveThirtyEight’s 2022 midterm forecast is doing this year.
In other words, the death of polling has been greatly exaggerated.
That doesn’t mean, however, that there aren’t reasons to be worried about the state of polling in 2022. One big concern is that we have fewer surveys of individual contests in 2022 than in previous midterms. And a larger share of that smaller pie has been conducted by partisan pollsters and/or sponsored by partisan organizations, based on an analysis of polls conducted from early May to late October in the 2010, 2014, 2018 and 2022 midterm election cycles. As a result, we have less information about individual contests and are more reliant on polls that could be notably biased toward one party.
Fewer polls from a smaller number of pollsters
Simply put, there have been way fewer polls in 2022 than in past cycles. In 2010, pollsters conducted almost 1,700 polls of individual races for Senate, House and governor between early May and late October. By comparison, we have slightly more than half that number this time around — about 900. But this dropoff isn’t sudden; it’s been a more gradual decline over the past decade and a half, as the chart below shows:
Though the overall number of individual races polled has decreased, the number of polls of Senate, House and governors races has varied over these 12 years. This year, we have around 350 polls for both Senate and gubernatorial contests. Compared with 2018, that’s slightly fewer polls of Senate races and slightly more polls of governors races. Still, this is far less than the 500-plus surveys we had of both of those types of races in 2010 and 2014. Meanwhile, the number of House polls has ebbed and flowed since 2010: We only have about 200 House polls over the past six months, almost identical to 2014’s total, whereas we had more than 400 House polls in 2018 and more than 500 in 2010.
Interestingly, as the number of surveys of races at the state or district level has fallen, the number of national polls that ask about the generic ballot — which asks respondents which party they plan to support in their local U.S. House election — has more than doubled in 2022. This could be down to a couple of factors: For one thing, politics today are more nationalized than in the past, so pollsters may be incentivized to conduct national surveys, which will get more clicks and views. And it’s also potentially more economical — and safer — to poll a national audience on different issues and the generic ballot — which, for instance, can produce multiple stories for a media organization — than to poll individual races at the state level. The former involves weighting polls by information about the national population, whereas state-level polls may require more difficult choices to properly interpret results, as we’ve consistently seen larger errors in certain states than at the national level.
Another factor in the decline of polling of individual contests? A smaller number of pollsters are conducting surveys — or at least, releasing results. Over the past six months or so, 150 different pollsters have surveyed Senate, House and gubernatorial elections, down from roughly 190 in 2018 and the smallest total for any midterm in our sample.
While concerning, this trend isn’t necessarily surprising. Polling has become more expensive and more challenging, as the response rate to more traditional polling methods, like live telephone calls, is sometimes below 1 percent. Moreover, recent polling misses in 2016 and 2020 — note that 2018 polls were comparatively better — may have also made major news organizations more hesitant to put themselves out there by releasing surveys of important statewide races. Meanwhile, up-and-comers with more experimental — and at times, less transparent — methodologies have increased their polling output.
As a result, we’re getting more polls from potentially less reliable sources.
Partisan sources are releasing a larger share of polls
That’s because, compared with past cycles, polls in 2022 are more likely to be sponsored or associated with partisan sources. This is a problem because partisan polls tend to be more inaccurate than polls conducted by or for nonpartisan groups. Internal polls, which are sponsored by candidates’ campaigns, often try to spin a certain narrative by showing their preferred candidate in a better position than what polls from nonpartisan sources are indicating.
On FiveThirtyEight’s polls page, we denote whether a poll is sponsored by an organization that is considered partisan or is sponsored by a campaign. When we looked at the share of partisan polls in 2022 and for previous midterms, we found that the percentage of polls in Senate, House and gubernatorial contests coming from partisan sources has reached new heights.
Now, it’s not unusual to have a high percentage of partisan polls when looking solely at individual House races. House districts usually have smaller populations than states, so they tend to be harder to poll, which means campaigns often do much of the survey work in them. From 2010 to 2018, nearly half of all House polls came from campaigns or partisan groups. Yet that share has climbed to 54 percent this cycle amid a smaller number of overall House polls. Perhaps more disconcertingly, the share of Senate and gubernatorial surveys from partisan sources has risen to about one-third and one-quarter, respectively, even as the overall number of each is similar to 2018.
Conversely, we’re seeing a smaller number of horse-race polls conducted by or sponsored by “legacy media” outlets, which we broadly define as longstanding news sources with a major national reach. For instance, in 2010, legacy media sponsored and/or conducted 85 Senate polls and 64 governors polls between May and late October. But in the same span this year, the corresponding figures are just 25 Senate polls and 33 governors surveys.
This situation is perhaps best-illustrated by the rise of The Trafalgar Group, a Republican pollster that reaches respondents in different ways by using a mixed-mode approach. This cycle, Trafalgar is tied for the most Senate polls conducted (23) and ranks second in gubernatorial surveys (25), after releasing only a handful of polls in the same period in 2018 (the pollster became more prominent in 2020). In both cases, Trafalgar is either tied with or just trails Emerson College, another mixed-mode pollster that has become prodigious over the past couple of election cycles — although Emerson is nonpartisan.
We’re flying blind in many House races
With fewer total surveys, fewer races have been polled — especially in the House. So far this cycle, we have polls from 96 districts, which represents only about one-fifth of the chamber and is far less polling than the almost 150 districts surveyed in 2018 and nearly 170 in 2010. However, that 96-district figure is a bit deceiving, because only about half of those districts have more than one poll, including races where the same pollster has released more than one. This is problematic not just for the 339 districts for which we have zero polls, but also for our forecast, which takes polling data from districts that have it and infers numbers for similar-looking districts that lack polls.
One silver lining, though, is that we have a decent amount of polling in competitive Senate and gubernatorial races. This year, we have an average of about 26 surveys in the eight Senate races rated somewhere from “lean Republican,” to “lean Democratic” by election handicappers at The Cook Political Report, Inside Elections and Sabato’s Crystal Ball, and an average of about 16 surveys in the eight highly competitive gubernatorial contests. That Senate figure is right in between the average number of polls in 2010 and 2014 (about 24 and 29, respectively), helped out in part by the fact that fewer Senate seats are in play this year. The gubernatorial figure is lower than 2010 or 2014 figures but is actually much better than 2018’s.
Still, we have less polling in seemingly uncompetitive statewide races than in most recent midterm cycles, which could make it more likely that we miss a late-developing and potentially surprising result.
The state of polling is disconcerting. We have fewer polls, and a larger share of them come from partisan sources. We also have less information about House races, making race-to-race forecasting more perilous. All of this is a challenge for what we do at FiveThirtyEight, and we hope that pollsters and news organizations figure out how to offer the public greater polling information in the future.
Cooper Burton and Mary Radcliffe contributed research.
