Polling Acceptability Standards Measured Across States

By Jim Ellis

April 15, 2021 — Five key Democratic pollsters recently released a report from a post-election meeting held to determine why their 2020 survey results again so often fell below acceptable accuracy standards.

Go to Democracy Docket to see all results by clicking on image above.

Representatives from the Democratic polling firms of ALG Research, Garin-Hart-Yang Research Group, GBAO Strategies, Global Strategy Group, and the Normington Petts firm met to develop a plan to correct some of their methodological flaws in order to provide more accurate information to their clients and the public in future elections.

Their key conclusion was that, generally, the polling community failed to accurately depict enough conservative/right-of-center vote coming from rural areas. They make the point that in politically changing places like Arizona and Georgia, the aggregate data was quite accurate, but they collectively missed in more rural Republican-leaning states such as Iowa.

While the polling firm reps stressed the point that their collective numbers were strong in the metropolitan areas, there seems to be, however, a further detail left unsaid or possibly unnoticed. That is, the polling community seemed to correctly project the outcomes in states that have one large central metropolitan area that is self-contained within a single media market such as Arizona (Maricopa County), Colorado (Denver Metro), Georgia (Atlanta Metro), and Nevada (Clark County), but not in states with diverse geographical configurations.

While the firms tended to miss by relatively large margins in rural states like Montana, and particularly so in that state’s Senate race, they also seemed to fare poorly in places where the population is more evenly spread and have several metropolitan areas. States in this category include several in the Midwest such as Ohio, Michigan, and Wisconsin, along with Florida, a major Sun Belt state.

In the latter place, while the polling was relatively accurate from a margin perspective, the aggregate polling community has consistently missed projecting the main Florida statewide winner(s) in every election since 2014. In each case, Democrats were predicted to win with a close margin while in actuality the Republican candidates claimed the election.

The charts below detail the states hosting the most competitive races and present the mean average polling during 2020 campaign’s final week. This average is then compared with the state’s final results. Notice that in places such as Michigan and Pennsylvania, culling the Democratic research firms from both the Republicans and the local pollsters – the latter of which in many places performed better than the national firms, i.e., in Maine, Iowa, and Kansas – drastically changes the average.

The Democratic pollsters’ meeting primarily revolved around presidential polling because of their uniform belief that former President Donald Trump had such a strong effect, both positive and negative, upon the electorate.

The Senate contests, however, contained some of the bigger polling misses. The most egregious is Maine, where the last week polling average showed Sen. Susan Collins (R) trailing by 5.7 percentage points when she would prove victorious with an 8.6% margin. Once more, we see a big miss in a largely rural state.

Therefore, to summarize, polling methodology flaws continued in 2020, and they center around Democratic pollsters and universities that employ similar sampling methods and turnout models that have repeatedly underestimated Republican/right-of-center voter strength. This was most often found in rural states, or in larger domains with multiple metro areas where the population is relatively evenly spread.

Conversely, and to reiterate, the pollsters proved highly accurate in states with one major metropolitan area self-contained within a single media market.

The main exception to this latter pattern is North Carolina, where the presidential polling proved accurate in a state with several metro areas and an evenly spread population. That being said, while the projected Senate race margin was relatively close to being correct, the polling community failed to spot the winner, predicting that Democrat Cal Cunningham would clinch the election when Sen. Thom Tillis (R) ultimately prevailed.

Presidential Polling

Senate Polling

NOTE: Georgia polling proved largely accurate for both races and in the runoff campaigns
AVG.: Mean average for the last week of polling