New Pollster Ratings

By Jim Ellis

March 30, 2021 — The FiveThirtyEight statistical organization released their latest political pollsters’ accuracy ranking chart toward the end of last week — a study that included 592 survey research firms.

Grades from A+ to C/D were assigned to the firms based upon four criteria:

1. Their 2020 election cycle precision
2. The predictive category that suggests how successful the firm will be in the future
3. The number of polls analyzed in this most recent cyccle, and
4. hat the FiveThirtyEight team terms as the company’s “mean-reverted bias” factor.

Four polling entities earned the top A+ rating. They are:

• the Iowa-based Selzer & Company
• ABC News/Washington Post
• Siena College/New York Times
• IDB/TIPP

Another six received A ratings:

• Survey USA
• Landmark Communications
• Research & Polling
• AtlasIntel
• Monmouth University
• Marist College
• Fox News

Seven more posted A- rankings:

• The Trafalgar Group
• Public Opinion Strategies
• CBS News/New York Times
• Mason-Dixon Polling & Strategy
• Public Policy Polling
• Emerson College
• Quinnipiac University

On the other end of the spectrum, 18 firms were at the bottom of the list but a dozen from this group conducted less than 10 surveys. Of those featuring more than 10 polls, the lowest ranked of the group were Nielsen Brothers Polling, the Humphrey Institute, FM3 Research, Opinion Research Associates, McLaughlin & Associates, and Brown University.

Selzer & Company has long been recognized as one of the country’s better pollsters based upon its strong record predicting the Iowa Caucus presidential results and other races most often from the Hawkeye State.

In 2020, 12 of 19 published polls projected Iowa Democrat Theresa Greenfield as leading Sen. Joni Ernst (R). Approaching Election Day, only two pollsters posted the incumbent ahead four or more percentage points: Selzer & Company and Insider Advantage. Sen. Ernst’s final victory margin was 6.6 percentage points. The IA firm came closest to the end result (Ernst, plus-6; actual, 6.6 percent). Surveying for the Des Moines Register newspaper, Selzer & Company again landed within the accuracy realm (Ernst plus-4).

The ratings also included the polling firms’ partisan bias factor. The bias swing in favor of one party or the other ranged from 4.7 points toward the Democrats (Survey Monkey) to a 3.1 sway for Republicans (MRG Research). The overwhelming majority of pollsters with the highest bias rating favored the Democrats. Of the 28 firms that registered a 2.0 or greater partisan bias factor, 26 favored Democratic candidates.

The two Republican firms with the most GOP bias were MRG Research at a plus-3.1, and ccAdvertising, which registered a plus-2.6 skew.

The firms with the most Democratic bias in addition to Survey Monkey were Swayable (plus-4.1, Dems), Nielsen Brothers Polling (plus-3.8, D), RMG Research (Plus-3.5, D), Data for Progress (plus-3.3, D), Civiqs for the Daily Kos Elections site (plus-2.9, D), Morning Consult (plus-2.9, D), and Change Research (plus-2.9, D).

Another dozen polling entities fell between plus-2.8, Dems and plus-2.1 D, inclusive. Within this group of 12 lie four universities: University of New Hampshire (plus-2.4 Democrat), University of Wyoming (plus-2.2, D), Monmouth University (plus-2.1, D), and Brigham Young University (plus-2.1, D).

As you will notice from the chart below, the polling industry results, particularly in the Senate contests, were mixed. While the industry as a whole seemed to correctly project Democratic candidate strength, their misses predominantly under-estimated the Republican totals.

Georgia, which isn’t included in the chart below because there was consistently very little difference in the aggregate polling, was the most accurately polled state. All Peach State pollsters finished within the same range, and close to the final totals in the presidential and two Senate races. The Minnesota Senate campaign wasn’t included on the chart because only one national pollster, Survey USA, tested the state during the final week.

Maine was the pollsters’ worst state. Few polls were conducted, and the ones that were commissioned all projected a substantial Democratic win. In the end, Republican Sen. Susan Collins won easily. Other missed polling states relating to victory margin included Michigan, Montana and South Carolina.

The following is a list of key races and identifies the polling firm with the closest result in the week before the election and the entity that fell furthest away:

President

Senate