NYT Polls Defeat NYT Political Analysis

NYT Polls Defeat NYT Political Analysis

Luppe B. Luppen — Read time: 3 minutes

NYT Polls Defeat NYT Political Analysis

A case study of narratives overwhelming the data.

An excerpt from NYT’s House of Representatives election results page.

About two weeks ago, the New York Times published an article that made a mess of polls it had conducted in four House races around the country. Its dour lede painted the results as “fresh evidence that Republicans are poised to retake Congress.” The paper’s analysis was saturated with gloom:

President Biden is unpopular everywhere. Economic concerns are mounting. Abortion rights are popular but social issues are more often secondary.

A new series of House polls by The New York Times and Siena College across four archetypal swing districts offers fresh evidence that Republicans are poised to retake Congress this fall as the party dominated among voters who care most about the economy.

The only issue was the polls it had conducted contained no such fresh evidence. While historical experience and the prevailing pre-election conventional wisdom certainly held that Republicans were poised to collect the traditional bounty of seats that tends to come to an out-of-power party in the midterm elections, the actual data collected from the “four archetypal swing districts” by Siena College contained not one scintilla of confirmation for those priors. As the article acknowledged dismissively in its third paragraph, none of the polls showed a Republican in the lead. Indeed, some of them were not very close.

The topline poll results Siena obtained:

Kansas-3: D+14

Nevada-1: EVEN

New Mexico-2: D+1

Pennsylvania-8: D+6

As of this writing on Wednesday morning after the election, these polls look prescient. Where things stand:

Kansas-3: Sharice Davids (D) 54.7 - Amanda Adkins (R) 43.0, D+12

Nevada-1: Dina Titus (D) 50.3 - Mark Robertson (R) 47.4, D+3

New Mexico-2: Gabe Vasquez (D) 50.3 - Yvette Herrell (R) 49.7, D+0.5

Pennsylvania-8: Matt Cartwright (D) 51.2 - Jim Bognet (R) 48.8, D+2.4

Kansas-3 is the only race to be called for the Democrat so far.

On social media, the NYT’s polling svengali Nate Cohn had called the Kansas-3 poll, which showed the Democrat leading by double digits, a “possible outlier.” He allowed that the poll vastly diverged from his preconceptions.

But there was no reason given, other than conflict with those preconceptions, for why the Kansas poll should be discounted. The Democrat it showed to be leading, Sharice Davids, was the incumbent, after all. Davids had won her 2018 race—flipping a previously Republican-held seat—by a comfortable 10 point margin. Then she had maintained that margin of 10 points in a 2020 contest against her opponent this year, Amanda Adkins. NYT/Siena had even polled the 2018 race, as Cohn acknowledged, and been surprised to find Davids leading the incumbent by 8—a forecast that predicted the election result with reasonable accuracy. It was certainly possible that Davids would suffer a reversal of fortune this year, with an unpopular president and painful levels of inflation, but a poll that showed her holding strong and improving on her performance in prior races should not have been lightly brushed off. That misconception carried straight through to election night, when NYT put her district in the column of races that Democrats were “expected to win narrowly.” As of this writing, that column is littered with blowouts.

In truth, all four of the NYT/Siena polls pointed to a remarkable, and largely unanticipated fact: Despite what appeared to be unfavorable political climate, Democrats were holding their own in House elections in all the “archetypal swing districts” the surveys had measured. The Democrats weren’t simply strong in a handful of unusual places, as Cohn argued when the article came under significant criticism. The Republicans didn’t appear to be breaking through anywhere they needed to. That could and should have an important finding, but instead it was overwhelmed by the newspaper’s preconceived narrative of Democratic weakness.