Nate Silver’s warning about midterm predictions that journalists won't want to hear
After the roller-coaster ride of 2016's election night, have journalists and political junkies learned not to let conventional wisdom substitute for hard knowledge? Nate Silver — the closest thing there is to a celebrity in the arcane field of statistical journalism — is not wildly optimistic.
"Media understanding about probability, margin of error and uncertainty is very poor," Silver said Monday afternoon when I stopped by the Manhattan office of his fivethirtyeight.com for a pre-election chat.
"That led them (in 2016) to be more surprised than they should have been," he said, based on the quite accurate polling numbers that were available.
Silver, who turned 40 this year, was in jeans and sneakers, his hair unruly, as he sat in his office before a whiteboard of incomprehensible — at least to this visitor — phrases and calculations.
Now that we're three weeks away from the midterm elections, Silver is seeing some of the same tendencies in media coverage and social-media chatter that plagued 2016's coverage.
Take the Senate challenge to Republican incumbent Ted Cruz of Texas by Beto O'Rourke, for instance. When the polls had O'Rourke's chances of upsetting Cruz at a 35 percent probability, the media chatter had it as almost a toss-up. Now that those chances have dropped to about 25 percent, the prevailing narrative has downgraded O'Rourke almost to dead-man-walking status.
"That's not a night-and-day change, but that's how it's being talked about," Silver said.
Which leads to his worry about coverage in the days leading up to Nov. 6.
"I get nervous about how people overstate things" he told me.
That, for example, it's "all but inevitable" that Democrats will win control of the House of Representatives or that there's really no way Republicans will lose the Senate.
"Saying (a GOP Senate win) is all but inevitable should signal it's at 98 percent, not 80 percent," which is the reality at the moment, he said.
While it's quite probable — and has become slightly more likely — that we'll see a split decision in Congress, there's a solid chance it doesn't go that way. There's actually a 40 percent chance that both houses of Congress will end up in the hands of one party, Silver said. That's because, in each case, there's about a one in five chance that the less likely outcome will happen: That Republicans will retain the House or that Democrats will win the Senate.
Silver himself doesn't pretend to perfection. He has had his own difficulties with predicting election outcomes, including during 2016.
He infamously stated during the presidential primary season that Trump's candidacy was not to be taken seriously. When proved wrong, he followed up with a thorough self-examination, published on his site, of what went wrong in his thinking. (The short form: He was thinking too much like a political journalist and not enough like a statistician.)
On the plus side, his final-days projections were better than most: He gave Hillary Clinton roughly a 7 in 10 chance of winning, with Trump at about 3 in 10.
And he doesn't fault the polls, which were quite accurate, he said: "There are lots of reasons the outcome was surprising, but the polls were the least of it."
Midterm elections present their own predictive challenges: There's much less data, with fewer polls to analyze in states and regions than in a national election.
So while many people think they know exactly what's going to happen — and are more than willing to hold forth as if they're infallible — there's still a lot of uncertainty, given the numbers.
"People should not be that surprised by a Democratic Senate or a Republican House," Silver told me. "We're in dangerous territory from a predictive/assumption standpoint."
A four-in-five chance may sound like a done deal, but it simply isn't.
Whether that caution will last through the final days of the campaign and into Election Day itself is another equation altogether.
Uncertainty — the mortal enemy of the chattering classes — is the one thing Silver says is baked into these midterms.
That doesn't mean that the polls are useless or that nobody knows anything. It just means that probability is one thing and inevitability is quite another.
Stories that may interest you
The government should not base its regulations on the assumption nuclear plants will be run by good actors. And reducing inspections and other requirements now seems like a particularly bad idea.
Democratic contenders should consider a pledge to restore constitutional balance.
If the attorney general won't defend competition in banking in Connecticut, no one else will.