A look back at my predictions for 2022

A look back at my predictions for 2022

Matthew Yglesias — Read time: 10 minutes

A look back at my predictions for 2022

Better than last year!

New episode of Bad Takes is out today all about the debate over Louisa May Alcott’s gender identity.

Each December here at Slow Boring, I like to end the year with both a list of probabilistic predictions for the year ahead and a look at the previous December’s predictions. According to the research presented by Philip Tetlock and Dan Gardner in their book “Superforecasters,” practicing predictions in this way can help you get better at predicting things over time.

And I sure hope that’s true because last December, when I did my first-ever look back, my results were terrible and I’ve been dreading this post all year.

The good news is that this year my forecasts did in fact get better — 75 percent of the things I said would happen with 90 percent confidence actually happened, as did 83 percent of the things I had 80 percent confidence in, 70 percent of the things I had 60 percent confidence in, and 50 percent of the things I had 60 percent confidence in.

Remember the goal here is not to get all the predictions right (which would mean only predicting very boring things) but to really nail the calibrations rate. If you say 10 different things have a 70 percent chance of happening, a good calibrations rate would mean you’d see seven of them happen. And since some error is inevitable, what you’d really like to see (and here I failed) is error happening symmetrically in both directions. You can see I suffered here from systematic overconfidence, the columnist’s cardinal sin.

And that’s the main point of the exercise: not that I per se want to become a superforecaster, but that writing these things down with odds attached is a good way of trying to beat back that overconfidence. “If I grab a pair of dice, I probably won’t roll snake eyes five times in a row” and “D.C. probably won’t cancel five days of school for snow this winter” are both predictions I would stand behind, but there’s actually an incredibly large gap between the probabilities associated with these two things. Casual writing tends to elide the difference between “this would be unusual” and “this is spectacularly unlikely.” It also often struggles with efforts to express an idea like “this probably won’t happen, but the odds of it happening aren’t tiny and are rising, and the consequences would be really bad so I’d like you to worry about it.” Attempting to quantify with exact numbers even occasionally is a way of trying to break bad habits.

But to do that, it’s important to examine what went right and what went wrong.

Some predictions that I made about politics

I started off with a troika of predictions about the midterms, all made with high confidence, of which only one came true:

Democrats lose both houses of Congress (90%)

Democrats lose at least two Senate seats (80%)

Democrats lose fewer than six Senate seats (80%)

We’ve discussed the midterms a fair amount in previous columns, but looking back on this, the very early overconfidence about Democratic Senate losses was based on very crude extrapolation from the historical record. The president’s party almost always loses ground in the midterms. The 2022 map, while probably the most friendly Senate map Democrats can get, left them with a bunch of vulnerable seats and zero margin for error. We know that the thermostatic pattern is sometimes disrupted (in 2002, for example), but those disruptions are rare. So I basically reasoned that because thermostatic disruptions are rare, Dems’ odds of holding the Senate were extremely bad.

The right way to think this through would have been to be more specific. If you’d asked me a year ago whether the Supreme Court strike down Roe v. Wade, my forecast would’ve been overwhelming odds of gutting its core protections (à la John Roberts’ proposal in Dobbs) and a greater than 50 percent chance that there would be five votes to rip off the band-aid and formally strike it down. And from there I could have reasoned that there were decent odds that overturning Roe would have a counter-thermostatic effect. I still would have ended up predicting Democrats lose the Senate — Dobbs explains a lot but there was more to it, like good ads from Democrats and bad candidates from the GOP — but I should have been able to reason my way to a more restrained forecast here.

Some better political forecasts:

Nancy Pelosi announces retirement plans (70%)

Stephen Breyer does not retire (60%)

Some version of Build Back Better passes (60%)

Joe Biden is still president (90%)

At least one Biden cabinet-rank official resigns (70%)

Democrats go down at least one governor on net (60%)

Liz Cheney loses primary (80%)

Some version of USICA passes Congress (70%)

I am glad Breyer proved me wrong and only regret not being more confident about Cheney.

One thing I want to point out is that at the time, my “it’s not dead yet” take on Democrats’ reconciliation legislation was moderately contrarian. I think the folks who spent months and months accusing Joe Manchin of acting in bad faith to protect his personal investments in fossil fuels have not publicly reckoned with their own bad forecasts.

I think my incorrect cabinet resignation forecast is fascinating. You don’t get news stories about things that don’t happen, but the level of continuity in the Biden cabinet has been extraordinary. Barack Obama’s cabinet was pretty stable, but by this point in his administration, Peter Orszag, Christina Romer, and Rahm Emanuel had all stepped down from cabinet-rank White House posts, while the only notable Biden change has been in the non-cabinet White House Press Secretary job.

Some predictions about the pandemic

My Covid forecasting was better than my political forecasting. This is perhaps due to good luck, but I think perhaps it’s actually easier to predict things you have a little more emotional and intellectual distance from:

Fewer U.S. Covid deaths in 2022 than in 2021 (80%)

Fewer U.S. Covid deaths in 2022 than in 2020 (80%)

China abandons Covid Zero (70%)

Additional booster shots authorized (80%)

You can start a fight on Twitter right now by going online and tweeting “the pandemic’s over, guys, and has been for a while,” but I think these accurate forecasts are the cash value of what it means for the pandemic to be over. The SARS-CoV-2 virus continues to exist and since it represents an addition to the extant stock of respiratory viruses, it continues to be very bad for the world. Life expectancy at 70 (or older) is just going to be lower going forward than it would have been in a world where the virus didn’t exist. But as a practical matter, the death toll is waning as people enter their twilight years with some built-up immunity from prior infections and booster shots.

The alternative of relying indefinitely on non-pharmaceutical interventions is just not viable, as even Xi Jinping has admitted. But just because “vaxxed and relaxed” is the best we can do in 2022 and 2023 doesn’t make it the best policy — we should be pulling levers to develop better SARS-CoV-2 vaccines (nasal sprays that would block transmission and pan-coronavirus vaccines that wouldn’t be so vulnerable to variants) and stepping-up research on things like virus-killing far-ultraviolet light.

Predictions about the economy

For the second year in a row I underestimated inflation, but less badly this time:

November 2022 year-on-year CPI growth is below 6% (70%)

November 2022 year-on-year CPI growth is above 4% (70%)

The Fed ends up doing more than its currently forecasted three interest rate hikes (60%)

No recession in 2021 (90%)

The unemployment rate stays between 4 and 5% (70%)

Basically, I correctly thought the Fed would hike more aggressively than it was saying, but I incorrectly assumed that would be enough to get us below 6 percent (and I thought unemployment would be higher). I’m going to chalk that up in part to the invasion of Ukraine disrupting world food supplies, but that in turn is just a reminder that bad news can come from any direction and you need to think comprehensively.

I thought it was weird that there was so much recession buzz last winter and felt like that one was kind of a gimme.

There is a lot of negativity bias in people’s processing of the news environment. This leads people to be simultaneously upset about various inflationary conditions like high levels of job openings, labor churn, and understaffing but also panicked about headlines indicating layoffs at various companies. But you have to try to actually integrate these trends in your mind. Nobody wants to get laid off from a cushy white-collar position and need to take a lower-paid retail job, but the fact that all those jobs are open is relevant context for understanding the economic situation of people facing layoffs. It continues to be a historically good time to find a job, and you need to read layoff news in that context.

For broadly similar reasons, I think that right now there is some serious underestimating of the feasibility of securing an economic “soft landing,” but that’s a story for a different piece.

Foreign politics

All of these things happened, except for Orbán losing:

Emmanuel Macron re-elected (60%)

The German traffic light coalition exploits loopholes to get around the constitutional debt brake (70%)

Lula elected president of Brazil (60%)

Viktor Orbán loses power in Hungary (60%)

Sinn Féin becomes the largest party in the Northern Ireland assembly (60%)

On the elections, I was basically just predicting that the polls would hold up, which is more often than not a good bet.

The Germany thing is interesting because I turned out to be right, but the specific sequence of events was different from what I expected. Russia invaded Ukraine and upended the European economy which, among other things, led to the suspension of the constitutional debt brake. But I didn’t think Russia would invade:

Russia does not invade Ukraine (60%)

New U.S. sanctions on Russia (70%)

Saudi Arabia and Israel establish diplomatic relations (60%)

The U.S. and Canada reach an agreement on softwood lumber (70%)

No military conflict between the PRC and Taiwan (90%)

At the time I made these forecasts, Metaculus had the odds of a Russian invasion lower than I did, so I feel okay about getting this one wrong even though it was obviously consequential. At the time, it was clear that Russia was making some kind of threats in the direction of invading, but I don’t think it was clear that an invasion was likely until late January. As I wrote back in early February before the invasion, invading Ukraine was a really bad idea, and I kind of thought Putin would recognize that and pocket some diplomatic concessions instead of wrecking his military.

The sanctions prediction is an example of me actually learning how to get better at predicting stuff. I saw there was a non-trivial chance of an invasion (which would lead to sanctions), but also a decent chance of something short of invasion that still led to sanctioning plus some outside chance of an unrelated dispute, so I put the odds of sanctions higher than the odds of war.

Saudi Arabia and Israel keep getting less and less clandestine about their cooperation, and I thought the Saudis would want to go public about it to gain leverage over the Biden administration, but so far they haven’t. I think maybe the right way to think about it is that a Saudi-Israeli diplomatic breakthrough would play as a “win” for Biden, but the Saudi government doesn’t like Biden and doesn’t want to give him a win.

Meanwhile, it’s just really stupid that this softwood lumber dispute hasn’t been resolved, but sometimes dumb stuff happens.

Uncertainty is annoying

After the midterms, we had the usual round of discourse where some people dunked on Nate Silver for making bad predictions and then other, smarter people countered that 538 does robust calibration checks and they come out well.

But the reason people think these forecasts are bad is that it’s not usual for 538 to say “so-and-so will probably win” and then he loses. But this is, of course, exactly what happens in a well-calibrated forecast. This call in WA-3 in particular got a lot of dunks. But the thing is, there are hundreds of House seats and they’re mostly safe seats — any given cycle will almost certainly have one or two weird “2 in 100” type outcomes. And 30 percent of the people who you say have a 70 percent chance of winning should lose.

The math here isn’t hard, and if you just think of it in terms of gambling it seems totally reasonable — nobody is so good at poker that they win every hand.

But when it comes to big topics of public concern, like who will win the election or what’s happening with the economy, people really want to make and read definitive predictions about what will happen. Even people who say they aren’t interested in predictions tend to implicitly make them, claiming this or that will have catastrophic effects or huge benefits. It’s really hard to admit, especially to yourself, that it’s hard to know what will happen in the future. It’s possible to get better at making predictions, and hopefully I’ll continue to improve. But a big part of doing that well is acknowledging uncertainty rather than insisting on making the correct call all the time. To the best of my knowledge, it’s actually not possible to do that.

Update your profile

Only paid subscribers can comment on this post

Check your email

For your security, we need to re-authenticate you.

Click the link we sent to tka.lee@gmail.com, or click here to sign in.