The crowd consensus forecast was last recorded at nearly a 30% chance. During the time this question was active, forecasters predicted it was more likely that the WHO would not make the declaration – the consensus forecast never went above a 46% chance. 

But, as we know now, it happened. So the question remains, why were we, as a crowd, trending in the wrong direction on this question? (This trend wasn’t just us: Metaculus faced similar issues.) Granted, with a 30% chance of occurring, it’s going to happen three out of ten times, but why wasn’t it higher? 

In short: forecasters didn’t expect Monkeypox. 

It might be because, as a global society, we’ve been laser-focused on COVID-19 for the last several years. COVID-19 has upended our lives and communities. Understandably, some may have thought that it would have been near-impossible to have two pandemics, one after another.

In Nassim Nicholas Taleb’s book, “The Black Swan,” he writes about a turkey. It is fed handsomely, sleeps soundly for the first thousand days of life, and is free to roam within reason. According to the turkey, every day is like the last—so why would tomorrow be any different? On the other hand, the farmer looks at the calendar and sees that Thanksgiving is coming up. On the thousand-and-first day of the turkey’s life, the farmer butchers it. This is a black swan event – an unpredictable event that is beyond what is normally expected of a situation. The turkey would never have known this was coming because, save a few of its turkey friends mysteriously disappearing approximately a year prior, nothing would have prepared it for this day.

For our forecasters, the first 332 days since the launch of the question revolved around one disease: COVID-19. Some focused on the Delta variant, others on Omicron. It wasn’t until May 19, 2022, that the conversation turned to Monkeypox—just a few days after the initial outbreak in the UK. After that, nearly every comment included the word “monkeypox.” 

At the end of the question period, “covid” was the most-used word in rationales. Monkeypox was third, even though it was the eventual reason for the closing of the question:

Even at the end of the question, rationales were roughly split between COVID and monkeypox. Look at the word cloud of all words in the rationales below and see for yourself:

Alright, we’ve figured out how we were wrong here, but how can we shed more light on these black swan events? What lessons can we take away from this? 

There’s a common phrase, “expect the unexpected,” but that is much easier said than done. An excellent way to start is by filling out a pre-mortem. INFER recently introduced a pre-mortem built into the forecasting platform, which serves as an attempt to think about why your forecast could be wrong.

Even then, a pre-mortem might not get us there; there will always be things and events we don’t expect. It is vital to update your forecasts consistently and stay up to date on the news surrounding relevant topics. That way, we can adjust our forecasts quickly when we see the unexpected occurring. In this question, those who updated their forecasts over 10 times had forecasts that were 15% higher than the crowd consensus.

Below is a comparison of the number of updates forecasters made and their forecast, where the red dashed line indicates the crowd consensus:

As we can see, those who updated their forecasts more often predicted a higher chance of a WHO declaration, and thus received better scores. It’s a strong reminder that we need to continuously update our forecasts based on new information.

“Forecasting is difficult,” the great Danish physicist Niels Bohr once said, “especially if it’s about the future.” However, it is not impossible. Doctors in Nigeria saw a resurgence in monkeypox cases five years ago. While it is certainly easier to look back and find evidence of a monkeypox outbreak in hindsight, the information about this resurgence was available at the beginning of the question. 

Looking back, my own forecasts were all about COVID, specifically the Delta and Omicron variants – it never occurred to me that there would be a second global health emergency overlapping with the COVID pandemic. Even if I didn’t know about monkeypox specifically, it would have been beneficial to have thought about the possibility of other outbreaks.

By using pre-mortems, staying up to date on topics, and continuing to hone your forecasting skills through practice, we can better see when black swans are on the horizon. Here are a few other black swan event questions to practice with: