Making your first forecast on a question can be exciting–or possibly a little daunting if you’re not sure where to begin. If you’re just starting out, this guide will provide steps and example prompts that you can use to approach any forecast question for the first time.
1. Start with the outside view; use base rates
Before you try to predict the future, it’s helpful to get an understanding of the past. When making forecasts, a “base rate” refers to the rate of occurrence of a particular outcome in similar situations.
Let’s say you’re forecasting the probability of a certain political party winning a national election. Most people tend to overemphasize what they already know about the topic. For example, thoughts like “the candidates in the opposing party are flawed” or “this party has a ton of resources behind it,” may lead you to a high probability estimate (e.g. 80%) that a certain political party will win.
But, savvy forecasters will start by looking at the outside view/base rates:
Across its entire history, how often has this political party won the national election?
Let's say the answer is 55%.
What portion of elections in the past decade has this political party won?
Let’s say the answer to this question is 30%.
In this case, looking at relevant base rates creates a more useful range (30-55% odds of winning) on which to anchor your forecast. Armed with base rates in hand, you might lower your initial forecast from 80% closer to 50%.
2. Consider the unique qualities of the case at hand
Let's continue with the election example. After identifying base rates, you can now return to evaluating the “inside view,” which is what you know about this election that makes it unique, for example:
Is this party beating the opposition in the average of opinion polls this election cycle?
Answer: Yes
Is the opposition candidate unusually weak or plagued by scandal?
Answer: Yes
Is this party’s candidate unusually controversial?
Answer: No
If all factors are mostly positive for the political party, then you might consider adjusting your forecast back up to about a 55-65% chance of success.
3. Seek out contradictory information
When looking for base rates or evaluating the inside view, make sure to play devil’s advocate with yourself. For example, you might consider reading articles that take the opposing perspective and cast doubt on the political prospects of the party that you forecasted would succeed in the election.
You don’t want to fall for common biases when forecasting. For instance, one such bias is the tendency to favor evidence that confirms a pre-existing belief (confirmation bias), and another is the tendency to be too optimistic (overconfidence bias). Looking for evidence that is contradictory to one’s prediction – whether in the question’s comments, external articles, or one’s own research – can help prevent you from falling for these biases.
4. Summarize your reasoning in your rationale – why you might be right
Be sure to explain your approach to the forecast in your rationale. Rationales provide context that is just as important as the numerical forecasts – it’s valuable to the broader community to help understand trends and differing perspectives.
For example, your rationale might include the relevant base rates you identified of prior elections, factors about this particular race from your “inside-view”, and responses to potential counter-arguments. It is often helpful to link to sources you’ve looked at or even tag other forecasters who provide interesting rationales as well.
You can also think of rationales as a diary you keep over time about your judgments on a particular question. When you return to update a forecast on a question, your initial forecast will often be your best resource in getting reacquainted with the relevant information. Reviewing a past rationale can also help you track any biases or blindspots.
5. Conduct a pre-mortem – why you might be wrong
INFER has an optional section for adding a pre-mortem (a brief explanation of why you might be wrong) alongside the rationale when submitting your forecast. This reinforces the mental exercise you did in Step 3 to seek out contradictory information to your intuition and helps avoid extreme forecasting (i.e. 0% or 100%) when it’s not warranted. You can often uncover a new and helpful perspective by simply asking yourself “if my forecast turns out to be wrong, what factors might have contributed to that unexpected result?”
For example, even if you’ve settled on a forecast that a certain political party has a 100% chance of winning, this is an opportunity to consider even the extreme outcomes: Will the favored leader of the political party become ill and therefore not be able to run for office? Could the polls showing a clear lead for the party be inaccurate?
6. Update your forecast over time
On INFER, you’re encouraged to regularly update forecasts on a question until the close date. Updating your forecasts as you learn new information will help to improve your score on each question and identify potential bias or outdated information in past forecasts.
For example, you might return to the political party question each time you learn of an update to the election. You might simply confirm your forecast – or you might have something brand new to consider. While it’s not fun to be wrong, it is fun to have an opportunity to change your perspective and be closer to getting it right! When you submit your forecast, be sure to use INFER’s built-in reminder feature to help you return to a forecast question regularly to submit any updates you might have.
Research has shown that employing the techniques above and regularly updating your forecasts in small intervals over time leads forecasters to be more accurate. So, keep practicing and happy forecasting!