In order to get a good forecast, you have to start with a good question. Each individual question must be specific and falsifiable, and the questions you ask should complement each other and give a deeper understanding of a broader topic or goal. As a result, it isn’t always easy to figure out what these questions should be.
Additionally, the types of people who consume forecasts don’t always want the same thing. Some are analysts who love getting into the nitty gritty of individual forecast question results. Others are high-level decision makers who think at a big picture level. To serve these different populations with forecasts that are both specific and actionable, the INFER team uses a process called “Strategic Question Decomposition.”
The origins of decomposition
Strategic Question Decomposition is a process that harmonizes two very different approaches to preparing for the future: scenario planning and probabilistic forecasting. Scenario planning involves identifying key uncertainties, reviewing potential scenarios where some combination of uncertainties comes to pass, and then planning potential courses of action for each. Scenario planning can help you make sense of complex problems, but the intrinsic difficulty in preparing for all possible scenarios can make it too hard to act. Probabilistic forecasting, on the other hand, uses models and data to calculate odds and quantify risk for specific scenarios. However, the farther out you try to predict, the less accurate forecasts become.
Because both approaches have their strengths, Peter Scoblic and Phil Tetlock advocated for combining the two by “developing clusters of questions that give early, forecastable indications of which envisioned future is likely to emerge” in their Foreign Affairs article entitled “A Better Crystal Ball.” A Georgetown think tank and Cultivate partner, the Center for Strategic and Emerging Technologies (CSET), began realizing what Scoblic and Tetlock proposed through their public crowd forecasting site Foretell (now INFER Public), which contributed to the development of this decomposition process. You can read more about the origins of Strategic Question Decomposition in this blog post by Cultivate Labs CEO, Adam Siegel.
What is a decomposition?
A strategic question decomposition, or simply “a decomposition,” is the result of breaking down a big strategic question into smaller parts in order to identify forecast questions that could help us better understand the original question and develop a sense of what’s likely to occur. INFER’s typical decomposition framework depicts the top-down relationship between the strategic question, its contributing factors and sub-factors, and the resulting forecast questions.
- Strategic questions represent the broad categories we want to learn more about. Breaking down a strategic question is the main focus of a decomposition.
- Contributing factors are the primary drivers of the strategic question. They directly influence the answer in one direction or another.
- Sub-factors are the individual elements that make up and influence a contributing factor. Depending on the size and scope of the strategic question, it may be possible to identify signals directly from the contributing factors without the need for sub-factors.
- Signals are specific metrics or events that tell us how a factor or sub-factor is trending.
Pictured: An example decomposition flowchart with the strategic question on the left, followed by its contributing factors and signals.
What happens after forecasts are made?
Once questions have collected forecasts, we can use the decomposition model to synthesize and analyze data from individual forecasts and glean information about how a strategic question might trend. This is recomposition—the process and product of combining forecasts together to provide insight into a broader topic.
Pictured: A framework to enable forecast results to be communicated to decision-makers in an actionable way.
Senior management, data analysts, and other subject matter experts inform our understanding of how different forecasts interact as part of the bigger picture question, and we put all that knowledge and insight together to help drive action. This final recomposition can take many forms, e.g., a dashboard, a summary report, or an index. One way we’ve found to be especially effective at communicating these insights visually is the heatmap.
Pictured: Example heatmap showing forecasting trends for a strategic question.
What’s next
This is the first entry in a series of blog posts accompanying the release of new forecast questions that have been developed using this decomposition process, starting with the topic of U.S. competitiveness in AI. We hope this first post was able to shed light on our process for breaking down strategic questions to develop questions that you can forecast on. The next blog post in this series will give an overview of our decomposition of the National Security Commission on Artificial Intelligence Report’s chapter on Microelectronics.
------
To keep up with the latest INFER updates, follow us on Twitter @INFERpub!