+
Looking for some inspiration?
Check out the newly released book.
"Lead with Empathy and Innovation:
Transform Your Organization Today."
More information"Lead with Empathy and Innovation:
Transform Your Organization Today."
I can’t describe how much I would like to take a look into the crystal ball right now and see how the next three, five, ten years will develop. Who wouldn’t? The world around us has changed so much that our old knowledge, insights and information no longer seem applicable when it comes to the further development of our business. When will normal operations be possible again? Will the needs of the customers still be the same? Who will the future customers actually be? And do we have to prepare for the fact that short-term shutdowns in our society will become part of everyday life?
Finding the answer to the emotional, long-term impact of the pandemic on humans is the biggest challenge we are facing now. The foundations of Maslow’s pyramid have been deeply shaken and many of us have fallen down a few steps quite simply. Our values are currently reshaping and we need to review the long-term effects and (fundamentally) revise our business models. Just think how strange the wearers of face masks were found to be just a few weeks ago. Soon, those who don’t wear one could be the exotics – just as smokers once experienced: “It’ll be like smoking in a restaurant: It will quickly go from causing outrage when people want to stop it to suddenly causing outrage if somebody does it.” Keith Chen, Associate professor of economics, UCLA.
One would love to be one of these Superforecasters, who find an answer to all these questions. It seems too difficult to learn the art of making good predictions. But predicting the probability of certain events is no magic. You can train yourself for this ability and even as a non-expert you can take up with the great analysts of this guild.
Philip E. Tetlock and Dan Gardner have dedicated a very exciting book to the subject “Superforcasting” a masterpiece on prediction, based on years of research and a competition in which hundreds of ordinary people competed against the elite forcasters: The Good Judgement Project. They have analyzed what makes success and failure and supplemented it with exciting interviews with the policymakers.
They prove that foresight is not a gift but a quality that can be trained. For everyone who wants to predict how the world will change or understand the probability of certain events I recommend reading this book. For those who want to get a sneak preview of the content, here are my main take-outs for becoming a future Superforecasters, which are discussed in detail in the book. But don’t forget: At the end it’s all about training.
Archaeologists find artifacts from the past and try to connect the dots and Superforecasters do the same. You just look into the future instead. You use artifacts from the present and past (news, statistics, studies, …) and connect the dots to calculate the probability of certain events in the future. Of course, you have a methodology and basic rules to follow. But that is all. The most important quality to bring along is a willingness to accept that everything is perpetual beta. This commitment to ongoing updating and self-improvement is roughly three times more important than pure intelligence.
Modelling a Superforecaster
You should make sure that you either bring the following qualities with you or add partners who can truly represent them and fill your blind-spots:
Philosophic Outlook
Abilities
Approach to Work
Attitude
If you can tick off the above boxes either by yourself or with the help of your team, you have laid the perfect foundation for the work of a Superforcaster.
No matter if Superforcaster or not, your first exposure to forecasts is their evaluation. An unclear choice of words opens the door to misunderstandings. Or, alternatively, they provide a protective cloak for unclean work. Let’s take as an example the often-quoted statement by Steve Ballmer in 2007, in which he did forecast no significant market share for the iPhone. He is often mocked for this today:
“There’s no chance that the iPhone is going to get any significant market share. No chance.”
Often this has been rated as one of the 10 biggest misperceptions about technology. But let’s pause for a minute and consider his words again. The key message here is “significant market share.” What does “significant” mean? In the US, worldwide and in which market? Smartphones or mobile phones in general? And by when? This is pretty much the biggest problem with superforcasting. Inaccurate use of language.
Considering the context of this statement, it is made in the context of the global mobile phone market. So, it would be wrong to consider this only in the context of the smartphone market, which was just emerging at that time.
The next would be “by when”. In a nutshell: If you look at the year 2013, when Ballmer was scolded for this statement in connection with his resignation from Microsoft, the mobile market share of these devices was about 6% according to Gartner. Well, you judge if this is significant or not.
We learn how important it is to understand the full context of predictions, how left out information can mislead us and how important it is to choose clear words. Tetlock even goes one step further and suggests that we combine percentage probabilities with an exclusive choice of words:
Certainty The General Area of Possibility
100% Certain
93% (give or take about 6%) Almost certain
75% (give or take about 12%) Probable
50% (give or take about 10%) Chances about even
30% (give or take about 10%) Probably not
7% (give or take about 5%) Almost certainly not
0% Impossible
This would be one of the first things you should learn. The use of clear language.
If you want to become a successful Superforecasters your focus should always be on the really hard questions and you should not spend time on those that can be answered by simple rules of thumb. It is best to focus on those that don’t feel too distant but far enough where enough variables exist for forecasting analysis. For example, answering the question of who will win the US elections in 2020 seems difficult but feasible. On the other hand, to answer who will win in 2024 or even 2028 is almost impossible.
Once you picked up your question the next most important thing is to make sure it’s clear and precise.
Bad: “When will we have more than 50% Electric Vehicles?” – this is by far not precise enough.
Better: “Will we have more than 50% privately owned Electric Vehicles driving on the streets of Europe by 2030?” – clarity about what kind of EV, location and time.
Next you divide the problem into its knowingly and unknowingly parts. Of course, there will be many questions you will not be able to answer due to a lack of available studies or information. Don’t worry. You will be fascinated how often rough estimates lead to a relatively valid result (see also Enrico Fermi and his Fermi questions). The discipline comes from finding small enough questions that allow for directionally accurate “best guesses”. This method is known as Digression.
Digression
In his book, for example, Tetlock describes a calculation based almost entirely on guesswork:
Question: “How many piano tuners are there in Chicago?”
He starts by breaking it down into sub-questions this way:
How many Pianos are there in Chicago?
Best guess: There are 50.000 pianos in Chicago
How often pianos are tuned each year?
How long does it take to tune a piano?
How many hours a year does the average piano tuner work?
The rest is math: 50.000 pianos x 2 hours tuning = 100.000 hours of tuning divided by 1.600 hours per tuner = 62.5 piano tuners in Chicago.
The reality? 83. But hey, for not having a clue, it’s incredibly close. And just consider, if you had done some research. So be brave and add guess work where you have empty spaces.
Superforecasters often tackle questions in similar ways. They start with the Outside View first (draft foundation, asking good questions and breaking them into knowable and unknowable parts) and add the Inside View (hypothesis and events that need to occur).
Let’s return to our “better” question from earlier and apply an Outside and Inside View:
“Will we have more than 50% privately owned Electric Vehicles driving on the streets of Europe by 2030?”
Foundation (Outside View)
Be creative in your research and try to compare it with past events where possible. Nothing is really new. How often do events you want to investigate take place in this way? And how?
Hypothesis (Inside View)
Here you start transforming your foundation into underlying hypotheses for the event to occur.
E. g. based on your foundation you are assuming for 50% privately owned Electric Vehicles driving on our streets of Europe by 2030 the following hypothesis:
Turning questions into quantified forecasts
Finally, you have to make an assessment of how important the individual points are for the main event to arrive. This is your best guess based on the overall foundation.
Probability
Next you break the single hypothesis down into a series of testable short-term indicators. Make sure that you cover a combination of logic and psychologic events. E. g.
Targets set by the authorities (here you would feed in announcements like California wanting to be carbon neutral by 2045, Europe by 2050, …)
General image of cars / car drivers (Greta effect, share of young adults with a driving license, …)
Based on the analysis of these short-term events you then determine the probability of each hypothesis:
Converting vague formulations into numbers may feel unfamiliar at first and requires a lot of patience and practical practice. But it is feasible and THE tool of the Superforecaster. Think in more granular ways about uncertainty and reduce complex hunches into scorable probabilities like we have done before. The more shading of probabilities you can develop for yourself, the better. A “maybe” is not really differentiating enough. Those who can describe the chance in 65:35 instead of 60:40 are clearly at an advantage. But you have to strike the right balance between under- and overconfidence, between prudence and decisiveness. Understand the risks of making a final judgement too quickly or spending too much time with the “maybes”.
The rest is a simple rule of three:
Hypotheses 1 Share x Probability + Hypotheses 2 Share x Probability + Hypotheses 3 Share x Probability + Hypotheses 4 Share x Probability + Hypotheses 5 Share x Probability = 74%
Prediction
It is probable that we will have more than 50% privately owned Electric Vehicles driving on the streets of Europe by 2030. But it is neither almost certain nor are the chances about even.
The final prediction requires both regular calibration and weighting of individual events which can lead to the revision of the hypothesis. So, don’t be afraid to prove you have been wrong in the first place. It is better to discover errors quickly than to hide behind a lot of words and self-fulfilling research.
Collaborate
Regularly checking your own hypotheses is as important as flossing the spaces between your teeth every day. It can be boring and sometimes annoying, but in the long run it pays off. What you need to find is the right balance between over- and under-reacting. Filtering the important information out of the jungle of messages and not being fooled by wishful thinking is very hard work.
This is the moment where disruption and diverse perspectives make a lot of sense (if you haven’t had the chance to incorporate it in your team from the beginning). Three, Four perspectives for aggregation are a great driver of forecast accuracy. So, try to collaborate while you calibrate.
1. Check your Superforecaster Modelling from the beginning and fill the blank spots.
2. Check your event list and consider, who could help you gain a fresh perspective.
For example:
General image of cars / car drivers (Greta effect, share of young adults with a driving licence, …)
Here it would be useful to integrate someone with a good insight into the soul of the young adults. For example, how would a product designer of smartphones view the development of the values, motivations and needs of this target group?
Superforecasting works best in teams. A group of assembled experts given an open, collaborative and curious environment bring out the best in each individual and in the total team. Different knowledge and questions as well as constructive discussions help to gain clarity. I can confirm that this has been one of the core elements of the and dos Santos success in the last years – letting experts from various fields amalgamate their expertise.
Another reason for collaboration is the identification of sporadically occurring opposing forces.
No argument without a counter-argument worth considering: Thesis meets antithesis and forms the synthesis. But the challenge for a Superforecaster is much more complex, because you have to deal not just with one but with a variety of possible synthesis along the way. Synthesis here is an art that requires relentlessly reconciling subjective judgements. The danger is that the media and momentary phenomena can make you blow your horn too early or you might overhear an approaching storm. Having the opportunity to play this game with experts from various fields makes your work much easier.
That is it. After all, it’s about trying again and again. All theory is nothing without practical experience with good feedback loops to understand your progress in analysis and evaluation. Superforecasting is the product of deep practice. Look for the errors behind your mistakes, but beware of the distortions in the rear view mirror. Do not try to justify or excuse failures. Face up to them. Perform sober postmortems. Learn and find the errors in your basic assumptions. In addition to performing postmortems about failures, it is important to do so for your victories, too.
I hope this has given you some inspiration for your predictions – maybe I have sparked some interest and you’ll want to take a closer look at this fascinating topic.
If you would like to discuss one or the other point in detail, have questions about our experience with the topic, or if you are looking for support in applying Superforecasting for you, we at … and dos Santos are happy to support you. With the help of our opinion leaders from the fields of business, science, technology and art, we can help you to build a tailor-made Superforecasting team for your needs. Please do not hesitate to contact me to discuss the details without binding obligation. Just send me a personal message.
Keep curious. Keep discovering.
