The Misuse of Models
Servaas van Bilsen is assistant professor of actuarial science and mathematical finance at Tilburg University (NETSPAR) and the University of Amsterdam. His research interests include pension economics and behavioral finance.
Econometricians love models. They use models to make predictions and to acquire a better understanding of the world. Many people (such as CEOs, journalists and politicians) use the predictions of models to inform the public or make decisions. Therefore, models and their predictions play a prominent role in society. This brings with it a huge responsibility: the shortcomings of models need to be emphasized to prevent social unrest.
The election of the 45th president of the United States of America provides a recent example of the prominent role of models. Almost all opinion polls pointed to a win for Clinton. One of the very few who predicted a Trump win was Professor Elmer Sterken from the University of Groningen. So why were so many predictions simply wrong? What did these polls overlook? Immediately after the election results, numerous experts appeared in the media speculating about the possible explanations: people had not been honest and were afraid to say they would vote for Trump; the voter turnout of some groups (Latinos for example) was lower than in previous elections; ‘angry’ white men were underrepresented in the polls. Many polling agencies will now certainly undergo a period of reconsidering their research methods. ‘How could we get it so wrong?’ A question many agencies will probably ask themselves. However, I am sceptical as to whether new research methods will deliver any better predictions next time. The world is dynamic and something that happened in the past may not happen again in the future. I would argue for a less important role for polls in future elections. Make people aware that polls are just models; explain the potential risks inherent in these models; communicate not only the ‘best estimate’ but also confidence bounds. If the media take on this role, then perhaps major disappointments when the polls turn out to be wrong (such as protests and riots following a presidential election) can be avoided.
Models also play a significant role in the financial industry. Banks and insurers need to manage risk as accurately as possible. The 2008 financial crisis clearly showed that some (or perhaps many) assumptions underlying financial models are far from accurate. At the time, who would have predicted that the bankruptcy of Lehman Brothers, which is a relatively small bank, could have seriously endangered the entire global financial system? Many models clearly missed a critical aspect of reality: the interdependency of financial institutions. So we must start a diligent search for alternative models. One promising alternative is the so-called Hawkes model: a crash in one region of the world (e.g. United States of America) leads to a higher chance of crashes occurring in the same region as well as in other regions (e.g. European Union). The search for better models is both a fascinating and challenging task for econometricians. However, at all times they should make it clear that no model is perfect and will always have inherent shortcomings.
Regarding pensions, models should start to play a different role as well. Currently, pension funds only tell workers what the expected pension payment will be. This gives many the impression that payments are guaranteed. However, the opposite is actually true: pensions are risky and they can even be reduced when pension funds are in financial distress. The fact that pensions are not guaranteed has only recently become clear to the greater public, as many pension funds have lowered their payments in recent years. This has resulted in extremely low levels of trust in pension funds. The trust in pension funds can perhaps only be restored if funds start communicating honestly about pensions. Funds should answer questions such as: what is the risk of my pension being reduced? What is the chance of my pension being 10% lower than the expected pension outcome?
The above stories demonstrate that models play a vital role in society. Econometricians must not only look for the ‘best’ model, but also clearly explain how models should be used and interpreted.
Text by: Servaas van Bilsen
