The Numbers Game
Society revolves around numbers. Whether we look at our exam results, our weight on a scale, or our income on a payslip, numbers are all around us. We even use numbers to quantify various abstract concepts: gross domestic product (GDP) to measure our economy, IQ scores to measure intelligence, and polls to measure the public opinion. All for good reason, because such measures help us to understand the world around us. They give us a grip on even the most abstract concepts, by looking at them from an objective and factual point of view: numerically. Well, that often seems to be the general consensus. However, using numbers is not quite as objective as we are often led to believe. In this article, I am going to elaborate on the (mis)use of numbers by considering several examples, and explain why we should question numbers just as much as we question words.
Written by: Ridho Hidayat
Using statistics
To start things off, we take a closer look at every econometrician’s favorite subject: statistics. This field of mathematics provides us with powerful tools to analyze data samples, which can be useful to infer conclusions about a given population. While these techniques are surely important, the philosophy of statistics is often undervalued. Most people are mainly interested in point estimates, confidence intervals and finding statistically significant differences. However, if we forget to think about the meaning, justification, use and misuse of statistics, we can easily end up misinterpreting the results.
Consider the following example: “The average income of group A is higher than the average income of group B.” What can we infer from this statement? A common misconception is that this statement gives information about every individual in these groups, for instance that the people in group A are better off than the people in group B. However, all we know for sure is that at least one person in group A has a higher income than the average income of group B, and at least one person in group B has a lower income than the average income of group A. It is possible that if Bill Gates is in group A, that on average every person in that group is a millionaire, while actually everyone besides Bill Gates is extremely poor. This is just a simple example, but it can become more controversial if, for instance, you look at IQ instead of income.
In 1994, Charles Murray and Richard Herrnstein published The Bell Curve, in which they stated that on average black Americans have a lower IQ than white Americans [6]. While this statement holds up in terms of test scores [3], we have to think carefully about the interpretation. For instance, can we conclude that black people are genetically less intelligent? Definitely not, and here is why.
First of all, to get a more complete picture of the test scores, take a look at the distributions that can be seen in Figure 1. It shows that there are indeed a lot of white Americans with a higher IQ than black americans, but there are also a lot of black Americans with a higher IQ than white Americans. Nothing shocking so far, but there is something much more important: intelligence is not the same as IQ. While this seems obvious, it is a distinction that is often blurred. On one hand, intelligence is the ability to acquire and apply knowledge and skills. On the other hand, an intelligence quotient (IQ) is a total score derived from several standardized test to assess human intelligence. It is an approximation of human intelligence, which is a very abstract concept, and there is a lot more to this IQ score than meets the eye, which shows that numbers are not purely objective.
IQ tests primarily measure an abstract way of thinking, such as completing series of numbers, understanding metaphors, and spatial reasoning. For many of us, it is natural to relate these skills to intelligence, because they play a dominant role in our society. A lot of modern jobs require such abstract thinking. However, is this truly what intelligence is about? The answer to that still remains a value judgement, it is subjective. If you would go back in time and ask some tribe what determines intelligence, they probably give completely different answers, skills such as hunting animals, cooking, or building houses.
Even if we compare our test results to those from the people who lived a hundred years ago, the results are already striking. Using their norms, the people today would have an average IQ of 130, and using our norms, our ancestors would have an average IQ of 70 [5]. However, this does not mean that we are all gifted, or that our ancestors were all mentally challenged. Over the years, this abstract way of thinking has become more important, which also resulted in a shift in education. Gradually, we developed these skills, and we therefore became better at these IQ tests. The social environment has a major impact on IQ. Going back to the American example, besides an income inequality between white and black Americans, there is a difference in the quality of the schools in the neighborhoods they live in, which is likely to have an effect on their ability to think in an abstract way.
In the end, IQ remains a single number based on several countable or measurable aspects of thinking. We sum up these separate intellectual qualities, which is already questionable, and we have to make a selection of the things that we include. Even just for our abstract thinking, we can never take everything into account. For instance, the creativity of a solution, the process of learning a new language, and dealing with mistakes, are all aspects that are not measured.
So should we stop using IQ tests? Certainly not, as they also serve a valuable purpose, they can help us to understand people. In fact, the IQ test was initially designed to determine which children needed special education. However, it is important to note that there are underlying values hidden in these numbers. Capturing some abstract concept in a single number can give an insightful and clear approximation, but things can get real ugly once we stop distinguishing this approximation from reality. Concluding that black Americans are less intelligent than white Americans based on IQ scores is a flawed way of thinking. Stating that there is a genetic difference in intelligence is even less substantiated, to say the least.
The climate change debate
Now that we have discussed the underlying values of numbers, we can take it one step further and ask ourselves: How can we question a number? For instance, numbers are often presented in the news, or used as arguments in debates. Knowing that they are not purely objective, how can we distinguish between fact and opinion? First of all, by itself, there is nothing wrong with using numbers in the news, or as arguments. They can definitely give valuable insights into a subject and contribute to a better comparison of two sides of a debate. However, we have to be careful that a news item or a debate does not turn into a numbers game. If the numbers are all that matters, we do not look at the story behind them. Once that happens, we are bound to make crucial mistakes. To give an example, we take a look at the climate change debate…
The scientific world has sounded the alarm, governments need to act to address climate change. A radical change of direction is needed: greenhouse gas emissions have to be mitigated to reduce global warming, or else the world is doomed.
Such threatening words have fueled the debate on climate change for years. The problem: how can we know for sure that these scientists are right? Mitigating greenhouse gas (GHG) emissions costs billions of dollars, what if we spend all this money for no good reason? The solution may sound simple, just look at the numbers. That is also what happened when the Dutch cabinet proposed that GHG emissions in the Netherlands should be reduced by 49% in 2030 compared to 1990. The numbers were clear: according to mathematical models, this reduction would lead to a decrease in temperature by 0.0003°C this century [8]. As a result, several opposition parties stated that the proposal should not be accepted. If all this effort results in such a small change in temperature, then clearly people do not have a significant effect on climate change. Reducing GHG emissions is a waste of money, right? There are a lot of problems with this train of thought. Let us take a look beyond the number.
1.Transparency
To begin with, whenever a number is presented, ask the question: where did the number come from? How was it measured or calculated? Here, there is hardly any transparency in the methodology that is used to calculate this 0.0003°C decrease in global average temperature. Regardless of whether the outcome is true or not, the data, models and assumptions that are used to come to this result should be stated clearly. They are the foundation on which this number is built, and without it, the number falls apart.
2. Context
Furthermore, we should ask the question: in what context should we interpret this number? What is the context of this 0.0003°C decrease in temperature? Without any context, people tend to rely on their gut feeling. It is a number close to zero, a difference in temperature which you cannot feel, so it is probably insignificant. However, if we look at the bigger picture, we can put things into perspective. In 2016, 195 members of the United Nations Framework Convention on Climate Change (UNFCCC) signed the Paris Agreement. Under this agreement, each country, including the Netherlands, must contribute to mitigate global warming. Various studies ([9], [11]) show the causal effect of carbon dioxide (CO2) on global temperature. With 0.35% of global emissions, the Netherlands’ share of GHG emissions may not be large, and hence the effect of mitigating these emissions may not be large, but it does play a more important role in reinforcing the Paris Agreement.
Furthermore, we should ask the question: in what context should we interpret this number? What is the context of this 0.0003°C decrease in temperature? Without any context, people tend to rely on their gut feeling. It is a number close to zero, a difference in temperature which you cannot feel, so it is probably insignificant. However, if we look at the bigger picture, we can put things into perspective. In 2016, 195 members of the United Nations Framework Convention on Climate Change (UNFCCC) signed the Paris Agreement. Under this agreement, each country, including the Netherlands, must contribute to mitigate global warming. Various studies ([9], [11]) show the causal effect of carbon dioxide (CO2) on global temperature. With 0.35% of global emissions, the Netherlands’ share of GHG emissions may not be large, and hence the effect of mitigating these emissions may not be large, but it does play a more important role in reinforcing the Paris Agreement.
Therefore, taking into account this context, we look at the effects of all planned actions combined. This can be seen in Figure 2, which shows the development of global GHG emissions in the future [10]. Several scenarios are compared, looking at the countries’ Intended Nationally Determined Contributions (INDCs), which are their planned actions following from the Paris Agreement. For these scenarios, the effect on global warming is estimated, which can be seen in Table 1. Given this context, it can be seen that the global effect is much larger. Also, the Paris Agreement sets targets for the year 2100, so if only the actions until 2030 are taken into account, we miss seventy years of progress. So just like words, numbers should be interpreted in the right context.
3. Importance
Another question we should ask ourselves with any number: is it actually important? Is it a valid argument in the discussion, or is it a just a fallacy? In the case of the climate change discussion, if the decrease in global temperature is small, does that mean the money spent is wasted? To answer this question, we need to review the impact of climate change. A quick Google search on “meta-analysis on climate change effects” shows millions of results, with studies from various angles. For instance, Figure 3 shows the estimated damage as percentage of GDP for varying temperature changes, based on different models [7]. Apart from the economic impact, studies on ecological impact show numerous effects, such as coral reef bleaching, diseases, and extinction risk for a variety of species [12]. Stating that the effects of reducing GHG emissions are small, and that these actions are therefore a waste of money, undermines the actual problem: without change, life in general is at risk. So even if a large investment is needed for a small change, it can be seen as the start of a necessary snowball effect.
4. Consensus
Even after looking at all these studies, we come back to the initial problem of the climate change debate: how can we know for sure that these scientists are right? Methodological mistakes may lead to wrong conclusions, and even with a correct approach estimations can turn out to be wrong, due to random variation. Often, there are also different studies with conflicting conclusions. Scientists on either side of a debate can even have an incentive to come to a certain conclusion, causing a conflict of interest. Perhaps they have a strong opinion on an issue, or they simply get money for it. Still, we want to make decisions based on the information that is available, so how do we choose what to believe? The answer to that is scientific consensus. If the vast majority of scientists, all with different backgrounds, different data and different methodologies, come to the same conclusion, then we know enough to believe in that conclusion. Science is not about a bunch of separate studies, but about the collection of studies as a whole. If the evidence for a certain conclusion is overwhelming, then a few studies with contradictory results are not going to undermine that conclusion.
On the topic of climate change, several studies aim to measure the scientific consensus. In Figure 4, we can see the results of a survey asking the question: Do you think human activity is a significant contributing factor in changing mean global temperatures? [4] Among the respondents, about 90 percent of all active publishers and climatologists agree that this is the case. Different studies even report a much higher consensus on anthropogenic global warming (AGW) [2]. Of course, we cannot be completely sure if this is the correct conclusion, but uncertainty is the nature of science. In this case, we can say that there is enough evidence for AGW. It is important to keep that in mind, because people can take advantage of uncertainty. Doubt becomes a product. Claiming that the dangers of climate change are unproven can sway the public opinion against the need for action. The same strategy was used in the tobacco industry: companies cast doubt on the ill-effects of tobacco to generate sales [1].
The four aspects we just discussed only form the basis of questioning the numbers we encounter. It is a starting point, and from here we can learn to look beyond the numbers. By learning more about numbers, we can start making up our own minds, instead of carelessly believing every number that is presented. As you may have noticed, I also present some numbers myself in this article. Therefore, I encourage you to challenge my numbers, and look for the complete picture.
Keep numbers in check
To conclude, there is still a lot to discuss about the use and misuse of numbers, such as the difference between correlation and causality, and the impact of big data and machine learning algorithms. However, all that is beyond the scope of this article. The goal of this article was to bring awareness that we should keep numbers in check. We should not see numbers as the undisputed truth, or as synonyms for reality. Instead, we should place them on the same level as words: questioned, and discussed. There is a whole story behind every number we see, and therefore I would like to advise everyone to go down the rabbit hole. Follow your curiosity and challenge your beliefs. Society does not revolve around numbers, numbers revolve around society.R
References
[1] – Bates, C., and Rowell, A., “Tobacco Explained… The truth about the tobacco industry… in its own words”, Center for Tobacco Control Research and Education (2004)
[2] – Cook, J., et al., “Quantifying the consensus on anthropogenic global warming in the scientific literature”, Environmental Research Letters (2013)
[3] – Dickens, W. T., and Flynn, J. R., “Black Americans Reduce the Racial IQ Gap Evidence From Standardization Samples”, Psychological Science (2006)
[4] – Doran, P.T., and Zimmerman, M. K., “Examining the scientific consensus on climate change”, Eos (2009)
[5] – Flynn, J. R., “What is intelligence?”, Cambridge University Press (2007)
[6] – Herrnstein, R. J., and Murray, C., “The Bell Curve: Intelligence and Class Structure in American Life”, Free Press (1994)
[7] – Howard, P. H., and Sterner, T., “Few and Not So Far Between: A Meta-analysis of Climate Damage Estimates”, Environmental and Resource Economics (2017)
[8] – Keulemans, M., “Nederlands klimaatbeleid scheelt maar 0,0003 graden opwarming – klopt dit wel?”, De Volkskrant (2017)
[9] – Lacis, A. A., Schmidt, G. A., Rind, D., and Ruedy, R. A., “Atmospheric CO2: Principal Control Knob Governing Earth’s Temperature”, Science (2010)
[10] – Rogelj, J., et al., “Paris Agreement climate proposals need a boost to keep warming well below 2 °C”, Nature (2016)
[11] – Stips, F., Macias, D., Coughlan, C., Garcia-Gorriz, E., and Liang, X. S., “On the causal structure between CO2 and global temperature”, Nature (2016)
[12] – Walther, G.-R., et al., “Ecological responses to recent climate change”, Nature (2002)
Inspired by Sanne Blauw, “Het best verkochte boek ooit (met deze titel)”, de Correspondent (2018)
