Paul Pfleiderer has done a public service in calling attention to
The Misuse of Theoretical Models in Finance and Economics (here)
h/t to William Briggs for noticing and linking
He coins the term “Chameleon” for the abuse of models, and explains in the abstract of his article:
In this essay I discuss how theoretical models in finance and economics are used in ways that make them “chameleons” and how chameleons devalue the intellectual currency and muddy policy debates. A model becomes a chameleon when it is built on assumptions with dubious connections to the real world but nevertheless has conclusions that are uncritically (or not critically enough) applied to understanding our economy. I discuss how chameleons are created and nurtured by the mistaken notion that one should not judge a model by its assumptions, by the unfounded argument that models should have equal standing until definitive empirical tests are conducted, and by misplaced appeals to “as-if” arguments, mathematical elegance, subtlety, references to assumptions that are “standard in the literature,” and the need for tractability.
Chameleon Climate Models
Pfleiderer is writing about his specialty, financial models, and even more particularly banking systems, and gives several examples of how dysfunctional is the problem. As we shall see below, climate models are an order of magnitude more complicated, and abused in the same way, only more flagrantly.
As the analogy suggests, a chameleon model changes color when it is moved to a different context. When politicians and activists refer to climate models, they assert the model outputs as “Predictions”. The media is rife with examples, but here is one from Climate Concern UK
Some predicted Future Effects of Climate Change
- Increased average temperatures: the IPCC (International Panel for Climate Change) predict a global rise of between 1.1ºC and 6.4ºC by 2100 depending on some scientific uncertainties and the extent to which the world decreases or increases greenhouse gas emissions.
- 50% less rainfall in the tropics. Severe water shortages within 25 years – potentially affecting 5 billion people. Widespread crop failures.
- 50% more river volume by 2100 in northern countries.
- Desertification and burning down of vast areas of agricultural land and forests.
- Continuing spread of malaria and other diseases, including from a much increased insect population in UK. Respiratory illnesses due to poor air quality with higher temperatures.
- Extinction of large numbers of animal and plant species.
- Sea level rise: due to both warmer water (greater volume) and melting ice. The IPCC predicts between 28cm and 43cm by 2100, with consequent high storm wave heights, threatening to displace up to 200 million people. At worst, if emissions this century were to set in place future melting of both the Greenland and West Antarctic ice caps, sea level would eventually rise approx 12m.
Now that alarming list of predictions is a claim to forecast what will be the future of the actual world as we know it.
Now for the switcheroo. When climate models are referenced by scientists or agencies likely to be held legally accountable for making claims, the model output is transformed into “Projections.” The difference is more than semantics:
Prediction: What will actually happen in the future.
Projection: What will possibly happen in the future.
In other words, the climate model has gone from the bookshelf world (possibilities) to the world of actualities and of policy decision-making. The step of applying reality filters to the climate models (verification) is skipped in order to score political and public relations points.
The ultimate proof of this is the existence of legal disclaimers exempting the modellers from accountability. One example is from ClimateData.US
Disclaimer NASA NEX-DCP30 Terms of Use
The maps are based on NASA’s NEX-DCP30 dataset that are provided to assist the science community in conducting studies of climate change impacts at local to regional scales, and to enhance public understanding of possible future climate patterns and climate impacts at the scale of individual neighborhoods and communities. The maps presented here are visual representations only and are not to be used for decision-making. The NEX-DCP30 dataset upon which these maps are derived is intended for use in scientific research only, and use of this dataset or visualizations for other purposes, such as commercial applications, and engineering or design studies is not recommended without consultation with a qualified expert. (my bold)
Conclusion:
Whereas some theoretical models can be immensely useful in developing intuitions, in essence a theoretical model is nothing more than an argument that a set of conclusions follows from a given set of assumptions. Being logically correct may earn a place for a theoretical model on the bookshelf, but when a theoretical model is taken off the shelf and applied to the real world, it is important to question whether the model’s assumptions are in accord with what we know about the world. Is the story behind the model one that captures what is important or is it a fiction that has little connection to what we see in practice? Have important factors been omitted? Are economic agents assumed to be doing things that we have serious doubts they are able to do? These questions and others like them allow us to filter out models that are ill suited to give us genuine insights. To be taken seriously models should pass through the real world filter.
Chameleons are models that are offered up as saying something significant about the real world even though they do not pass through the filter. When the assumptions of a chameleon are challenged, various defenses are made (e.g., one shouldn’t judge a model by its assumptions, any model has equal standing with all other models until the proper empirical tests have been run, etc.). In many cases the chameleon will change colors as necessary, taking on the colors of a bookshelf model when challenged, but reverting back to the colors of a model that claims to apply the real world when not challenged.
A model becomes a chameleon when it is built on assumptions with dubious connections to the real world but nevertheless has conclusions that are uncritically (or not critically enough) applied to understanding our economy. Chameleons are not just mischievous they can be harmful − especially when used to inform policy and other decision making − and they devalue the intellectual currency.
Thank you Dr. Pfleiderer for showing us how the sleight-of-hand occurs in economic considerations. The same abuse prevails in the world of climate science.
Paul Pfleiderer, Stanford University Faculty
C.O.G. Miller Distinguished Professor of Finance
Senior Associate Dean for Academic Affairs
Professor of Law (by courtesy), School of Law
Footnote:
There are a series of posts here which apply reality filters to attest climate models. The first was Temperatures According to Climate Models where both hindcasting and forecasting were seen to be flawed.
Others in the Series are:
Sea Level Rise: Just the Facts
Data vs. Models #1: Arctic Warming
Science Matters 
One comment