June 13, 2020

When Do Stories Work? Evidence and Illustration in the Social Sciences


Author: Andrew-Gelman Source: http://www.stat.columbia.edu/~gelman/research/published/storytelling.pdf

Good stories are anomalous. As the saying goes, ‘Dog bites man is not news. Man bites dog is news.’’ When we use atypical stories to develop general understanding, this presents both a risk and an opportunity. The risk is obvious: By definition, atypical events do not capture most of life; thus we must be careful not to think that a strategy that works in an interesting setting will necessarily apply to mundane everyday reality. The opportunity arises because atypical stories can be those that are not easily explained by existing theories. In that sense, such stories play the role of the experimental anomalies that have such an important role in the philosophies of science of Popper, Kuhn, and Lakatos. Just as the progress of science is stimulated by unexplained phenomena, social science can move forward through serious engagement with puzzling sequences of events. This is one reason we believe that stories are central to so many important works of social science. As Bearman and Stovel (2000) put it, ‘people construct stories to account for noncanonical events that cannot otherwise be accounted for by culturally agreed upon narrative expectations.’’ Good stories are immutable. Much can be learned from a true anecdote. The rough edges—the places where the anecdote does not fit your thesis—are where you learn. There is a saying in statistics that God is in every leaf of every tree. What this means is that if you study any problem carefully and seriously enough, you will come to interesting open research problems. This is related to the concept of thick description in anthropology. Details matter; this is the difference between character and caricature. In statistics, this relates to the idea that assumptions can be checked by comparing data to simulated replications from the fitted model (Gelman and Shalizi 2012). Whether the comparison is qualitative or quantitative, the point remains, that is, the story or data used as a comparison must not be so pliable that it can be interpreted as consistent with any model (this was Karl Popper’s famous criticism of the Freudian and Marxian paradigms).

One could construct a rough hierarchy of stories along the following lines:

  1. Joke, or fiction such as Harry Potter,
  2. Urban legend,
  3. Dubious story (the lost soldiers in the Alps),
  4. Plausible but undocumented story (e.g., Iraq’s weapons of mass destruction), and
  5. Documented anecdote

Is this missing people telling stories to deliberately mislead?

Updated Jul, 03 2020

#stories #metascience