Behavioural science is unlikely to change the world without a heterogeneity revolution

Nat Hum Behav. 2021 Aug;5(8):980-989. doi: 10.1038/s41562-021-01143-3. Epub 2021 Jul 22.

Abstract

In the past decade, behavioural science has gained influence in policymaking but suffered a crisis of confidence in the replicability of its findings. Here, we describe a nascent heterogeneity revolution that we believe these twin historical trends have triggered. This revolution will be defined by the recognition that most treatment effects are heterogeneous, so the variation in effect estimates across studies that defines the replication crisis is to be expected as long as heterogeneous effects are studied without a systematic approach to sampling and moderation. When studied systematically, heterogeneity can be leveraged to build more complete theories of causal mechanism that could inform nuanced and dependable guidance to policymakers. We recommend investment in shared research infrastructure to make it feasible to study behavioural interventions in heterogeneous and generalizable samples, and suggest low-cost steps researchers can take immediately to avoid being misled by heterogeneity and begin to learn from it instead.

Publication types

  • Review

MeSH terms

  • Artificial Intelligence
  • Behavior Control
  • Behavioral Sciences*
  • Causality
  • Humans
  • Policy Making*
  • Reproducibility of Results*
  • Research Design*
  • Scientific Experimental Error