Papers - Artefactual
Marco Castillo,
John A List,
Ragan Petrie,
Anya Samek
Cited by*: None Downloads*: None 
We use field experiments with nearly 900 children to investigate how skills developed at ages 3-5 drive later-life outcomes. We find that skills map onto three distinct factors - cognitive skills, executive functions, and economic preferences. Returning to the children up to 7 years later, we find that executive functions, but not cognitive skills, predict the likelihood of receiving disciplinary referrals. Economic preferences have an independent effect: children who displayed impatience at ages 3-5 were more likely to receive disciplinary referrals. Random assignment to a parenting program reduced disciplinary referrals. This effect was not mediated by skills or preferences.
John A List
Cited by*: None Downloads*: None
Last year I put together a summary of data from my field experiments website that pertained to artefactual field experiments. Several people have asked me if I have and update. In this document I update all figures and numbers to show the details for 2020. I also include the description from the 2019 paper.
John A List
Cited by*: None Downloads*: None
All happy families are alike; each unhappy family is unhappy in its own way.
-Leo Tolstoy, Anna Karenina
Eliot Abrams,
Jonathan Libgober,
John A List
Cited by*: None Downloads*: None
The past few decades have ushered in an experimental revolution in economics whereby scholars are now much more likely to generate their own data. While there are virtues associated with this movement, there are concomitant difficulties. Several scientific disciplines, including economics, have launched research registries in an effort to attenuate key inferential issues. This study assesses registries both empirically and theoretically, with a special focus on the AEA registry. We find that over 90% of randomized controlled trials (RCTs) in economics do not register, only 50% of the RCTs that register do so before the intervention begins, and the majority of these preregistrations are not detailed enough to significantly aid inference. Our empirical analysis further shows that using other scientific registries as aspirational examples is misguided, as their perceived success in tackling the main issues is largely a myth. In light of these facts, we advance a simple economic model to explore potential improvements. A key insight from the model is that removal of the (current) option to register completed RCTs could increase the fraction of trials that register. We also argue that linking IRB applications to registrations could further increase registry effectiveness.
Avner Ben-Ner,
John A List,
Louis Putterman,
Anya Samek
Cited by*: None Downloads*: None
An active area of research within the social sciences concerns the underlying motivation for sharing resources and engaging in other pro-social actions. In this paper we ask: do parents model social preference behavior to children, and do children emulate this behavior? We develop a theoretical framework to examine this question, and conduct an experiment with 147 3 to 5 year old children and their parents, using dictator games to measure generosity. We find (1) evidence of parental teaching/modeling in the case of fathers and in that of parents of relatively generous children, and (2) an emulation effect such that children who initially share less than half of their endowment subsequently share more the more they see a parent or other adult share. We find little correlation between baseline sharing of children and the parents, with the possible exception of the oldest children.
Brian Albrecht,
Omar Al-Ubaydli,
Peter Boettke
Cited by*: None Downloads*: None
Economists well understand that the work of Friedrich Hayek contains important theoretical insights. It is less often acknowledged that his work contains testable predictions about the nature of market processes. Vernon Smith termed the most important one the 'Hayek hypothesis': that gains from trade can be realized in the presence of diffuse, decentralized information, and in the absence of price-taking behavior and centralized market direction. Vernon Smith tested this prediction by surveying data on laboratory experimental markets and found strong support. We extend Smith's work first by showing how subsequent theoretical advances provide a theoretical foundation for the Hayek Hypothesis. We then test the hypothesis using recent field experimental market data. Using field experiments allows us to test several other predictions from Hayek, such as that market experience increases the realized gains from trade. Generally speaking, we find support for Hayek's theories.
Yan Chen,
Peter Cramton,
John A List,
Axel Ockenfels
Cited by*: None Downloads*: None
We review past research and discuss future directions on how the vibrant research areas of market design and behavioral economics have influenced and will continue to impact the science and practice of management in both the private and public sectors. Using examples from various auction markets, reputation and feedback systems in online markets, matching markets in education, and labor markets, we demonstrate that combining market design theory, behavioral insights, and experimental methods can lead to fruitful implementation of superior market designs in practice.
John A List
Cited by*: None Downloads*: None
No abstract available
John A List
Cited by*: None Downloads*: None
All happy families are alike; each unhappy family is unhappy in its own way.
-Leo Tolstoy, Anna Karenina
Eduardo Fe,
David Gill,
Victoria Prowse
Cited by*: None Downloads*: None
We investigate how childhood cognitive skills affect strategic sophistication and adult outcomes. In particular, we emphasize the importance of childhood theory-of-mind as a cognitive skill. We collected experimental data from more than seven hundred children in a variety of strategic interactions. First, we find that theory-of-mind ability and cognitive ability both predict level-k behavior. Second, older children respond to information about the cognitive ability of their opponent, which provides support for the emergence of a sophisticated strategic theory-of-mind. Third, theory-of-mind and age strongly predict whether children respond to intentions in a gift-exchange game, while cognitive ability has no influence, suggesting that different measures of cognitive skill correspond to different cognitive processes in strategic situations that involve understanding intentions. Using the ALSPAC birth-cohort study, we find that childhood theory-of-mind and cognitive ability are both associated with enhanced adult social skills, higher educational participation, better educational attainment, and lower fertility in young adulthood. Finally, we provide evidence that school spending improves theory-of-mind in childhood.
Uri Gneezy,
Moshe Hoffman,
Mark A. Lane,
John A List,
Jeffrey A Livingston,
Michael J. Seiler
Cited by*: None Downloads*: None
Recent theoretical work shows that the better-than-average effect, where a majority
believes their ability to be better than average, can be perfectly consistent with Bayesian updating. However, later experiments that account for this theoretical advance still find behavior consistent with overconfidence. The literature notes that overoptimism can be caused by either overconfidence (optimism about performance), wishful thinking (optimism about outcomes), or both. To test whether the better-than-average effect might be explained by wishful thinking instead of overconfidence, we conduct an experiment that is similar to those used in the overconfidence literature, but removes performance as a potential channel. We find evidence that wishful thinking might explain overconfidence only among the most optimistic subjects, and that conservatism is possibly more of a worry; if unaccounted for, overconfidence might be underestimated.
Snigdha Gupta,
John A List,
Lauren Supplee,
Dana L Suskind
Cited by*: None Downloads*: None
Failed to Scale: Embracing the Challenge of Scaling in Early Childhood
Snigdha Gupta,
Maggie C. Kane,
John A List,
Liz Sablich,
Lauren Supplee,
Dana L Suskind
Cited by*: None Downloads*: None
Recommendations for Mitigating Threats to Scaling
Omar Al-Ubaydli,
John A List,
Claire Mackevicius,
Min Sok Lee,
Dana L Suskind
Cited by*: None Downloads*: None
Policymakers are increasingly turning to insights gained from the experimental method as a means to inform large scale public policies. Critics view this increased usage as premature, pointing to the fact that many experimentally-tested programs fail to deliver their promise at scale. Under this view, the experimental approach drives too much public policy. Yet, if policymakers could be more confident that the original research findings would be delivered at scale, even the staunchest critics would carve out a larger role for experiments to inform policy. Leveraging the economic framework of Al-Ubaydli et al. (2019), we put forward 12 simple proposals, spanning researchers, policymakers, funders, and stakeholders, which together tackle the most vexing scalability threats. The framework highlights that only after we deepen our understanding of the scale up problem will we be on solid ground to argue that scientific experiments should hold a more prominent place in the policymaker's quiver.
Syon Bhanot,
Jiyoung Han,
Chaning Jang
Cited by*: None Downloads*: None
Restrictions like work requirements and constraints on voucher transfers are often used in social welfare systems, but little empirical evidence exists on their impact on wellbeing. We conducted a 10-day randomized experiment with 432 individuals living below the poverty line in the Kawangware settlement of Nairobi, kenya, testing two elements of social welfare design: workfare versus welfare and restricted versus unrestricted vouchers. Participants were randomly assigned to a "Work" condition, involving daily work for unrestricted vouchers, or one of two "Wait" conditions, involving daily waiting for vouchers that were either unrestricted or partially restricted to staple foods. We find that working improved psychological wellbeing relative to waiting, suggesting that means of implementing welfare programs may have important effects on individuals beyond the impact of monetary benefit alone. Furthermore, although restrictions were inframarginal, partially restricted vouchers crowded-in spending on staple foods, suggesting the existence of a "flypaper effect" in spending from restricted vouchers.
Omar Al-Ubaydli,
John A List
Cited by*: None Downloads*: None
This is a review of the literature of field experimental studies of markets. The main results covered by the review are as follows: (1) Generally speaking, markets organize the efficient exchange of commodities; (2) There are some behavioral anomalies that impede efficient exchange; (3) Many behavioral anomalies disappear when traders are experienced.
Luigi Butera,
Philip J Grossman,
Daniel Houser,
John A List,
Marie-Claire Villeval
Cited by*: None Downloads*: None
Creation of empirical knowledge in economics has taken a dramatic turn in the past few decades. One feature of the new research landscape is the nature and extent to which scholars generate data. Today, in nearly every field the experimental approach plays an increasingly crucial role in testing theories and informing organizational decisions. Whereas there is much to appreciate about this revolution, recently a credibility crisis has taken hold across the social sciences, arguing that an important component of Fischer (1935)'s tripod has not been fully embraced: replication. Indeed, while the importance of replications is not debatable scientifically, current incentives are not sufficient to encourage replications from the individual researcher's perspective. We propose a novel mechanism that promotes replications by leveraging mutually beneficial gains between scholars and editors. We develop a model capturing the trade-offs involved in seeking independent replications before submission of a paper to journals. We showcase our method via an investigation of the effects of Knightian uncertainty on cooperation rates in public goods games, a pervasive and yet largely unexplored feature in the literature.
Daniel J Benjamin,
James O Berger,
Magnus Johannesson,
Brian A Nosek,
E. J Wagenmakers,
Richard Berk,
Kenneth A Bollen,
Bjorn Brembs,
Lawrence Brown,
Colin F Camerer,
David Cesarini,
Christopher D. Chambers,
Merlise Clyde,
Thomas D Cook,
Paul De Boeck,
Zoltan Dienes,
Anna Dreber,
Kenny Easwaran,
Charles Efferson,
Ernst Fehr,
Fiona Fidler,
Andy P. Field,
Malcom Forster,
Edward I. George,
Tarun Ramadorai,
Richard Gonzalez,
Steven Goodman,
Edwin Green,
Donald P Green,
Anthony Greenwald,
Jarrod D. Hadfield,
Larry V. Hedges,
Leonhard Held,
Teck Hau Ho,
Herbert Hoijtink,
James Holland Jones,
Daniel J Hruschka,
Kosuke Imai,
Guido Imbens,
John P.A. Ioannidis,
Minjeong Jeon,
Michael Kirchler,
David Laibson ,
John A List,
Roderick Little,
Arthur Lupia,
Edouard Machery,
Scott E. Maxwell,
Michael McCarthy,
Don Moore,
Stephen L. Morgan,
Marcus Munafo,
Shinichi Nakagawa,
Brendan Nyhan,
Timothy H Parker,
Luis Pericchi,
Marco Perugini,
Jeff Rouder,
Judith Rousseau,
Victoria Savalei,
Felix D. Schonbrodt,
Thomas Sellke,
Betsy Sinclair,
Dustin Tingley,
Trisha Van Zandt,
Simine Vazire,
Duncan J. Watts,
Christopher Winship,
Robert L. Wolpert,
Yu Xie,
Cristobal Young,
Jonathan Zinman,
Valen E. Johnson
Cited by*: 1 Downloads*: 965
We propose to change the default P-value threshold for statistical significance for claims of new discoveries from 0.05 to 0.005.
Luigi Butera,
John A List
Cited by*: 1 Downloads*: 355
Novel empirical insights by their very nature tend to be unanticipated, and in some cases at odds with the current state of knowledge on the topic. The mechanics of statistical inference suggest that such initial findings, even when robust and statistically significant within the study, should not appreciably move priors about the phenomenon under investigation. Yet, a few well-conceived independent replications dramatically improve the reliability of novel findings. Nevertheless, the incentives to replicate are seldom in place in the sciences, especially within the social sciences. We propose a simple incentive-compatible mechanism to promote replications, and use experimental economics to highlight our approach. We begin by reporting results from an experiment in which we investigate how cooperation in allocation games is affected by the presence of Knightian uncertainty (ambiguity), a pervasive and yet unexplored characteristic of most public goods. Unexpectedly, we find that adding uncertainty enhances cooperation. This surprising result serves as a test case for our mechanism: instead of sending this paper to a peer-reviewed journal, we make it available online as a working paper, but we commit never to submit it to a journal for publication. We instead offered co-authorship for a second, yet to be written, paper to other scholars willing to independently replicate our study. That second paper will reference this working paper, will include all replications, and will be submitted to a peer- reviewed journal for publication. Our mechanism allows mutually-beneficial gains from trade between the original investigators and other scholars, alleviates the publication bias problem that often surrounds novel experimental results, and accelerates the advancement of economic science by leveraging the mechanics of statistical inference.
Glenn W Harrison,
Morten I Lau,
Elisabet E Rutstrom,
Melonie B Williams
Cited by*: 29 Downloads*: 303
We design experiments to jointly elicit risk and time preferences for the adult Danish population. The experimental procedures build on laboratory experiments that have been evaluated using traditional subject pools. The field experiments utilize field sampling designs that we developed, and procedures that were chosen to be relatively transparent in the field with non-standard subject pools. Our overall design was also intended to be a general template for such field experiments in other countries. We examine the characterization of risk over a wider domain for each subject than previous experiments, allowing more precise estimates of risk attitudes. We also examine individual discount rates over six time horizons, as the first stage in a panel experiment in which we revisit subjects to test consistency and stability of responses over time. Risk and time preferences are heterogeneous, varying by observable individual characteristics. On a methodological level, we implement a refinement of existing procedures which elicits much more precise estimates, and also mitigates framing effects.