Damon Clark, David Gill, Victoria Prowse , Mark Rush
Cited by*: Downloads*:

Will college students who set goals for themselves work harder and achieve better outcomes? In theory, setting goals can help present-biased students to mitigate their self-control problem. In practice, there is little credible evidence on the causal effects on goal setting for college students. We report the result of two field experiments that involved almost four thousand college students in total. One experiments asked treated students to set goals for performance in the course; the other asked treated students to set goals for a particular task (completing online practice exams). Task-based goals had robust positive effects on the level of task completion, and task-based goals also increased course performance. We also find that performance-based goals had positive but small effects on course performance. We use a theoretical framework that builds on present bias and loss aversion to interpret our results. Since task-based goal setting is low-cost, scalable and logistically simple, we conclude that our findings have important implications for educational practice and future research.
John A List, Zacharias Maniadis, Fabio Tufano
Cited by*: Downloads*:

The sciences are in an era o fan alleged "credibility crisis'. In this study, we discuss the reproducibility of empirical results, focusing on economics research. By combining theory and empirical evidence, we discuss the import of replication studies, and whether they improve our confidence in novel findings. The theory sheds light on the importance of replications, even when replications are subject to bias. We then present a pilot meta-study of replication in experimental economics, a subfield serving as a positive benchmark for investigating the credibility of economics. Our meta-study highlights certain difficulties when applying meta-research (Ioannidis et al., 2015) and systematizing the economics literature.
Omar Isaac Asensio, Magali A Delmas
Cited by*: 3 Downloads*: None

Little is known about the effect of message framing on conservation behavior over time. In a randomized controlled trial with residential households, we use advanced metering and information technologies to test how different messages about household energy use impact the dynamics of conservation behavior down to the appliance level. Our results, based on 374 million panel observations of kilowatt-hour (kWh) electricity consumption for 118 households over 9 months, show that differences in behavioral responses due to message framing become more significant over time. We find that a health-based frame, in which households consider the human health effects of their marginal electricity use, induced persistent energy savings behavior of 8-10% over 100 days; whereas a more traditional cost savings frame, drove sharp attenuation of treatment effects after 2 weeks with no significant savings versus control after 7 weeks. We discuss the implications for the design of effective information campaigns to engage households in conservation behavior.
Andreas Leibbrandt, John A List
Cited by*: Downloads*:

Labor force composition and the allocation of talent remain of vital import to modern economies. For their part, governments and companies around the globe have implemented equal employment opportunity (EEO) regulations to influence labor market flows. Even though such regulations are pervasive, surprisingly little is known about their impacts. We use a natural field experiment conducted across 10 U.S. cities to investigate if EEO statements in job advertisements affect the first step in the employment process, application rates. Making use of data from nearly 2,500 job seekers, we find considerable policy effects, but in an unexpected direction: the presence of an EEO statement dampens rather than encourages racial minorities willingness to apply for jobs. Importantly, the effects are particularly pronounced for educated job seekers and in cities with white majority populations. Complementary survey evidence suggests the underlying mechanism at work is "tokenism", revealing that EEO statements backfire because racial minorities avoid environments in which they are perceived as regulatory, or symbolic, hires rather than being hired on their own merits. Beyond their practical and theoretical importance, our results highlight how field experiments can significantly improve policy making. In this case, if one goal of EEO regulations is to enhance the pool of minority applicants, then it is not working.
Daniel J Benjamin, James O Berger, Magnus Johannesson, Brian A Nosek, E. J Wagenmakers, Richard Berk, Kenneth A Bollen, Bjorn Brembs, Lawrence Brown, Colin F Camerer, David Cesarini, Christopher D. Chambers, Merlise Clyde, Thomas D Cook, Paul De Boeck, Zoltan Dienes, Anna Dreber, Kenny Easwaran, Charles Efferson, Ernst Fehr, Fiona Fidler, Andy P. Field, Malcom Forster, Edward I. George, Tarun Ramadorai, Richard Gonzalez, Steven Goodman, Edwin Green, Donald P Green, Anthony Greenwald, Jarrod D. Hadfield, Larry V. Hedges, Leonhard Held, Teck Hau Ho, Herbert Hoijtink, James Holland Jones, Daniel J Hruschka, Kosuke Imai, Guido Imbens, John P.A. Ioannidis, Minjeong Jeon, Michael Kirchler, David Laibson , John A List, Roderick Little, Arthur Lupia, Edouard Machery, Scott E. Maxwell, Michael McCarthy, Don Moore, Stephen L. Morgan, Marcus Munafo, Shinichi Nakagawa, Brendan Nyhan, Timothy H Parker, Luis Pericchi, Marco Perugini, Jeff Rouder, Judith Rousseau, Victoria Savalei, Felix D. Schonbrodt, Thomas Sellke, Betsy Sinclair, Dustin Tingley, Trisha Van Zandt, Simine Vazire, Duncan J. Watts, Christopher Winship, Robert L. Wolpert, Yu Xie, Cristobal Young, Jonathan Zinman, Valen E. Johnson
Cited by*: 1 Downloads*: 965

We propose to change the default P-value threshold for statistical significance for claims of new discoveries from 0.05 to 0.005.
Michael Fix, Raymond J Struyk
Cited by*: 32 Downloads*: 955

Auditing is a technique used to test for discrimination. The concept is straightforward: Two individuals are matched on all relevant characteristics except the one presumed to lead to discrimination. Each person then applies for the same job, housing, mortgage loan, or credit card. The differential treatment they receive provides a measure of discrimination. The authors argue that the value of auditing has grown in the current legal and political environment because it can detect subtle forms of discrimination.
Abhijit Banerjee, Angus Deaton, Esther Duflo
Cited by*: 28 Downloads*: 482

What are the determinants of the health and of well-being? Income and wealth are clearly part of the story, but does access to health-care have a large independent effect, as the advocates of more investment in health-care, such as the World Health Organization's Commission on Macroeconomics and Health (Commission on Macroeconomics and Health (2001)), have argued? This paper reports on a recent survey in a poor rural area of the state of Rajasthan in India intended to shed some light on this issue, where there was an attempt to use a set of interlocking surveys to collect data on health and economic status, as well as the public and private provision of health care.
Luigi Butera, John A List
Cited by*: 1 Downloads*: 355

Novel empirical insights by their very nature tend to be unanticipated, and in some cases at odds with the current state of knowledge on the topic. The mechanics of statistical inference suggest that such initial findings, even when robust and statistically significant within the study, should not appreciably move priors about the phenomenon under investigation. Yet, a few well-conceived independent replications dramatically improve the reliability of novel findings. Nevertheless, the incentives to replicate are seldom in place in the sciences, especially within the social sciences. We propose a simple incentive-compatible mechanism to promote replications, and use experimental economics to highlight our approach. We begin by reporting results from an experiment in which we investigate how cooperation in allocation games is affected by the presence of Knightian uncertainty (ambiguity), a pervasive and yet unexplored characteristic of most public goods. Unexpectedly, we find that adding uncertainty enhances cooperation. This surprising result serves as a test case for our mechanism: instead of sending this paper to a peer-reviewed journal, we make it available online as a working paper, but we commit never to submit it to a journal for publication. We instead offered co-authorship for a second, yet to be written, paper to other scholars willing to independently replicate our study. That second paper will reference this working paper, will include all replications, and will be submitted to a peer- reviewed journal for publication. Our mechanism allows mutually-beneficial gains from trade between the original investigators and other scholars, alleviates the publication bias problem that often surrounds novel experimental results, and accelerates the advancement of economic science by leveraging the mechanics of statistical inference.
Glenn W Harrison, Morten I Lau, Elisabet E Rutstrom, Melonie B Williams
Cited by*: 29 Downloads*: 303

We design experiments to jointly elicit risk and time preferences for the adult Danish population. The experimental procedures build on laboratory experiments that have been evaluated using traditional subject pools. The field experiments utilize field sampling designs that we developed, and procedures that were chosen to be relatively transparent in the field with non-standard subject pools. Our overall design was also intended to be a general template for such field experiments in other countries. We examine the characterization of risk over a wider domain for each subject than previous experiments, allowing more precise estimates of risk attitudes. We also examine individual discount rates over six time horizons, as the first stage in a panel experiment in which we revisit subjects to test consistency and stability of responses over time. Risk and time preferences are heterogeneous, varying by observable individual characteristics. On a methodological level, we implement a refinement of existing procedures which elicits much more precise estimates, and also mitigates framing effects.
John A List, Azeem M Shaikh, Yang Xu
Cited by*: 33 Downloads*: 278

Empiricism in the sciences allows us to test theories, formulate optimal policies, and learn how the world works. In this manner, it is critical that our empirical work provides accurate conclusions about underlying data patterns. False positives represent an especially important problem, as vast public and private resources can be misguided if we base decisions on false discovery. This study explores one especially pernicious influence on false positives-multiple hypothesis testing (MHT). While MHT potentially affects all types of empirical work, we consider three common scenarios where MHT influences inference within experimental economics: jointly identifying treatment effects for a set of outcomes, estimating heterogenous treatment effects through subgroup analysis, and conducting hypothesis testing for multiple treatment conditions. Building upon the work of Romano and Wolf (2010), we present a correction procedure that incorporates the three scenarios, and illustrate the improvement in power by comparing our results with those obtained by the classic studies due to Bonferroni (1935) and Holm (1979). Importantly, under weak assumptions, our testing procedure asymptotically controls the familywise error rate - the probability of one false rejection - and is asymptotically balanced. We showcase our approach by revisiting the data reported in Karlan and List (2007), to deepen our understanding of why people give to charitable causes.