Papers - Natural

Ryan D Friedrichs, David C King, David W Nickerson

Cited by*: 0 Downloads*: 5

Recent large-scale field experiments of get out the vote (GOTV) drives have been non-partisan and may not accurately capture the effectiveness of partisan campaign outreach. In the 2002 Michigan gubernatorial election, a large field experiment across 14 state house districts evaluated the cost effectiveness of three mobilization technologies utilized by the Michigan Democratic Party's Youth Coordinated Campaign: door hangers, volunteer phone calls, and face-to-face visits. The results indicate that all three GOTV strategies possess similar cost-effectiveness.

Benjamin A Olken

Cited by*: 19 Downloads*: 38

This paper uses a randomized field experiment to examine several approaches to reducing corruption. I measure missing expenditures in over 600 village road projects in Indonesia by having engineers independently estimate the prices and quantities of all inputs used in each road, and then comparing these estimates to villages' official expenditure reports. I find that announcing an increased probability of a government audit, from a baseline of 4 percent to 100 percent, reduced missing expenditures by about 8 percentage points, more than enough to make these audits cost-effective. By contrast, I find that increasing grass-roots participation in the monitoring process only reduced missing wages, with no effect on missing materials expenditures. Since materials account for three-quarters of total expenditures, increasing grass-roots participation had little impact overall. The findings suggest that grass-roots monitoring may be subject to free-rider problems. Overall, the results suggest that traditional top-down monitoring can play an important role in reducing corruption, even in a highly corrupt environment.

Andrew Dustan, Juan Manuel Hernandez-Agramonte, Stanislao Maldonado

Cited by*: None Downloads*: None

We study how non-monetary incentives, motivated by recent advances in behavioral economics, affect civil servant performance in a context where state capacity is weak. We collaborated with a government agency in Peru to experimentally vary the context of text messages targeted to civil servants in charge of a school maintenance program. These messages incorporated behavioral insights in dimensions related to information provision, social norms, and weak forms of monitoring and auditing. We find that these messages are a very cost-effective strategy to enforce compliance with national policies among civil servants. We further study the role of social norms and the salience of social benefits in a follow-up experiment and explore the external validity or our original results by implementing a related experiment with civil servants from a different national program. The findings of these new experiments support our original results and provide additional insights regarding the context in which these incentives may work. Our results highlight the importance of carefully designed non-monetary incentives as a tool to improve civil servant performance when the state lacks institutional mechanisms to enforce compliance.

Kenneth Leonard, Melkiory Masatu

Cited by*: 0 Downloads*: 12

The most important issue facing experimental economists is the generalizability of lab results. This letter examines more than 1200 doctor/patient consultations, in which scrutiny and duration of treatment were varied. We show that scrutiny has an important but short-lived effect.

John A List, Azeem M Shaikh, Yang Xu

Cited by*: 33 Downloads*: 278

Empiricism in the sciences allows us to test theories, formulate optimal policies, and learn how the world works. In this manner, it is critical that our empirical work provides accurate conclusions about underlying data patterns. False positives represent an especially important problem, as vast public and private resources can be misguided if we base decisions on false discovery. This study explores one especially pernicious influence on false positives-multiple hypothesis testing (MHT). While MHT potentially affects all types of empirical work, we consider three common scenarios where MHT influences inference within experimental economics: jointly identifying treatment effects for a set of outcomes, estimating heterogenous treatment effects through subgroup analysis, and conducting hypothesis testing for multiple treatment conditions. Building upon the work of Romano and Wolf (2010), we present a correction procedure that incorporates the three scenarios, and illustrate the improvement in power by comparing our results with those obtained by the classic studies due to Bonferroni (1935) and Holm (1979). Importantly, under weak assumptions, our testing procedure asymptotically controls the familywise error rate - the probability of one false rejection - and is asymptotically balanced. We showcase our approach by revisiting the data reported in Karlan and List (2007), to deepen our understanding of why people give to charitable causes.

John A List, Azeem M Shaikh, Atom Vayalinkal

Cited by*: None Downloads*: None

List et al. (2019) provides a framework for testing multiple null hypotheses simultaneously using experimental data in which simple random sampling is used to assign treatment status to units. As in List et al. (2019), we rely on general results in Romano and Wolf (2010) to develop under weak assumptions a procedure that (i) asymptotically controls the familywise error rate – the probability of one or more false rejections – and (ii) is asymptotically balanced in that the marginal probability of rejecting any true null hypothesis is approximately equal in large samples. Our analysis departs from List et al. (2019) in that it further exploits observed, baseline covariates. The precise way in which these covariates are incorporated is based upon results in Ye et al. (2022) in order to ensure that inferences are typically more powerful in large samples.

John A List

Cited by*: None Downloads*: None

In 2019, I put together a summary of data from my field experiments website that pertained to natural field experiments (Harrison and List, 2024). Several people have asked me for updates. In this document I update all figures and numbers to show the details for 2023. I also include the description from the original paper below.

John A. List

Cited by*: Downloads*:

In 2019, I put together a summary of data from my field experiments website that pertained to natural field experiments (Harrison and List, 2004). Several people have asked me for updates. In this document I update all figures and numbers to show the details for 2024. I also include the description from the original paper below.

Glenn W Harrison, John A List

Cited by*: 23 Downloads*: 10

There has been a dramatic increase in the use of experimental methods in the past two decades. An oft-cited reason for this rise in popularity is that experimental methods provide the necessary control to estimate treatment effects in isolation of other confounding factors. We examine the relevance of experimental findings from laboratory settings that abstract from the field context of the task that theory purports to explain. Using common value auction theory as our guide, we identify naturally occurring settings in which one can test the theory. In our treatments the subjects are not picked at random, as in lab experiments with student subjects, but are deliberately identified by their trading roles in the natural field setting. We find that experienced agents bidding in familiar roles do not fall prey to the winner's curse. Yet, when experienced agents are observed bidding in an unfamiliar role, we find that they frequently fall prey to the winner's curse. We conclude that the theory predicts field behavior well when one is able to identify naturally occurring field counterparts to the key theoretical conditions.

Glenn W Harrison, John A List, Charles Towe

Cited by*: 1 Downloads*: 29

Does individual behavior in a laboratory setting provide a reliable indicator of behavior in a naturally occurring setting? We consider this general methodological question in the context of eliciting risk attitudes. The controls that are typically employed in laboratory settings, such as the use of abstract lotteries, could lead subjects to employ behavioral rules that differ from the ones they employ in the field. Because it is field behavior that we are interested in understanding, those controls might be a confound in themselves if they result in differences in behavior. We find that the use of artificial monetary prizes provides a reliable measure of risk attitudes when the natural counterpart outcome has minimal uncertainty, but that it can provide an unreliable measure when the natural counterpart outcome has background risk. Behavior tended to be moderately risk averse when artificial monetary prizes were used or when there was minimal uncertainty in the natural nonmonetary outcome, but subjects drawn from the same population were much more risk averse when their attitudes were elicited using the natural nonmonetary outcome that had some background risk. These results are consistent with conventional expected utility theory for the effects of background risk on attitudes to risk.

Paul Dolan, Robert D Metcalfe

Cited by*: 8 Downloads*: 6

There is increasing research on the exogenous impact of descriptive social norms on economic behavior. The research to date has a number of limitations: 1) it has not de-coupled the impact of the norm and the knowledge required to understand how to change behavior based upon it; 2) it has exclusively used offline but not online (i.e. emails) methods; and 3) it has not understood the impact of financial incentives in conjunction with norms. We address these three limitations using two natural field experiments. We find, firstly, that norms change energy behavior over a 15 month treatment period irrespective of whether information is provided or not. We find that social norms reduce consumption by around 6% (0.2 standard deviations). Norms have has their largest impact on the day that information on the social norm is received, and then decreases over time. Secondly, we do not find that social norms work online (even with experienced consumers who are used to online billing) - social norms de- livered online may have very little beneficial effects on reducing energy use. Thirdly, we find that large financial rewards work very well online in reducing consumption, with a 0.35 change in energy consumption over a four month period. Perhaps most interestingly, we find that the large effect of financial incentives is completely removed when information on social norms is added online.

Michael Kremer, Edward Miguel

Cited by*: 16 Downloads*: 19

We examine social learning using data from a program that promoted use of deworming medicine in Kenyan schools. These drugs kill worms in the body; although people are soon reinfected, treatment interferes with the cycle of transmission, generating positive externalities. Individuals randomly exposed to more information about deworming drugs through their social network were significantly less likely to take the drugs and more likely to believe the drugs are "not effective." This finding is consistent with the hypothesis that those exposed to the program had overly optimistic prior beliefs about net private drug benefits. The combination of strong social effects and extensive social networks among teenagers implies that a "child-to-child" public health approach focused on teenagers will speed social learning. There are large differences between social effect estimates relying on experimental variation (negative estimates) and nonexperimental methods (positive estimates).

John A List

Cited by*: None Downloads*: None

While empirical economics has made important strides over the past half century, there is a recent attack that threatens the foundations of the empirical approach in economics: external validity. Certain dogmatic arguments are not new, yet in some circles the generalizability question is beyond dispute, rendering empirical work as a passive enterprise based on frivolity. Such arguments serve to caution even the staunchest empirical advocates from even starting an empirical inquiry in a novel setting. In its simplest form, questions of external validity revolve around whether the results of the received study can be generalized to different people, situations, stimuli, and time periods. This study clarifies and places the external validity crisis into perspective by taking a unique glimpse into the grandest of trials: The External Validity Trial. A key outcome of the proceedings is an Author Onus Probandi, which outlines four key areas that every study should report to address external validity. Such an evaluative approach properly rewards empirical advances and justly recognizes inherent empirical limitations.

Junsoo Lee, John A List, Mark Strazicich

Cited by*: 0 Downloads*: 2

In this paper we examine temporal properties of eleven natural resource real price series from 1870-1990 by employing a Lagrangian Multiplier unit root test that allows for two endogenously determined structural breaks with and without a quadratic trend. Contrary to previous research, we find evidence against the unit root hypothesis for all price series. Our findings support characterizing natural resource prices as stationary around deterministic trends with structural breaks. This result is important in both a positive and normative sense. For example, without an appropriate understanding of the dynamics of a time series, empirical verification of theories, forecasting, and proper inference are potentially fruitless. More generally, we show that both pre-testing for unit roots with breaks and allowing for breaks in the forecast model can improve forecast accuracy.

Loukas Balafoutas, Nikos Nikiforakis

Cited by*: 22 Downloads*: 77

Extensive evidence from laboratory experiments indicates that many individuals are willing to use costly punishment to enforce social norms, even in one-shot interactions. However, there appears to be little evidence in the literature of such behavior in the field. We study the propensity to punish norm violators in a natural field experiment conducted in the main subway station in Athens, Greece. The large number of passengers ensures that strategic motives for punishing are minimized. We study violations of two distinct efficiency enhancing social norms. In line with laboratory evidence, we find that individuals punish norm violators. However, these individuals are a minority. Men are more likely than women to punish violators, while the decision to punish is unaffected by the violator's height and gender. Interestingly, we find that violations of the better known of the two norms are substantially less likely to trigger punishment. We present additional evidence from two surveys providing insights into the determinants of norm enforcement.

Indranil Goswami, Oleg Urminsky

Cited by*: None Downloads*: None

We present a complete empirical case study of fundraising decisions that demonstrates the importance of in-context field experiments. We first design novel matching-based fundraising appeals. We derive theory-based predictions from the standard impure altruism model and solicit expert opinion about the potential performance of our interventions. Both theory-based predictions and descriptive advice suggest improved fundraising performance from a framing intervention that credited donors for the matched funds (compared to a typical match framing). However, results from a natural field experiment with prior donors of a non-profit showed significantly poorer performance of this framing compared to a regularly framed matching intervention. This surprising finding was confirmed in a second natural field experiment, to establish the ground truth. Theoretically, our results highlight the limitations of both impure altruism models and of expert opinion in prediction complex "warm glow" motivation. More practically, our results question the availability of useful guidance, and suggest the indispensability of field testing for interventions in fundraising.

Pablo Celhay, Paul Gertler, Paula Giavagnoli, Christel Vermeersch

Cited by*: 0 Downloads*: 32

We show that fixed costs of adjustment as opposed to low returns likely explain why better quality care practices diffuse slowly in the medical industry. Using a randomized field experiment conducted in Argentina, we find that temporary financial incentives paid to health clinics for the early initiation of prenatal care 'nudged' providers to test and develop new data driven strategies to locate and encourage likely pregnant women to seek care in the first trimester of pregnancy. These innovations raised the rate of early initiation of prenatal care by 34% while the incentives were being paid in the treatment period. We follow health clinics over time and find that this increase persisted for at least 24 months after the incentives ended. In the absence of incentives, even though it is in the clinics' interest to stimulate early initiation of care, the presence of hard to change habits and cost of experimentation made it too expensive to develop and implement new methods to increase early initiation of care. Despite the large increases in early initiation of prenatal care, we find no effects on health outcomes.

Dean S Karlan, Jonathan Zinman

Cited by*: 2 Downloads*: 20

Information asymmetries are important in theory but difficult to identify in practice. We estimate the empirical importance of adverse selection and moral hazard in a consumer credit market using a new field experiment methodology. We randomized 58,000 direct mail offers issued by a major South African lender along three dimensions: 1) the initial "offer interest rate" appearing on direct mail solicitations; 2) a "contract interest rate" equal to or less than the offer interest rate and revealed to the over 4,000 borrowers who agreed to the initial offer rate; and 3) a dynamic repayment incentive that extends preferential pricing on future loans to borrowers who remain in good standing. These three randomizations, combined with complete knowledge of the Lender's information set, permit identification of specific types of private information problems. Specifically, our setup distinguishes adverse selection from moral hazard effects on repayment, and thereby generates unique evidence on the existence and magnitudes of specific credit market failures. We find evidence of both adverse selection (among women) and moral hazard (predominantly among men), and the findings suggest that about 20% of default is due to asymmetric information problems. This helps explain the prevalence of credit constraints even in a market that specializes in financing high-risk borrowers at very high rates.

Uri Gneezy, Andreas Leibbrandt, John A List

Cited by*: 1 Downloads*: 9

The functioning and well-being of any society and organization critically hinges on norms of cooperation that regulate social activities. Empirical evidence on how such norms emerge and in which environments they thrive remains a clear void in the literature. To provide an initial set of insights, we overlay a set of field experiments in a natural setting. Our approach is to compare behavior in Brazilian fishermen societies that differ along one major dimension: the workplace organization. In one society (located by the sea) fishermen are forced to work in groups whereas in the adjacent society (located on a lake) fishing is inherently an individual activity. We report sharp evidence that the sea fishermen trust and cooperate more and have greater ability to coordinate group actions than their lake fishermen counterparts. These findings are consistent with the argument that people internalize social norms that emerge from specific needs and support the idea that socio-ecological factors play a decisive role in the proliferation of pro-social behaviors.

Amee Kamdar, Steven D Levitt, John A List, Brian Mullaney, Chad Syverson

Cited by*: None Downloads*: None

In this paper, we present the results of a two-year series of large-scale natural field experiments involving hundreds of thousands of subjects.