Field experiments in economics: The past, the present, and the future

Levitt Steven D; List John A; List John A; Shaikh Azeem M; Xu Yang; Al-Ubaydli Omar; List John A; Levitt Steven D; List John A; Syverson Chad; Lange Andreas; List John A; Price Michael K; Fredriksson  Per; List John A; Millimet Daniel L; Fredriksson  Per; List John A; Millimet Daniel L; Bulte Erwin; List John A; Shogren Jason F; List John A; Lucking-Reiley David; List John A; Margolis Michael; Shogren Jason F; Larson Francis; List John A; Metcalfe Robert D; Fryer  Roland; Levitt Steven D; List John A; Gneezy Uri; List John A; Price Michael K; List John A; Livingston Jeffrey A; Neckermann Susanne; Bulte Erwin; List John A; van Soest Daan; List John A; List John A; List John A; Suskind Dana L; List John A; List John A

Field Experiments In Economics: The Past, The Present, And The Future

Steven D Levitt, John A List

Cited by*: 58 Downloads*: 52

This study presents an overview of modern field experiments and their usage in economics. Our discussion focuses on three distinct periods of field experimentation that have influenced the economics literature. The first might well be thought of as the dawn of "field" experimentation: the work of Neyman and Fisher, who laid the experimental foundation in the 1920s and 1930s by conceptualizing randomization as an instrument to achieve identification via experimentation with agricultural plots. The second, the large-scale social experiments conducted by government agencies in the mid-twentieth century, moved the exploration from plots of land to groups of individuals. More recently, the nature and range of field experiments has expanded, with a diverse set of controlled experiments being completed outside of the typical laboratory environment. With this growth, the number and types of questions that can be explored using field experiments has grown tremendously. After discussing these three distinct phases, we speculate on the future of field experimental methods, a future that we envision including a strong collaborative effort with outside parties, most importantly private entities.

Multiple Hypothesis Testing In Experimental Economics

John A List, Azeem M Shaikh, Yang Xu

Cited by*: 33 Downloads*: 278

Empiricism in the sciences allows us to test theories, formulate optimal policies, and learn how the world works. In this manner, it is critical that our empirical work provides accurate conclusions about underlying data patterns. False positives represent an especially important problem, as vast public and private resources can be misguided if we base decisions on false discovery. This study explores one especially pernicious influence on false positives-multiple hypothesis testing (MHT). While MHT potentially affects all types of empirical work, we consider three common scenarios where MHT influences inference within experimental economics: jointly identifying treatment effects for a set of outcomes, estimating heterogenous treatment effects through subgroup analysis, and conducting hypothesis testing for multiple treatment conditions. Building upon the work of Romano and Wolf (2010), we present a correction procedure that incorporates the three scenarios, and illustrate the improvement in power by comparing our results with those obtained by the classic studies due to Bonferroni (1935) and Holm (1979). Importantly, under weak assumptions, our testing procedure asymptotically controls the familywise error rate - the probability of one false rejection - and is asymptotically balanced. We showcase our approach by revisiting the data reported in Karlan and List (2007), to deepen our understanding of why people give to charitable causes.

Do Natural Field Experiments Afford Researchers More Or Less Control Than Laboratory Experiments? A Simple Model

Omar Al-Ubaydli, John A List

Cited by*: 5 Downloads*: 89

A commonly held view is that laboratory experiments provide researchers with more "control" than natural field experiments, and that this advantage is to be balanced against the disadvantage that laboratory experiments are less generalizable. This paper presents a simple model that explores circumstances under which natural field experiments provide researchers with more control than laboratory experiments afford. This stems from the covertness of natural field experiments: laboratory experiments provide researchers with a high degree of control in the environment which participants agree to be experimental subjects. When participants systematically opt out of laboratory experiments, the researcher's ability to manipulate certain variables is limited. In contrast, natural field experiments bypass the participation decision altogether and allow for a potentially more diverse participant pool within the market of interest. We show one particular case where such selection is invaluable: when treatment effects interact with participant characteristics.

Toward An Understanding Of Learning By Doing: Evidence From An Automobile Plant

Steven D Levitt, John A List, Chad Syverson

Cited by*: 7 Downloads*: 12

Productivity improvements within establishments (e.g., factories, mines, or retail stores) are an important source of aggregate productivity growth. Past research has documented that learning by doing-productivity improvements that occur in concert with production increases-is one source of such improvements. Yet little is known about the specific mechanisms through which such learning occurs. We address this question using extremely detailed data from an assembly plant of a major auto producer. Beyond showing that there is rapid learning by doing at the plant, we are able to pinpoint the processes by which these improvements have occurred.

Auctions With Resale When Private Values Are Uncertain: Evidence From The Lab And Field

Andreas Lange, John A List, Michael K Price

Cited by*: 0 Downloads*: 0

Auction theory represents one of the richest areas of research in economics over the past three decades. Yet, whether, and to what extent, the introduction of secondary resale markets influences bidding behavior in sealed bid first-price auctions remains under researched. This study begins by examining field data from a unique data set that includes nearly 3,000 auctions (over 10,000 individual bids) for cutting rights of standing timber in British Columbia from 1996-2000. In comparing bidding patterns across agents who are likely to have resale opportunities with those who likely do not, we find evidence that is consistent with theory. Critical evaluation of the reduced-form bidding model, however, reveals that sharp tests of the theoretical predictions are not possible because several other differences may exist across these bidder types. We therefore use a laboratory experiment to examine if the resale opportunity by itself can have the predicted theoretical effect. We find that while it does have the predicted effect, a theoretical model based on risk-averse bidders explains the overall data patterns more accurately than a model based on risk-neutral bidders. Beyond testing theory, the paper highlights the inferential power of combining naturally occurring data with laboratory data.

Bureaucratic Corruption, Environmental Policy And Inbound Us Fdi: Theory And Evidence

Per Fredriksson , John A List, Daniel L Millimet

Cited by*: 33 Downloads*: 18

Previous studies have proposed that equilibrium capital flows are affected by environmental regulations-the commonly coined 'pollution haven' hypothesis. We revisit this issue by treating environmental policies as endogenous and allowing governmental corruption to influence foreign direct investment patterns. Via these two simple extensions, we are able to provide a much richer model of international capital flows. The theoretical model presumes that the effect of corruption on FDI operates via two channels: corruption affects capital flows through its impact on environmental policy stringency and due to greater theft of public funds earmarked for public spending. We empirically examine the implications of the model using US state-level panel data from four industrial sectors over the period 1977-1987. Empirical results suggest environmental policy and corruption both play a significant role in determining the spatial allocation of inbound US FDI. In addition, the estimated effect of environmental policy is found to depend critically on exogeneity assumptions.

Chasing The Smokestack: Strategic Policymaking With Multiple Instruments

Per Fredriksson , John A List, Daniel L Millimet

Cited by*: 8 Downloads*: 1

Empirical evidence suggesting that a considerable amount of horizontal strategic interaction exists amongst governments is important in light of recent devolutionary trends of many important public programs. The empirical approach in these studies typically relies on estimating reaction functions in a uni-dimensional policy framework, where a nonzero slope estimate is interpreted as evidence in support of strategic interactions. While this framework is a useful representation within certain contexts, it is potentially too restrictive; for example, in models of resource competition, localities may use multiple instruments in their recruiting pursuits, leading to potential strategic interactions across policy instruments. In this study, we first develop a simple theoretic construct that includes resource competition in a world of three-dimensional policy choice. The model suggests that while a zero-sloped reaction function may exist for any particular policy, this does not necessarily imply the absence of strategic interactions. We examine the implications of the model empirically using US state-level panel data over the period 1977-1994. The results suggest that important cross-policy strategic interactions exist, lending support in favor of the multi-dimensional framework, and indicate that uni-dimensional frameworks may present lower bound estimates of the degree of strategic interaction.

"Beggar Thy Neighbor:" Testing For Free Riding In State-Level Endangered Species Expenditures

Erwin Bulte, John A List, Jason F Shogren

Cited by*: 1 Downloads*: 2

An important public policy question that remains unresolved is whether devolution will enhance sensible policy making by exploiting informational asymmetries or, instead, trigger a "beggar thy neighbor" response and stimulate free riding amongst localities. We analyze this question within the framework of U.S. environmental policy making by scrutinizing a unique panel data set on state-level endangered species expenditure patterns. Our empirical estimates are consistent with the notion that states free ride, which may lead to an expenditure equilibrium that is not Pareto efficient.

Demand Reduction In Multiunit Auctions: Evidence From A Sportscard Field Experiment

John A List, David Lucking-Reiley

Cited by*: 29 Downloads*: 1

n/a

Hypothetical-Actual Bid Calibration Of A Multigood Auction

John A List, Michael Margolis, Jason F Shogren

Cited by*: 3 Downloads*: 0

Evidence suggests the calibration of hypothetical and actual behavior is good-specific. We examine whether clustering commodities into mutual categories can reduce the burden. While we reject a common calibration across sets of commodities, a sport-specific calibration function cannot be rejected.

Can Myopic Loss Aversion Explain The Equity Premium Puzzle? Evidence From A Natural Field Experiment With Professional Traders

Francis Larson, John A List, Robert D Metcalfe

Cited by*: 1 Downloads*: 61

Behavioral economists have recently put forth a theoretical explanation for the equity premium puzzle based on combining myopia and loss aversion. Complementing the behavioral theory is evidence from laboratory experiments, which provide strong empirical support consistent with myopic loss aversion (MLA). Yet, whether, and to what extent, such preferences underlie behaviors of traders in their natural domain remains unknown. Indeed, a necessary condition for the MLA theory to explain the equity premium puzzle is for marginal traders in markets to exhibit such preferences. Using minute-by-minute trading observations from over 864,000 price realizations in a natural field experiment, we find data patterns consonant with MLA: in their normal course of business, professional traders who receive infrequent price information invest 33% more in risky assets, yielding profits that are 53% higher, compared to traders who receive frequent price information. Beyond testing theory, these results have important implications for efficient resource allocation as well as characterizing the optimal structure of social and economic policies.

Parental Incentives And Early Childhood Achievement: A Field Experiment In Chicago Heights

Roland Fryer , Steven D Levitt, John A List

Cited by*: 9 Downloads*: 6

This article describes a randomized field experiment in which parents were provided financial incentives to engage in behaviors designed to increase early childhood cognitive and executive function skills through a parent academy. Parents were rewarded for attendance at early childhood sessions, completing homework assignments with their children, and for their child's demonstration of mastery on interim assessments. This intervention had large and statistically significant positive impacts on both cognitive and non-cognitive test scores of Hispanics and Whites, but no impact on Blacks. These differential outcomes across races are not attributable to differences in observable characteristics (e.g. family size, mother's age, mother's education) or to the intensity of engagement with the program. Children with above median (pre-treatment) non cognitive scores accrue the most benefits from treatment.

Toward An Understanding Of Why People Discriminate: Evidence From A Series Of Natural Field Experiments

Uri Gneezy, John A List, Michael K Price

Cited by*: 12 Downloads*: 7

Social scientists have presented evidence that suggests discrimination is ubiquitous: women, nonwhites, and the elderly have been found to be the target of discriminatory behavior across several labor and product markets. Scholars have been less successful at pinpointing the underlying motives for such discriminatory patterns. We employ a series of field experiments across several market and agent types to examine the nature and extent of discrimination. Our exploration includes examining discrimination based on gender, age, sexual orientation, race, and disability. Using data from more than 3000 individual transactions, we find evidence of discrimination in each market. Interestingly, we find that when the discriminator believes the object of discrimination is controllable, any observed discrimination is motivated by animus. When the object of discrimination is not due to choice, the evidence suggests that statistical discrimination is the underlying reason for the disparate behavior.