Deception about the purpose of the study does not affect the behavior of the participants
The use of deception in the social sciences has long been controversial1. Deception is generally defined as the intentional and explicit deception of participants, as opposed to withholding information about hypotheses or experimental manipulations.2. Disclosures of human subject abuse in deceptive wartime experiments (e.g., Nazi medical trials3) and in times of peace (e.g. Milgram’s obedience experiments4) has led to major reforms in the ethical control of scientific research. The principles set out in the American Psychological Association (APA) Code of Ethics5 and the Belmont Report6 continue to guide internal review boards (IRBs) and ethics committees in the United States and beyondseven.
Deception practices differ across disciplines8,9,10. Economists generally prohibit deception in experimental studies. Their objections are mainly motivated by practical concerns11.12notably that deception will invoke distrust and suspicion of the experimenter, both invalidating tests of economic theory and affecting participant behavior in future studies, with consequences such as selective attrition13,14,15.
However, experimental evidence for these effects is sparse, with a few notable exceptions: Jamison et al.14 found that providing misleading information about partner identity (i.e., role deception) did not affect participants’ behavior in prisoner dilemma and dictator games over weeks following, noting that only ~55% of participants returned after the initial phase of the study. They did, however, find evidence, albeit weak, of inconsistent behavior in a gambling task used to assess risk preferences, which may reflect participants’ diminished seriousness. A replication of this work that minimized the attrition rate between phases (8%) found no difference in behavior12. Another concern is selective attrition in future experiments. Jamisson et al.14 found no difference between participants who had been cheated or not, although they found weak evidence that cheated women were less likely to participate in future experiments than uncheated women. Additionally, in a survey of mostly American college students, 25% said they would be less willing to participate in future experiments if they knew they had been cheated.16.
Regardless of the source of concern and whether supported by empirical evidence, there are tangible negative consequences for economics researchers who use deception. Perhaps most notable is that articles deploying deception are not eligible for publication in economic journals.17—a policy that is sometimes explicitly stated (for example, Experimental economics18). In addition, more interdisciplinary journals, such as the Proceedings of the National Academy of Sciencesas well as grant funding bodies may reject research that contains or proposes deception13,14,19,20. Together, these factors can be tangible barriers to career advancement12.
In psychology, on the other hand, there is no general ban on experimental deception. Rather, it has been argued that deception offers a way to study important but uncomfortable aspects of the human condition (e.g. conformity, obedience)21 and that results may otherwise be biased by social desirability. Professional bodies such as the APA and many academic IRBs discourage the use of deception except where there is notable scientific value and non-deceptive alternatives are not viable5.20. However, the frequency of using deception1.22 suggests that its use is not restricted to studies of particular importance. Furthermore, there is evidence that, at least for studies of honesty and prosocial behavior, non-misleading alternatives are commonly available but not used.23.
Experimental deception comes in many forms and there is often disagreement as to what precisely can be identified as deception.16.24. Sieber et al.25 The taxonomy identifies eight types of deception, ranging from declaring a false objective to not informing subjects that they are part of a study. We have updated this taxonomy to reflect changes in experimental practices over the past decades, including the proliferation of online experiments (Table S1). Here we focus on using deception for false purposes – the most common form of deception in psychology.1where participants “may receive or be tricked into knowing false information about the main purpose of the study”25. An example would be to say that a study is about life and satisfaction when it actually aims to investigate how work culture affects honesty26.
Variations in the stated experimental purpose can affect participants’ behavior in different ways. Participants who are told that a study is about honesty or who suspect that is the true experimental goal may succumb to the effects of experimenter demand or social desirability bias, or simply behave more honestly middle of a high sense of observation27,28,29,30. Alternatively, participants who suspect the experimenter deceived them about the purpose of the study may retaliate by increasing their dishonesty.27,31,32. Another possibility is that there is simply no effect on behavior33.
The results of a meta-analysis of 565 experiments on (dis)honest individual behavior suggest that misleading participants about the procedure leads to less dishonesty, but only in sender-receiver games (k= 165), which involve at least role deception30. There was no evidence that the use of deception affected honesty in the coin-flip (k= 163) or die roll (k= 129) tasks. For games involving collaborative dishonesty, a meta-analysis of 123 tasks, 16 of which involved deception, found that use of deception was associated with less dishonesty when all decision structures (e.g., joint, sequential ) were grouped together, but not when considered individually34. The authors hypothesize that participants who feel they are being deceived may suspect that their behavior is not anonymous and that their actions will in fact not benefit the group, leading to less collaborative dishonesty.
This work was prompted by a methodological question that arose during the replication of a study26.35: While the original researchers deployed deception on purpose, we were only able to use the incomplete disclosure due to concerns from partners in the field and the IRB. There seems to be a strong consensus that using fake goals is a relatively benign form of deception, as opposed to other forms of deception.24— and that it has generated only low levels of concern among researchers, participants, and the general population, both in recent decades36 and more recently37.38. In this context, we expected that there would be minimal harm, if any, to the participants in our experiments.
To our knowledge, Gallo et al.39 is the only prior study to experimentally manipulate deception solely in terms of false purpose. While the 3 (experimental disclosure) × 2 (task difficulty) design was pioneering at the time the study was conducted in the 1970s, it relied on a small (not= 120), non-representative sample (women, first-year psychology students at a university). The authors found no difference in results due to experimental disclosure, although half of those misled about the purpose of the research (compliance) reported being suspicious. Participants were not encouraged to be exact.
Given the prevalence of deception to false goals in academic research today, we add to this literature by providing the first new evidence in nearly four decades: our two studies adhere to today’s best research practices. today, because they are large-scale, very powerful, deploy incentives, and are pre-registered. Our sample is drawn from a globally used research job market – Amazon Mechanical Turk – which has played a major role in the social and behavioral sciences since its launch in 200540. The platform’s cohort of US workers is more representative than the in-person samples, although less representative than the panels41. Compared to the general population, American Amazon Mechanical Turk workers tend to be younger, European or Asian in descent, college-educated, have liberal political views, and have agnostic or atheistic views on religion.40. Wage is generally lower than the prevailing U.S. federal minimum wage42.