View source for Theme:349f9eb9-b796-46cb-9a98-214c06db9046

{{Theme
|Theme Type=Good Practices
|Has Parent Theme=Theme:7e07b7fa-793a-4282-a036-6fefe8480b3a
|Title=Statistical pre-registration
|Is About=In a statistical pre-registration, the purpose and type of statistical analysis to be conducted in a study should be disclosed before the study begins. Statistical pre-registration helps to clearly distinguish between hypothesis generating (exploratory) studies, for which statistical significance is not meaningful, and hypothesis testing (confirmatory) studies for which statistical significance is meaningful<ref>Joe Leif Uri. Data Colada. How to Properly Preregister a Study. 2017 Nov 6. [cited 2020 Aug 12]. Available from: http://datacolada.org/64</ref>.
<references />
|Important Because=Exploratory and confirmatory parts of research are two different stages and it is important to clearly distinguish them. However, researchers very often don’t make that distinction; they generate or change their hypotheses after observing the data and yet they report their study as exploratory or hypothesis-generating (see HARKing thematic page)<ref name=":0">Lindsay DS, Simons DJ, Lilienfeld SO. Research Preregistration 101. APS. 2016 Nov 30. [cited 2020 Aug 12]. Available from: https://www.psychologicalscience.org/observer/research-preregistration-101</ref>. Observing the data before choosing which statistical tests to conduct can show statistical significance where there actually isn’t any<ref name=":0" />. Subsequently, this leads to false positive results. Stopping these practices is important in every research area, but especially in medicine. Hypotheses regarding drug effects generated in exploratory research are tested in confirmatory research<ref>Kimmelman J, Mogil JS, Dirnagl U. Distinguishing between Exploratory and Confirmatory Preclinical Research Will Improve Translation. PLoS Biol. 2014;12(5):e1001863</ref>. Without careful hypotheses testing, patients are put at risk<ref>Carlisle B, Federico CA, Kimmelman J. Trials that say ‘maybe’: the disconnect between exploratory and confirmatory testing after drug approval. BMJ 2018;360:k959</ref>.

To avoid this kind of inappropriate research, and all the consequences that can arise from it, pre-registration of a study is essential. Pre-registration consists of specifying and recording your study plans before you even start observing the data<ref name=":0" /><ref name=":1">Nosek BA, Ebersole CR, DeHaven AC, Mellor DT. The preregistration revolution. PNAS 2018;115(11):2600-2606.</ref>. It refers to recording hypotheses, subjects, statistical analysis and predictions<ref>APS. Easier Done Than Said: Lessons from 6 Years of Preregistration. 2020 March 11. [cited 2020 Aug 12]. Available from: https://www.psychologicalscience.org/publications/observer/obsonline/easier-done-than-said-lessons-from-6-years-of-preregistration.html</ref>. This will help clearly distinguish the exploratory and confirmatory data<ref name=":0" /><ref name=":1" /><ref name=":2">Wagenmakers EJ, Dutilh G. Seven Selfish Reasons for Preregistration. APS. 2016 Oct 31. [cited 2020 Aug 12]. Available from: https://www.psychologicalscience.org/observer/seven-selfish-reasons-for-preregistration</ref>, prevent bias<ref name=":2" />, avoid p-hacking<ref name=":3">APS. Preregistration of Research Plans. [cited 2020 Aug 12]. Available from: https://www.psychologicalscience.org/publications/psychological_science/preregistration</ref>, ensure transparency and openness of the analysis plans<ref name=":3" />, and increase trustworthiness and reproducibility of study findings<ref name=":1" />.
<references />
|Important For=Researchers; Policy makers; Journal editors
|Has Best Practice=The most relevant examples are studies needed for drug approval. The approval procedure usually requires a series of clinical trials divided into three phases. Phase I and II can involve model building according to European Medicines Agency (EMA), however phase III trial is always designed as a confirmatory trial<ref name=":0">Carlisle B, Federico CA, Kimmelman J. Trials that say ‘maybe’: the disconnect between exploratory and confirmatory testing after drug approval. BMJ 2018;360:k959</ref>. Both the EMA and the Food and Drug Administration (FDA) require statistical pre-registration before the beginning of a trial. Exploratory trials aim to produce evidence of effectiveness of new drugs. These results then lead to confirmatory trials<ref name=":0" />. The need for pre-registrations of trials is demonstrated by the following case. In 2004, the New York attorney general’s office filed a lawsuit against pharmaceutical company GlaxoSmithKline. Four unpublished clinical trials showed evidence that use of the antidepressant Paxil increases the risk of suicidal tendencies amongst young people. This lawsuit helped raise awareness that studies need to be pre-registered<ref name=":1">Should You Pre-Register Your Research Study? A Quick Guide. Enago Academy. 2020 April 3. [cited 2020 Aug 12]. Available from: https://www.enago.com/academy/pre-registration-of-your-research/</ref>. That year the International Committee of Medical Journal Editors required pre-registration of clinical trials<ref name=":1" />. 

US laws also require clinical trials to be pre-registered<ref name=":2">Nosek BA, Ebersole CR, DeHaven AC, Mellor DT. The preregistration revolution. PNAS 2018;115(11):2600-2606.</ref>. Journals and research funders support pre-registration and some organizations promote it as an important step towards openness and transparency in research<ref name=":3">Mellor D. Preregistration: improved rigor, improved workflow. BMC. 2016 Feb 17. [cited 2020 Aug 12]. Available from: https://blogs.biomedcentral.com/bmcblog/2016/02/17/preregistration-improved-rigor-improved-workflow/</ref>. There have been some other interesting efforts that promote pre-registration, such as the Preregistration Challenge, sponsored by the Center for Open Science. This campaign offered $1,000 awards to researchers who pre-register their studies and publish their results within a deadline<ref name=":2" /><ref name=":3" /><ref>Mellor DT, Esposito J, Hardwicke TE, Nosek BA, Cohoon J, Soderberg CK, et al. Preregistration Challenge: Plan, Test, Discover. OSF HOME. 2019 Sep 20. [cited 2020 Aug 12]. Available from:  https://osf.io/x5w7h/</ref>. These initiatives have led to a certain “cultural shift” - there are now more than 8,000 pre-registrations on Open Science Framework for research in different disciplines<ref name=":2" />. APS journals also began to practice pre-registration in 2014. From 2014 to 2019, 43 of 154 articles published in Psychological Science earned “Preregistered badge” due to pre-registration of design and analysis plan of their studies<ref>APS. Open Practice Badges. [cited 2020 Aug 12]. Available from: https://www.psychologicalscience.org/publications/badges</ref>. 

Although pre-registration has demonstrated benefits for the trustworthiness of research, the practice still needs to be widely adopted across the scientific community.
<references />
}}
{{Related To
|Related To Theme=Theme:26631aa0-18f0-4635-b71b-80a6f4e58d33;Theme:D44fd22a-ed5d-4120-a78b-8881747131fd
}}
{{Tags
|Involves=David T Mellor; Brian Nosek
|Has Virtue And Value=Honesty; Reliability; Transparency
|Has Good Practice And Misconduct=Pre-registrations; Reproducibility; Research with Humans
|Related To Research Area=LS - Life Sciences
}}