Non-disclosure of changes to the research design
ThemesNon-disclosure of changes to the research design
What is this about?
Not disclosing changes to the research design is considered a questionable research practice.[1] At the start of research, the study design should be determined to ensure validity and verifiability. Whilst performing research, it may turn out that the study design contains flaws or is insufficient. Changing the study design without disclosing those changes undermines the transparency of the research process. Altering the design without disclosing the changes provides researchers with the opportunity to present their results in a skewed way. For example, the researcher could select the most spectacular results, or choose a different test that leads to statistical significance.
- ↑ Bouter, L. M., Tijdink, J., Axelsen, N., Martinson, B. C., & Ter Riet, G. (2016). Ranking major and minor research misbehaviors: results from a survey among participants of four World Conferences on Research Integrity. Research Integrity and Peer Review, 1(1), 17.
Why is this important?
Not disclosing changes creates a biased view of the research performed. Some of the changes that researchers perform after the first analyses include P-hacking, HARK-ing, cherry picking results, or performing explorative subgroup analyses. In qualitative research the methods can also be changed, for instance, changing the research question after data collection.
For whom is this important?
What are the best practices?
There are several ways to deal with this questionable research practice. The first is disclosing the changes made to the research design. The second is preregistration of studies.
Disclosing changes
Deviating from the research design is allowed under certain instances. For instance, new sub-questions can surface when further progressing with the project. These can only be answered by performing extra analyses or different tests. In those cases, disclosing changes to the research design is considered a good practice. These analyses or changes should be presented as explorative, rather than final.
Preregistration
Preregistration is the process of submitting the research design before performing the study. Preregistration can be seen as an effective way to address researchers from not ‘luring’ them into changing methods to present results more spectacularly.
Iris Lechner, Natalie Evans contributed to this theme. Latest contribution was Oct 27, 2020
