You do not have permission to edit this page, for the following reason:

Only users who are logged in using ORCID can make a change or add a contribution. Please login here.

Back

Edit a theme

We invite you to share your knowledge and write about a theme which you find relevant, interesting or controversial. You can share your own expertise, present your new initiative, write a commentary about one of the resources hosted on The Embassy, or simply describe a topic that you think should be shared with the research community.

Before you submit, please be aware of our community guidelines.

Theme

P-value hacking, also known as data dredging, data fishing, data snooping or data butchery, is an exploitation of data analysis in order to discover patterns which would be presented as statistically significant, when in reality, there is no underlying effect.<ref>Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD. The extent and consequences of p-hacking in science. PLoS Biol. 2015;13(3).</ref><ref>Norman G. Data dredging, salami-slicing, and other successful strategies to ensure rejection: twelve tips on how to not get your paper published: Adv Health Sci Educ Theory Pract. 2014 Mar;19(1):1-5. doi: 10.1007/s10459-014-9494-8.</ref> In other words, p-hacking is running statistical tests on a set of data until some statistically significant results arise. That can be done in a few different ways, for example: by stopping the collection of data once you get a P<0.05, analyzing many outcomes, but only reporting those with P<0.05, using covariates, excluding participants, etc.
<references />

It’s difficult to address the issue of P-value hacking, especially since there aren’t many incentives to replicate research. However, some steps can be taken in order to prevent it. Cross-validation, or out-of-sample testing is a statistical method used to create two sets of data. The first set of data is then used for statistical analysis, to develop new models or hypotheses, and the other, independent set is then used to verify them.<ref>Berk R, Brown L, Zhao L. Statistical Inference After Model Selection. Journal of Quantitative Criminology. 2010;26(2):217-36.</ref> A number of statistical analyses is also available to check for p-value hacking, such as Bonferonni correction, Scheffé's method and false discovery rate. A lot of journals will now ask for raw data to be published, or shift their way of work to registered report format. That is a publication process in which journals accept the publications based on theoretical justification and methodology only, without looking at results. <ref>Simons DJ, Holcombe AO, Spellman BA. An Introduction to Registered Replication Reports at Perspectives on Psychological Science. Perspect Psychol Sci. 2014;9(5):552-5.</ref>
<references />

Related To

Edit Theme: Theme:6b584d4e-2c9d-4e27-b370-5fbdb983ab46

Edit a theme