View source for Theme:6b584d4e-2c9d-4e27-b370-5fbdb983ab46

{{Theme
|Theme Type=Misconduct & Misbehaviors
|Has Parent Theme=Theme:48185295-9e1e-41fb-ab70-948596e588d5
|Title=P-value hacking
|Is About=P-value hacking, also known as data dredging, data fishing, data snooping or data butchery, is an exploitation of data analysis in order to discover patterns which would be presented as statistically significant, when in reality, there is no underlying effect.<ref>Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD. The extent and consequences of p-hacking in science. PLoS Biol. 2015;13(3).</ref><ref>Norman G. Data dredging, salami-slicing, and other successful strategies to ensure rejection: twelve tips on how to not get your paper published: Adv Health Sci Educ Theory Pract. 2014 Mar;19(1):1-5. doi: 10.1007/s10459-014-9494-8.</ref> In other words, p-hacking is running statistical tests on a set of data until some statistically significant results arise. That can be done in a few different ways, for example: by stopping the collection of data once you get a P<0.05, analyzing many outcomes, but only reporting those with P<0.05, using covariates, excluding participants, etc.
<references />
|Important Because=Unfortunately, current practices in science show that journals that are considered of high quality (those with high impact factors) predominately publish statistically significant results. Researchers want to publish in such journals because it's important for their academic prestige and job.<ref>Ekmekci PE. An increasing problem in publication ethics: Publication bias and editors' role in avoiding it. Med Health Care Philos. 2017;20(2):171-8.</ref>  This creates pressure on researchers, and can lead to P-value hacking. P-value hacking leads to false positive results, which can get published, and have a negative impact on future research in the field, secondary research and systematic reviews and human knowledge in general.<ref>Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008;3(8):0003081.</ref>
<references />
|Important For=Students; PhD Students; Scientists; Researchers; Supervisors; Postdocs; Journal editors; Junior researchers; Senior researchers
|Has Best Practice=It’s difficult to address the issue of P-value hacking, especially since there aren’t many incentives to replicate research. However, some steps can be taken in order to prevent it. Cross-validation, or out-of-sample testing is a statistical method used to create two sets of data. The first set of data is then used for statistical analysis, to develop new models or hypotheses, and the other, independent set is then used to verify them.<ref>Berk R, Brown L, Zhao L. Statistical Inference After Model Selection. Journal of Quantitative Criminology. 2010;26(2):217-36.</ref> A number of statistical analyses is also available to check for p-value hacking, such as Bonferonni correction, Scheffé's method and false discovery rate. A lot of journals will now ask for raw data to be published, or shift their way of work to registered report format. That is a publication process in which journals accept the publications based on theoretical justification and methodology only, without looking at results. <ref>Simons DJ, Holcombe AO, Spellman BA. An Introduction to Registered Replication Reports at Perspectives on Psychological Science. Perspect Psychol Sci. 2014;9(5):552-5.</ref>
<references />
}}
{{Related To
|Related To Resource=Resource:8354ff67-9da4-4325-8395-d16e30059fb2;Resource:226c89f1-a061-4bb0-8ec4-79583de2ddf0;Resource:47bfd883-c518-4a97-98fb-86b5cf442d3e
|Related To Theme=Theme:24e87492-7020-4fc0-ab37-dd88bcf9f637;Theme:88b73549-fec0-4fb9-99f6-fe1055d6b76a
}}
{{Tags
|Has Virtue And Value=Reliability
|Has Good Practice And Misconduct=Questionable research practice
}}