Questionable Research Practices in Analysis and Reporting

Themes

Questionable Research Practices in Analysis and Reporting

What is this about?

In a list of major and minor research misbehaviours collaboratively developed by a group of research integrity experts, the research phase ‘Reporting’ (which describes analysis and publication of results jointly) contains the most items. Needless to say, the potential for misbehaviors in this phase is exacerbated by the vast number of decisions researchers must take during the course of analysis and reporting.

Why is this important?

Integrity in analysis and reporting of results is important to fully understand your data. Misbehaviors related to analysis and reporting include:

1. Report on data driven hypotheses without disclosure [‘HARKing’ ‐ Hypothesizing After Results are Known ‐ typically with a view to make results appear more spectacular (‘Chrysalis effect’)]   
2. Delete data before performing data analysis without disclosure   
3. Selectively delete data, modify data or add fabricated data after performing initial data‐analyses  [in other words: falsification or fabrication of data]   
4. Perform data‐analyses not stated in the study protocol without disclosure  [or in predefined data‐analysis plan – also called ‘Significance chasing’, ‘P-hacking’, ‘data dredging’,  ‘fishing expedition’ or explorative subgroup analyses]   
5. Report an incorrect downwardly rounded p‐value [e.g. by reporting a p value of .054 as being less than .05]   
6. Not report all study protocol‐stipulated results  [in the aggregate of all published reports on the study at issue]   
7. Not publish a valid ‘negative’ study  [in a form that is publicly available or accessible behind a paywall (article, report, website etc.)]   
8. Report an unexpected finding as having been hypothesized from the start   
9. Conceal results that contradict your earlier findings or convictions   
10. Not report clearly relevant details of study methods  
11. Not report replication problems   
12. Selectively cite to enhance your own findings or convictions   
13. Selectively cite to please editors, reviewers or colleagues   
14. Selectively cite or cite your own work to improve citation metrics  [e.g. Impact Factor, H‐index]   
15. Let your convictions influence the conclusions substantially   
16. Insufficiently report study flaws and limitations   
17. Spread study results over more papers than needed [‘salami slicing’]   
18. Duplicate publication without disclosure   
19. Re‐use of previously published data without disclosure [which may lead to double counting in meta‐analyses]   
20. Modify the results or conclusions of a study due to pressure of a sponsor  [commercial or not‐for‐profit funder of the study]   
21. Failure to disclose a sponsor of the study   
22. Failure to disclose a relevant financial or intellectual conflict of interest  [in publications, when reviewing grant proposals, or evaluating persons or institutions]  
23. Handle existing conflicts of interest inadequately   
24. Communicate results to the general public before a peer reviewed publication is available  
25. Deliberately communicate findings inaccurately in the media or during presentations   
26. Make no clear distinction between personal views and professional comments (List from Bouter et al 2016)

For whom is this important?

Early career researchersJunior researchersSenior researchersResearchersScientists

Natalie Evans contributed to this theme. Latest contribution was Oct 28, 2020

Other information