In Detail (Has Detail)
From The Embassy of Good Science
Here you can go in more detail and even add subheaders to give more relevant background information.
- ⧼SA Foundation Data Type⧽: Text
D
The detailed report on national guidelines in Europe is available here:
[https://community.embassy.science/c/entire/entirerep/24 Latest EnTIRE/Repository topics - The Embassy of Good Science]
References:
1. Moher D, Bouter L, Kleinert S, et al. The Hong Kong Principles for assessing researchers: Fostering research integrity. PLOS Biology 2020;18(7): e3000737.
2. Declaration on Research Assessment (DORA). The San Francisco Declaration on Research Assessment (DORA), [https://sfdora.org/about-dora/ https://sfdora.org/about-dora/#]; 2012 [accessed 2022.11th January].
3. All European Academies (ALLEA). The European Code of Conduct for Research Integrity, https://www.allea.org/wp-content/uploads/2017/05/ALLEA-European-Code-of-Conduct-for-Research-Integrity-2017.pdf; 2017 [accessed 2021.11th January].
4. Council ER. Estonian Code of Conduct for Research Integrity, https://www.eetika.ee/sites/default/files/www_ut/hea_teadustava_eng_trukis.pdf; 2017.
5. Ministry of Higher Education and Science. Danish Code of Conduct for Research Integrity, https://ufm.dk/en/publications/2014/files-2014-1/the-danish-code-of-conduct-for-research-integrity.pdf; 2014.
6. The Swiss Academies of Arts and Sciences. Code of Conduct for Scientific Integrity, https://api.swiss-academies.ch/site/assets/files/25709/kodex_layout_en_web.pdf; 2021.
7. Forum RIN. Irish National Policy Statement for Ensuring Research Integrity, https://www.iua.ie/wp-content/uploads/2019/08/IUA_Research_Integrity_in_Ireland_Report_2019.pdf; 2014.
<br /> +
Q
1. Tenny S, Brannan GD, Brannan JM, Sharts-Hopko NC. Qualitative Study. 2020
2. Giacomini, M. K. & Cook, D. J. Users’ guides to the medical literature: XXIII. Qualitative research in health care. B. What are the results and how do they help me care for my patients? Evidence-Based MedicineWorking Group. JAMA 2000; 284: 478–482. +
V
This list might help when using virtues in e.g. teaching about responsible conduct of research. *Virtues in bold mirror the principles for research integrity outlined in the [[European Code of Conduct for Research Integrity]].
{| class="wikitable"
|+
!Virtue
!Meanings in research integrity & ethics
|-
|'''Accountability'''*
|Responsibility, answerability
|-
|Availability
|Efficaciousness, readiness to come to effect
|-
|Clarity of purpose
|Visionary, targeted, zeal
|-
|Collaborative spirit
|Cooperative, synergistic, sharing
|-
|Competency
|Expertise, proficiency, capability
|-
|Compliance
|Willingness to conform/follow
|-
|Courage
|Braveness, heroic resoluteness
|-
|Creativity
|Inventiveness, imagination, originality
|-
|Critical awareness
|Analytic, insightful, rationality
|-
|Curiosity
|Eagerness to know or to explore, inquisitiveness
|-
|Diligence
|Earnestness, perseverance in carrying out action
|-
|Empathy
|Understanding, compassion, recognition
|-
|Fairness
|Justice, equity
|-
|'''Honesty'''*
|Truthfulness, candidness, sincerity
|-
|Humility
|Humbleness, modesty
|-
|Loyalty
|Faithfulness, allegiance, fidelity
|-
|Moderation
|Temperance, patience
|-
|Morality
|Ethicalness, righteousness, decency
|-
|Objectivity
|Neutrality, unbiased, impartiality, open-minded
|-
|Open-mindedness
|Willing to reconsider views, receptiveness, tolerance
|-
|Patience
|Perseverance, willingness to endure
|-
|Perseverance
|Dedication, determination, persistence
|-
|Positivity
|Alacrity, willingness
|-
|Punctuality
|Readiness, promptness, steadiness
|-
|Reflexivity
|Thoughtfulness, contemplativeness, deliberation
|-
|'''Reliability'''*
|Trustworthiness, accuracy, dependability
|-
|Resoluteness
|Determination, persistence, purposefulness
|-
|'''Respectfulness'''*
|Politeness, having good manners, courtesy
|-
|Responsibility
|Accountability, liable, trustworthiness, truthfulness
|-
|Selflessness
|Altruism, benevolence
|-
|Sincerity
|Earnestness, truthfulness, veracity, honesty
|-
|Thoroughness
|Care, scrupulousness
|-
|Transparency
|Clarity, not hiding, honesty, openness
|-
|Trustworthiness/truthfulness
|Honesty, accuracy, sincerity
|}
[[File:Virtue wheel2 NW.png|center|frameless|1050x1050px]]A visual ‘wheel’ of virtues is also presented as a useful resource for teaching (with no virtue prioritized over others to inspire but not lead participants). This resource was created by Neelam Wright, University of Surrey.
[
While all COIs are related to contradictory primary and secondary interests, they can be traced to different causes and result in different types of detrimental research practices. Moreover, safeguards to address COIs vary depending on whether they are avoidable and how they can be mitigated. It is important to emphasize that while COIs can lead to intentional breaches of research integrity and research ethics, conflicting interests are more likely to result in unintentional bias.
In general, it is common to distinguish between financial and non-financial COIs. Financial COIs occur if researchers receive direct payments from the sponsor of a study, hold stocks in the sponsoring company, receive financial benefits from the sponsor for services, or have any other financial relationship with the producer of the product investigated in a study.<sup>[6]</sup> Due to the resulting incentive structure, financial COIs increase the likelihood of bias towards results favorable to the company directly or indirectly paying the researchers.
Non-financial COIs, by contrast, are unrelated to financial remuneration and tend to be more difficult to identify. At least three types of non-financial COIs can be differentiated: personal COIs, intellectual COIs, and medical COIs.<sup>[7]</sup>
Personal COIs are usually attributable to positive or negative personal relationships, even though conflicts of conscience (that is, conflicts between the personal values of researchers and the demands placed on them by their institution) also fall into this category. Biased peer-review is a well-known example of the detrimental effects of personal relationships because the peer review system is based on the premise of reviewer neutrality. Reviewers who have recently collaborated with the author(s) of a paper or grant application or who work in the same department might have a more favorable view on the paper or application under review than reviewers without such personal associations. Even if reviews are based on a blinded system, reviewers might know or be able to guess who the authors are, especially in highly specialized and rather small fields of research or when researchers can suggest reviewers. Negative personal relationships, by contrast, can have the opposite effect.<sup>[8]</sup>
Intellectual COIs occur when researchers become so convinced of the truth of a particular finding or paradigm that they become biased against alternative explanations and dismissive of contradictory findings, regardless of the quality of the evidence.<sup>[9]</sup>
Medical COIs refer to situations where the personal medical experiences of researchers could bias their research in that area. For example, a researcher who has suffered from a particular disease, could be inclined to view this disease in a way colored by personal experiences.<sup>[10]</sup> This may have methodologically unjustifiable effects on research design or, in the same way as intellectual COIs, lead to biased interpretations of results.
COIs are addressed by three types of safeguards: disclosure, management and prohibition. They can be disclosed to the research institution, research ethics committees or institutional review boards, journal editors and readers, and research participants. Management refers to erecting procedures that mitigate the risk of COIs, for example by replacing researchers with a COI with researchers without a COI for certain parts of a research project. Prohibition refers to prohibiting certain types of research if a COI exists. Employees of pharmaceutical companies cannot lead clinical drug trials of drugs produced by their company, for example.<sup>[11]</sup>
According to current regulations, financial COIs must be declared so that readers are made aware of conflicting interests and can scrutinize papers accordingly. Non-financial COIs are regulated less stringently and perhaps also less amenable to formal regulation. Overall, it is strongly recommended to avoid personal COIs by refraining from suggesting reviewers who might be biased as well as by refraining from accepting to review manuscripts or grant applications one cannot assess neutrally. Intellectual COIs can perhaps best be managed by regular self-reflection and regular participation in reflection-based research integrity trainings, such as, for example, the VIRT2UE training or the PRINTEGER Upright program. Medical COIs could potentially be declared, yet this is not usually done and would be challenging both legally and ethically because it would require the disclosure of sensitive personal information.
Overall, COIs can take many forms and be mitigated through different types of safeguards. As such, COIs are not breaches of research integrity or research ethics, yet they increase the likelihood of violations of good scientific practice. Therefore, the research community has adopted mitigatory measures to manage COIs that might require further refinement in the future.
'''References'''
[1] Emanuel E.J., & Thompson D.F. (2008). The concept of conflict of interest. In: Emanual, E.J., Grady, C., Crouch, R.A, Lie, R.K., Miller, F.G., & Wendler, D.D. ''The Oxford Textbook of Clinical Research Ethics''. Oxford: Oxford University Press, 758-766.
[2] https://eneri.mobali.com/content/conflict-interest, Key issues
[3] Ibid.
[4] WMA International Code of Medical Ethics, 2006. https://www.wma.net/policies-post/wma-international-code-of-medical-ethics/
[5] https://eneri.mobali.com/content/conflict-interest, Learning objectives and introduction
[6] Ibid.
[7] https://eneri.mobali.com/content/conflict-interest, Key issues
[8] Ibid.
[9] Ibid.
[10] Ibid.
[11] Ibid.
'''References'''
Kaye J, Boddington P, de Vries J, Hawkins N, Melham K. Ethical implications of the use of whole genome methods in medical research. European journal of human genetics : EJHG. 2010;18(4):398-403. Epub 2009/11/06.
<br /> +
1. Kitchin R. The Data Revolution: Big Data, Open Data, Data Infrastructures, and Their Consequences. SAGE; 2014. 241 p.
2. Tedersoo L, Küngas R, Oras E, Köster K, Eenmaa H, Leijen Ä, et al. Data sharing practices and data availability upon request differ across scientific disciplines. Sci Data. 2021 Jul 27;8:192.
3. Fischer BA, Zigmond MJ. The essential nature of sharing in science. Sci Eng Ethics. 2010 Dec;16(4):783–99.
4. Gabelica M, Bojcic R, Puljak L. Many researchers were not compliant with their published data sharing statement: mixed-methods study. Journal of Clinical Epidemiology. 2022 May 30;150:33-41. doi: 10.1016/j.jclinepi.2022.05.019.
5. Sustainable Digital Data Preservation and Access Network Partners. NSF - National Science Foundation [Internet]. [cited 2022 Mar 2]. Available from: '"`UNIQ--nowiki-0000095E-QINU`"'
6. Tenopir C, Allard S, Douglass K, Aydinoglu AU, Wu L, Read E, et al. Data Sharing by Scientists: Practices and Perceptions. PLOS ONE. 2011 Jun 29;6(6):e21101.
<br /> +
The big issue in clinical research is collecting an informed consent from all participants included in research. This can become a dilemma in psychiatry research, too. In psychiatry, the patient should be stable and informed about all procedures that will be taken. However, collecting an informed consent from psychiatry patient can become a challenge. A patient should be fit for work tasks and should be cognitive capable of understanding that he is a part of research. +
'''Juniors only''' In 2018, the session was extended with a second meeting where only junior researchers attend. In this session they have the opportunity to discuss issues that they did not feel comfortable to discuss with the seniors present and/or issues related to supervisors.
'''Organizing members''' The ‘Billen Bloot’ sessions are organized by and for researchers from Alzheimer Center Amsterdam, Amsterdam UMC, which is also where the initiative was conceived by prof. dr. Van der Flier (head of research), in 2015. The ‘Billen Bloot’ sessions are organized by the research quality committee of the Alzheimer Center Amsterdam and receives no support from external parties. +
'''Bottlenecks'''
According to Yan Wang, data stewards at the TU Delft have been very lucky, because all the necessary resources, such as the funding, positions and material were provided by the university. Although there haven’t been any urgent issues during implementation, there are /were a few points of attention. Sometimes, data stewards have so much work that they need to limit their role. They cannot provide as much support as they would like to. Luckily, the university has agreed to recruit eight data managers that can specifically help with the data, thereby giving space to the data stewards. Moreover, some requests are too technical for data stewardship. In those cases, they send out the researcher’s request to the ICT board. Lastly, Yang Wang expressed that it took time to get adjusted to the university structure.
'''Future scope'''
The pilot of the data stewardship will finish at the end of 2020, along with the funding. Meanwhile, data stewardship has proven to be successful and therefore, in the future each data steward will be paid by the faculty, thereby becoming a permanent employee at the faculty. In this way, the initiative remains sustainable.
The TU Delft data stewardship likes to create awareness about data management. One way to approach this is by going to external events, such as the INSPIRE symposium in January 2020. The university supports these events as long as it is related to data stewardship or if it is good promotion.
'''Any tips?'''
“''The most important thing is to really translate the objectives into the perspective of the researchers. For instance, if there is resistance to the word integrity, then use it in another way. There is always a way to work around it. And always reflect on the initiative itself.''” From an interview with Yan Wang, current data steward at the architecture and building faculty at TU Delft. +
All of them, especially junior researchers would be more ready to talk about their dilemmas or obstacles they are facing during their research work. They might not hesitate to ask for help or for any other suggestion which will improve the quality of their scientific investigation.
For senior researchers, these meetings would be the valuable feedback about their overall research giving them the opportunity to stimulate their junior colleagues if it is necessary and/ or to support their work, efforts, ideas etc.
The frequency and duration of these meetings depend on the researchers themselves, their satisfaction from previous meetings, the quality of their achievements so far, and the necessity for further improvement of their research work.
From time to time, maybe, it would be valuable to everyone to invite the research leader on ''Pizza-Gathering'' emphasizing the integrity of the team and the integrity of the research, as well. +
'''Examples of common situations which involve dilemmas'''
{| class="wikitable" border="1" cellspacing="0" cellpadding="0"
| width="200" valign="top"|'''Dilemma'''
| width="200" valign="top"|'''Value A'''
| width="200" valign="top"|'''Value B'''
|-
| width="200" valign="top"|Co-authorship
<br />
| width="200" valign="top"|intellectual contribution
| width="200" valign="top"|collegiality
|-
|Sharing your data openly
|Verification & re-use
|Rebuttal
|}
'''Examples of common values & principles involved in dilemmas'''
{| class="wikitable" border="1" cellspacing="0" cellpadding="0"
| width="120" valign="top"|'''Personal'''
| width="120" valign="top"|'''Scientific'''
| width="120" valign="top"|'''Technological'''
| width="120" valign="top"|'''Professional'''
|-
| width="120" valign="top"|Honesty
| width="120" valign="top"|Accuracy
| width="120" valign="top"|Utility
| width="120" valign="top"|Mentorship
|-
| width="120" valign="top"|Curiosity
| width="120" valign="top"|Completeness
| width="120" valign="top"|
| width="120" valign="top"|Collaboration
|-
| width="120" valign="top"|Trustworthiness
| width="120" valign="top"|Consistency
| width="120" valign="top"|
| width="120" valign="top"|
|-
| width="120" valign="top"|Respect
| width="120" valign="top"|Objectivity
| width="120" valign="top"|
| width="120" valign="top"|
|-
| width="120" valign="top"|Autonomy
| width="120" valign="top"|Auditability
| width="120" valign="top"|
| width="120" valign="top"|
|-
| width="120" valign="top"|Collegiality
| width="120" valign="top"|Universality
| width="120" valign="top"|
| width="120" valign="top"|
|-
| width="120" valign="top"|
| width="120" valign="top"|Precision
| width="120" valign="top"|
| width="120" valign="top"|
|-
| width="120" valign="top"|
| width="120" valign="top"|Verification
| width="120" valign="top"|
| width="120" valign="top"|
|}
<br /> +
The PRO-RES Framework is built upon a set of resources that both help to generate ethical research and assess its integrity. The resources are operationalised in a toolbox that includes several means for assessing ethical research. The final ‘pillar’ in the PRO-RES Framework is ‘The Accord’ – a statement of principles for ethical evidence-gathering that individuals, agencies and organisations can sign up to as an assurance of their best intentions when gathering and using evidence to inform policies. +
The formal learning pathway of Path2Integrity consists of 29 so-called [https://embassy.science/wiki-wiki/index.php/Resource:C13f2fea-2e63-4da9-8b9a-e4039c3dfde1 learning cards], which describe step-by-step how topics related to research integrity can be taught in units of 90-120 minutes. All learning cards address topics identified as crucial for research integrity in the European Code of Conduct for Research Integrity.
There are three series of learning cards:
*S-series: for secondary school students and undergraduate students (pre-disciplinary)
*M-series: for graduate students (disciplinary)
*Y-series: for early career researcher (post-disciplinary)
All learning cards are based on one of three dialogical methods: role-play, storytelling or coming to an agreement. The learning cards are [https://path2integrity.eu/ri-materials#LearningCards openly available on the Path2Integrity website] and described in more detail in the educational resources section.
The learning cards can not only be used in classroom settings, but also in online teaching. To facilitate online teaching, Path2Integrity has created an [https://www.learning-p2i.eu/ online learning platform], which allows for combining asynchronous and synchronous phases of learning.
The informal learning pathway of Path2Integrity consists of a [https://embassy.science/wiki-wiki/index.php/Resource:1ac96d0e-6379-41ca-8cb4-cd219aecf298 campaign] that aims to raise awareness about the importance of research integrity for science and society. Campaign materials include posters, postcards, leaflets, booklets and videos, many of which are not only available in English, but also in Bulgarian, Spanish, Catalan, Danish, German and Polish). Many of the materials rely on prominent researchers as role models to inspire pupils, students and young researchers to strive for integrity in their own work.
Path2Integrity evaluates the effectiveness of all materials throughout all phases of the project in order to identify crucial factors for effective research integrity education.
Both, SSH and STEM models do have pros and cons. Most of pros and cons are explained in the article. I think that it will be the best to try to use both models since both could provide benefits. However, we should be aware of the cons and try to minimize them. +
Since research in which children participate are very demanding, the research approach should be explained in great details, so I would like to add some more important notes.
It is always necessary to obtain the consent of the Ethics Committee first, no matter what type of study is conducted. Certainly, it is good to have any kind of consent from the Community, association, or similar.
Considering the fact that the research including children do not have same protocols, Ethics Committees might also ask for the consent signed by the child. For example, if the child can comprehend research, in addition to asking for the consent of the parents or guardian, it would be good to have the child's consent as well. It may happen that the parent/guardian agrees, but the child does not agree. The research should always be explained in detail to the parent, but also to the child if it is old enough to understand it. The parent, as well as the child, may later decide that they want to leave the research.
It is also necessary to explain in detail how the identity of the participant will be protected. In this case, it is best to anonymize the data so that no one could identify participants.
Data should be stored in a safe place, where no one has access without authorization. Data can also be coded, that is data pseudonymization. In this way, you need to take even more care of the data, because even though it is coded, the data is still very vulnerable. Certainly, it is necessary to follow all research instructions and local legislative, and GDPR. In the application to the Ethics Committee, it is necessary to state whether the research participant has any benefits or risks from participating. Whoever submits the research should clearly emphasize this in the application. Also, if it is a questionnaire, it would be the best id the children could fill in the questionnaire by themselves, so that there is no inherent bias. However, there is always the question if the child is mature enough to comprehend the meaning of the questions asked in the questionnaire. Therefore, when dealing with children, always have in mind if there are mature enough.
References:
1. Kreiman J. On peer review. ''J Speech, Lang Hear Res''. 2016;59(3):480-483. doi:10.1044/2016_JSLHR-S-16-0043
2. Huisman J, Smits J. Duration and quality of the peer review process: the author’s perspective. ''Sci 2017 1131''. 2017;113(1):633-650. doi:10.1007/S11192-017-2310-5
3. Ravinetto R, Caillet C, Zaman MH, et al. Preprints in times of COVID19: the time is ripe for agreeing on terminology and good practices. ''BMC Med Ethics 2021 221''. 2021;22(1):1-5. doi:10.1186/S12910-021-00667-7
4. Sarabipour S, Debat HJ, Emmott E, Burgess SJ, Schwessinger B, Hensel Z. On the value of preprints: An early career researcher perspective. ''PLoS Biol''. 2019;17(2). doi:10.1371/JOURNAL.PBIO.3000151
5. Fraser N, Brierley L, Dey G, et al. The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape. ''PLOS Biol''. 2021;19(4):e3000959. doi:10.1371/JOURNAL.PBIO.3000959
6. Reddick R. Preprints: how draft academic papers have become essential in the fight against COVID. Published 2021. Accessed October 11, 2021. '"`UNIQ--nowiki-000006E2-QINU`"'
7. Ledford H. Coronavirus breakthrough: dexamethasone is first drug shown to save lives. Accessed October 11, 2021. '"`UNIQ--nowiki-000006E3-QINU`"'
8. Horby P, Lim WS, Emberson J, et al. Effect of Dexamethasone in Hospitalized Patients with COVID-19 – Preliminary Report. ''medRxiv''. Published online June 22, 2020:2020.06.22.20137273. doi:10.1101/2020.06.22.20137273
9. Davido B, Lansaman T, Bessis S, et al. Hydroxychloroquine plus azithromycin: a potential interest in reducing in-hospital morbidity due to COVID-19 pneumonia (HI-ZY-COVID)? ''medRxiv''. Published online May 11, 2020:2020.05.05.20088757. doi:10.1101/2020.05.05.20088757
10. Piller C. ‘This is insane!’ Many scientists lament Trump’s embrace of risky malaria drugs for coronavirus. ''Science (80- )''. Published online March 26, 2020. doi:10.1126/SCIENCE.ABB9021
11. Pradhan P, Pandey AK, Mishra A, et al. Uncanny similarity of unique inserts in the 2019-nCoV spike protein to HIV-1 gp120 and Gag. ''bioRxiv''. Published online January 31, 2020:2020.01.30.927871. doi:10.1101/2020.01.30.927871
12. Between fast science and fake news: Preprint servers are political: Impact of Social Sciences. Accessed October 11, 2021. '"`UNIQ--nowiki-000006E4-QINU`"'
13. The European Code of Conduct for Research Integrity. Accessed October 11, 2021. www.allea.org
14. Organization WH. ''Ethical Standards for Research during Public Health Emergencies: Distilling Existing Guidance to Support COVID-19 R&D''. World Health Organization; 2020.
The main method for assessing the efficacy and safety of novel medications and other medical technology is clinical trials. The results of these research, when published in peer-reviewed publications, not only large medical community a scientific foundation for making treatment decisions, but also fund further research. However, pharmaceutical firms with large financial interests in the items being tested finance the majority of clinical trials. Additionally, they frequently pay the scientists who plan, carry out, evaluate, and report clinical trials in the form of wages or consultancy fees. This raises important questions like, should those who have a financial interest in clinical trial results be so invested in its execution? What potential sources of bias may the industry funding of these research have? How are reliable and impartial trial report data ensured by medical journals? Also, it brings a certain stigma and mistrust from the general public.
When searching the literature notes and guidelines can be found on proper steps when publishing a research with funding from external sources. All research articles should have a funding acknowledgement statement included in the manuscript in the form of a sentence under a separate heading entitled ‘Funding’ directly after your Acknowledgements and Declaration of Conflicting Interests, if applicable, and prior to any Notes and your References. The funding agency should be written out in full, followed by the grant number in square brackets, see following example (the text in bold is mandatory, unless specified otherwise by the journal).
A comma and space should be used to separate multiple grant numbers. When more than one agency contributed to funding a study, a semicolon should be used to separate each agency, and the word "and" should come before the last funder.
Industry-funded studies will only be taken into consideration by SAGE publications if sponsorship is explicitly disclosed in the manuscript (where applicable). The funding source's involvement in the study's design, data analysis, and interpretation must also be disclosed by the authors.
According to some researchers, to assure impartiality in clinical research industry-academia partnerships should only continue if academic medical centers take full ownership of the planning, execution, analysis, and reporting of clinical trials. There is an idea of setting up conflict-of-interest committees at academic institutions to maintain a close eye on the financial motivations of both clinician-investigators and institutional decision-makers. Such approaches may aid in reducing the possibility of bias in industry-sponsored research by introducing checks and balances for academic-industry collaborations.
The term HARKing was coined in a seminal article by Kerr<sup>[1]</sup> and is usually used synonymously with accommodational hypothesizing<sup>[2]</sup> and presenting post hoc hypothesis as a priori (PPHA).<sup>[3]</sup> Kerr identified twelve potential costs of HARKing:
1. Translating Type I errors into hard-to-eradicate theory.
2. Propounding theories that cannot (pending replication) pass Popper’s disconfirmability test.
3. Disguising post hoc explanations as a priori explanations (when the former ted also be more ad hoc, and consequently, less useful).
4. Not communicating valuable information about what did not work.
5. Taking unjustified statistical licence.
6. Presenting an inaccurate model of science to students.
7. Encouraging “fudging” in other grey areas.
8. Making us less receptive to serendipitous findings.
9. Encouraging adoption of narrow, context-bound new theory.
10. Encouraging retention of too-broad, disconfirmable old theory.
11. Inhibiting identification of plausible alternative hypotheses.
12. Implicitly violating basic ethical principles.
While Kerr’s article initially was not widely cited, this changed in the wake of the replication crisis and empirical studies into the prevalence and drivers of detrimental research practices and research misconduct. The surge of interest in HARKing worryingly showed that it indeed is a rather prevalent practice. Various studies on the prevalence of detrimental research practices found that a sizeable proportion of researchers (up to 58% in one study) from different disciplines (most notably psychology) did engage in HARKing in the past.<sup>[4]</sup>
To identify measures to reduce HARKing, it is necessary to understand its causes. A key driving factor of HARKing most likely is publication bias: it is much more difficult to publish negative findings than positive findings, and confirmatory research seemingly following a hypothetico-deductive model is generally higher valued than exploratory research, at least in most fields of research. The number of publications, however, still is one of the most important metrics commonly used in researcher evaluation. As a result, researchers have an incentive to publish as much as possible, while the publication system rewards analyses that (seemingly) yield positive findings derived from hypothesis testing research.
One pathway to reduce HARKing thus is changing the incentives for researchers by, for example, evaluating the quality rather than the quantity of publications and recognizing the value of replication studies. The latter also would be facilitated by a comprehensive move towards open science and a recognition of the value of open science practices. Another pathway to reduce HARKing is preregistration because it helps tying the hands of researchers before the data analysis. If researchers decide to preregister a study, they submit a time-stamped paper describing the rationale of their study, the experimental and analytical methods they will use, and their hypotheses. This document cannot be changed at a later stage so that HARKing would be easily detectable and lead to inconsistencies in the line of argument. If the pre-registered study is reviewed, publication is guaranteed if the registered protocol is followed, regardless of the results. Consequently, preregistration and changes in the incentive system are potentially mutually reinforcing.
However, it is worth noting that it is in principle possible to preregister studies after the results are known (PARKing) and thereby reap the reputational benefits coming with what seems to be a commitment to methodological rigor without actually following the practice.<sup>[5]</sup>
Although pure HARKing is unquestionably a detrimental research practice because it misportrays the research process, tends to bias results and ultimately deceives readers, the same cannot necessarily be said about other forms of post hoc hypothesizing. Using the fictional example of a group of epidemiologists conducting a drug trial to cure a new life-threatening disease, Hollenbeck and Wright argue that HARKing is not detrimental to science if it is done transparently and informed by theory, a practice they call THARKing (transparently hypothesizing after the results are known).<sup>[6]</sup> In their example, the epidemiologists initially find no effect of the tested drug, but know of cases where it apparently worked. Discussing about these cases, they recognize that all cured patients they know of are female, yet a reanalysis of the data turns out insignificant, even though the effect size for women is larger than for men. They continue discussing if gender could be an important factor and, drawing on their implicit theoretical knowledge, develop the hypothesis that estrogen levels (that peak at certain ages) might be a crucial moderating variable. A reanalysis of their data corroborates their hypothesis. They publish an article summarizing their study, noting in the discussion section that the age-by-gender interaction was the result of an exploratory analysis conducted after the main effects turned out to be insignificant. Other research teams replicate their study, and eventually a meta-analysis confirms their findings. Hollenbeck and Wright argue that THARKing, unlike secretly hypothesizing after the results are known, SHARKing or pure HARKing), is justifiable if readers are transparently informed that a hypothesis is post hoc rather than a priori in the discussion section of an article (in other words, the introduction in their view should only include a priori hypotheses).
In general, pure HARKing is a detrimental research practice and hampers scientific progress. It can be disincentivized by changes in the research system, such as changes in researcher assessment and increasing preregistration of studies. Transparent post hoc hypothesizing, by contrast, seems justifiable if the exploratory nature of results is clearly stated.
'''References'''
[1] Kerr, N. (1998). HARKing: Hypothesizing After the Results are Known. ''Personality and Social Psychology Review, 2''(3), 196-217. doi:[https://doi.org/10.1207/s15327957pspr0203_4 10.1207/s15327957pspr0203_4]
[2] Hitchcock, C., & Sober, E. (2004). Prediction versus Accommodation and the Risk of Overfitting. ''The British Journal for the Philosophy of Science'', ''55''(1), 1–34. http://www.jstor.org/stable/3541832
[3] Leung, K. (2011). Presenting Post Hoc Hypotheses as A Priori: Ethical and Theoretical Issues. ''Management and Organization Review, 7''(3)'','' 471-479. doi: [https://doi.org/10.1111/j.1740-8784.2011 10.1111/j.1740-8784.2011]
[4] An overview of different studies on the prevalence of HARKing can be found in Table 1 in Rubin, M. (2017). When does HARKing hurt? Identifying when different types of undisclosed post hoc hypothesizing harm scientific progress. ''Review of General Psychology, 21,'' 308-320''.'' doi: [[10.1037/gpr0000128]]
[5] Yamada, Y. (2018). How to Crack Pre-registration: Toward Transparent and Open Science. ''Frontiers in Psychology, 9:1831.'' doi: [https://doi.org/10.3389/fpsyg.2018.01831 10.3389/fpsyg.2018.01831]
[6] Hollenbeck, J. R., & Wright, P. M. (2017). Harking, Sharking, and Tharking: Making the Case for Post Hoc Analysis of Scientific Data. ''International Journal of Qualitative Methods'', ''43''(1), 5-18. [https://doi.org/10.1177/1609406920947600 10.1177/1609406920947600]
Part of the problem is scholarly journals which are prone to only publishing positive results. Changes in publishing policies and fees, especially in the era of digital, publicly available databases and journals, could provide a climate for publishing negative results. Pre-registrations of trials/research can only solve a problem if complete results are published after completion. +
<br />
'"`UNIQ--references-00000289-QINU`"' +
