Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience
Undergraduate Student / Graduate Student
Research Integrity Officer / Committee Member / Committee Member
This module gives an overview of '''tools and resources''' that can be used to '''tailor''' your research integrity training to your audience and '''measure its effectiveness'''.
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# QUERY46367ffa4752e1bd69a6b00e7851f8cd
-Has subobject::Instruction:204c2375-0867-4302-845f-cdc99e3d38bc
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# QUERYd7c62bce2b22c5b52c4683c30029fe21
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# QUERY2aa917532a484e20511a0a1c5a64da3e
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# QUERY8d36b23be3d2f2e74e9a4e8e262ce7a0
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# QUERY7165af2829eafe7c329b535150c6af19
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# QUERY8fd2f68a484afad5e554d924a7eb42af
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# QUERYd2f765b425ed3e0bbe23a6f6a1722c84
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# QUERYd8ccdcc1b50550a28afcec343dc1b4b4
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# QUERY32d7b96a9b43a5c59519aa0e12092ac0
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# QUERY62a01b5e23fd324a26b4c11c0cc79a9a
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# QUERY4adfaf8b3b951261ca69ca854a47930d
Has subobject.Guide Step Element::Instruction:204c2375-0867-4302-845f-cdc99e3d38bc +
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# QUERY0d691a6f8a04ceaa8b72192784c77d6a
In this module you will find examples of training materials on research ethics and integrity that can be used to train at specific career stages. The following introduces a collection of trainings developed by EU-funded initiatives and indicate, which trainings or parts of them are suitable for 1) secondary schools, 2) bachelor and master students, 3) PhD students and early career researchers, 4) academics and experts in research ethics and integrity, and 5) supervisors. Although we have categorised the training materials by career stage, we recommend that trainers and teachers define the needs and skill level of the group to be trained before selecting the material to be used. This is because career stage often does not reflect the level of skills and competences in terms of research ethics and integrity. The following training resources have been developed by EU-funded initiatives namely: PRINTEGER, ENERI, RID-SSISS, EnTIRE, VIRT2UE, Path2Integrity, INTEGRITY, BRIDGE and ROSiE. All the training resources presented in this module provide examples of training materials to promote RE/RI and prevent research misconduct from the perspective of individual and institutional responsibilities. Active learning, like case studies, engages learners and helps connect content to real ethical dilemmas. Reflection helps learners explore personal values, making ethical principles and theories more meaningful and guiding individual growth. Role-playing and simulations offer experiential insights, allowing learners to practice decision-making in a safe environment, fostering empathy and ethical confidence. A culturally sensitive approach prepares learners for global collaboration by recognizing diverse ethical norms. Lastly, continuous assessment and constructive feedback reinforce learning, enabling learners to apply ethical principles thoughtfully in real-life scenarios, supporting long-term commitment to research integrity. Moreover, for each target group this module recommends specific tools and methods to measure the short-, medium- and long-term impact of RE/RI trainings (presented below). While materials and resources have been developed for RE/RI training, they may not always include a plan for the assessment of learning or suggestions on how information about learning can be used as an indicator of training effect. The same training material presented below are also presented in the last section of the BEYOND trainer guide divided by type of training activity and by the specific EU project responsible for its development. This alternative categorization allows for a more tailored approach, enabling users to quickly identify relevant resources that align with their needs, when they already know what training approaches they wish to use, or seeking further guidance related to using specific teaching approaches rather than seeking resources for a specific target group. By connecting each piece of training material to its originating EU project, we can highlight the distinct contributions of each initiative, showcasing their unique approaches to training.
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# af01cf5b421da5ddebcb0f4c878e2bbf
This project has received funding from the European Union's Horizon Europe research and innovation programme under GA No 101094714 (University of Oslo). UK participants in BEYOND are supported by UKRI grant number 10062742 (Trilateral Research) and by UKRI grant number 10067440 (Heriot-Watt University). Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union, Research Executive Agency, or UKRI. Neither the European Union nor the granting authority nor UKRI can be held responsible for them.)
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# 7ff254d01e90893d2a28aa39f3f41f77
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# f3c4366dec7ec59f1903bda041a5cab9
Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience# 5c120aed00383839b3c07e2e82bb923b
The aim of educating secondary school students is to raise their awareness of the ethical and integrity challenges they may face. Resources for secondary school students include (please see the last section of the BEYOND trainer guide for an overview of materials divided by topics and target groups): *The Integrity Teacher Guide for secondary school students' education. *The Path2integrity Learning cards (Path2Integrity learning cards S) focusing on students in high schools, and a dedicated handbook (S-Series handbook). For training effectiveness measurement facilitators can use the following tools for learning output collection and for analysing collected material: {| class="wikitable" |+ !'''Tool for collecting learning outputs''' !'''Details''' !'''Analysis instrument **''' |- |'''ProLearning app''' |''ProLearning'': https://www.epfl.ch/labs/chili/dualt/current-projects/realto/ |learning analytics |- |'''Engagement app''' |''ForgetNot'' (by EduLog): https://web.htk.tlu.ee/forgetnot |learning analytics |- |'''Self-Reflection Form/Compass''' |App under development, [https://docs.google.com/forms/d/17ORaVeaLjBYucufYNGF6TNjgtqqNdlk5BhSp5bfM5eA/copy form] * (for copying and editing) |SOLO taxonomy, reflection levels, content criteria |- |'''Pre-post texts''' |Collect a short text (e.g. a response to a case or short essay) before the training and after the training |SOLO taxonomy, reflection levels, content criteria |- |'''Learning diaries''' |Ask learners keep a diary over a certain period, for each submission provide some guiding questions or topics |SOLO taxonomy, reflection levels, content criteria |- |'''Group reports''' |Ask groups working together to provide a (short) group report (or provide a template with points to work on) |SOLO taxonomy, content criteria |- |'''Group discussions''' |Monitor the group discussions to evaluate the level of understanding and content discussed (scaffold as appropriate) |SOLO taxonomy, content criteria |- |'''Group dynamics''' |''CoTrack'' application: https://www.cotrack.website/en/ |learning analytics |- |'''Retention check''' |After a certain time (few weeks/months) ask learners to provide a short text (analysis of a case, short essay on an ethics topic/question). Compare the levels of understanding to another piece collected during or right after the training. |SOLO taxonomy, content criteria |} For instance, to measure participants’ reactions during or right after the training, ProLearning app or Self-Reflection Form can be used. In addition, if learners worked in groups and provided a group-report, the learning process can be evaluated based on the SOLO taxonomy to measure the levels of understanding. Moreover, if possible, a couple of months after the training an additional case study could be given to the same learners, and the content of their analysis could again be evaluated with the SOLO taxonomy. This kind of effectiveness measure would give a possibility to triangulate the measurement in different time points.
Training research ethics and integrity experts is vital to promoting a culture of responsible research and safeguarding public trust science. These experts play a key role in developing policies, advising researchers, and providing education on ethical practices, data management, and compliance with regulations. Resources for research ethics and integrity experts include (please see the last section of the BEYOND trainer guide for an overview of materials divided by topics and target groups): *The [https://eneri.eu/eneri-classroom/ ENERI Classroom] material exploring [https://classroom.eneri.eu/research-integrity research integrity], [https://classroom.eneri.eu/research-ethics research ethics], [https://classroom.eneri.eu/overlapping-issues Comprehensive and overlapping issues] and the [https://classroom.eneri.eu/node/63 development of proper infrastructure] relevant for academics, and research ethics and integrity experts. *The dilemmas developed by the Upright training ([https://printeger.eu/upright/toc/dilemma-with-a-little-help/ With a little help], [https://printeger.eu/upright/toc/dilemma-mutual-favours/ Mutual favours], [https://printeger.eu/upright/toc/dilemma-sharing-data/ Sharing data], [https://printeger.eu/upright/toc/dilemma-so-close/ So close], [https://printeger.eu/upright/toc/dilemma-different-results/ Different results], [https://printeger.eu/upright/toc/dilemma-put-your-supervisor-first/ Put your supervisor first], [https://printeger.eu/upright/toc/dilemma-flexible-scope/ Flexible scope], [https://printeger.eu/upright/toc/dilemma-outliers/ Outliers], and [https://printeger.eu/upright/toc/dilemma-invalid-data/ Invalid data]) *The scenarios produced by EnTIRE for The Embassy of Good Science s [https://embassy.science/wiki-wiki/index.php/Resource:F6100097-fddb-4c77-9098-1bc767c34a6a Research Procedures and Research Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:1d26fd13-1ced-44bc-8d19-e094b37f8f70 Collaborative Working Between Academia and Industry];[https://embassy.science/wiki-wiki/index.php/Resource:45a04c31-5a75-4816-8484-2dd9b71d1674 Data Practices, Data Management and FAIR Principles];[https://embassy.science/wiki-wiki/index.php/Resource:Aef6b98d-9cc5-4db0-bffd-4a3daa99a3f3 Publication, Dissemination and Research Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:C99f17ec-3d1e-4f7a-bfc7-3e3607934ead Research Environments and Research Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:7f7810d8-74a2-42ac-906c-7f6a73fcd183 Reviewing, Evaluating, Editing and Research Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:67caae86-68db-49ea-8305-2010fe701aa6 Training, Supervision and Mentoring with Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:E99e20d0-8116-4d77-84ec-7df396703bf4 Safeguards, Data-sharing and the Disclosure of Sensitive Results].
The professionalization of research integrity and ethics trainers is crucial for ensuring high-quality, consistent, and impactful education that fosters a culture of ethical research practices and responsible scientific conduct. The VIRT2UE project designed a Train-the-Trainer training programme with a virtue ethics approach to Ethics and Research Integrity. The VIRT2UE training programme takes a blended learning approach and consists of four components: 1) online course (consisting of 4 modules, addressing introduction to research integrity, introduction of virtue ethics to research integrity, virtue ethics under current research conditions and supervision and mentorship practices, preparatory readings and preparatory watching;2) two consecutive participatory session (during which participants experience five participatory exercises) : Debate and Dialogue, Virtues and Norms, The Middle Position, the Modified Dilemma Game, and the Self-Declaration Approach;3) interim practice work (during which trainers in training go back to their institution practice with the exercises and reflect on their own teaching practices by means of guided self-reflection) and 4) follow up participatory group/reflection section, in which future trainers can reflect on the experience, learn about teaching strategies and reflect on their own teaching style and audience. The train-the-trainer program developed precise instructions for trainers on how to prepare, organize and facilitate the section. These instructions can be found by clicking on the trainer tab on the training webpage.
Training Materials for Training Research Integrity and Ethics Trainers
The aim of training students in research ethics and integrity is to promote an understanding of ethical principles and practices in research. This includes exploring aspects of honesty, transparency, objectivity and accountability. Although at these levels, students are marginally involved in doing research, it is important to equip them with the knowledge and skills to conduct research responsibly, avoid misconduct such as plagiarism or falsification, comply with relevant regulations and guidelines, and uphold the integrity of the scientific community. Resources for Bachelor and master students include (please see the last section of the BEYOND trainer guide for an overview of materials divided by topics and target groups): *The [https://printeger.eu/upright/toc/obas-introduction/ introductory module][https://printeger.eu/upright/toc/ , and modules focusing specific RE and/or RI issues of the Upright training] developed by the [https://printeger.eu/ PRINTEGER] project *The e-learning modules on research integrity, virtue ethics relevant for RI, the series of introductory videos and The Modified Rotterdam Dilemma Game developed by the VIRT2UE project. *The introductory videos and [https://www.path2integrity.eu/ri-materials information on the teaching methodology discussing research integrity and its significance], the series of learning cards ([https://www.path2integrity.eu/ri-materials Path2Integrity learning cards S]) focusing on bachelor students, alongside a dedicated handbook ([https://www.path2integrity.eu/ri-materials S-Series handbook]), and a series of learning cards ([https://www.path2integrity.eu/ri-materials Path2Integrity learning cards M]) focusing on bachelor students, alongside a dedicated handbook [https://www.path2integrity.eu/ri-materials (M-Series handbook]) developed byt the Path2Integrity project. *The [https://en.researchethicscompass.net/ RID-SSISS training Foundation level] focuses on developing (but also helping learners to recall) central concepts of RE/RI *The [https://www.academicintegrity.eu/wp/bridge/ BRIDGE project] [https://www.academicintegrity.eu/wp/bridge-modules-2/ training modules] and [https://www.academicintegrity.eu/wp/vignettes_interactive/ vignettes] *The INTEGRITY project training tool for [https://h2020integrity.eu/toolkit/tools-undergraduate-students/integrity-games/ undergraduate]. *The [https://www.academicintegrity.eu/wp/bridge/ BRIDGE project] online games, board games, role-play games and scenarios game can be found [https://www.academicintegrity.eu/wp/bridge-games/ here]. *The [https://rosie-project.eu/ ROSiE] Training Materials for Responsible Open Science and the learning resources produced by the ROSIE project available on [https://rosie-project.eu/rosie-knowledge-hub/ ROSiE Knowledge Hub]. Trainers can select one or more of the following tools for evaluating training effectiveness for bachelor and master students. {| class="wikitable" |+Table 5: Tools for evaluating training effectiveness for bachelor and master students !'''Tool for collecting learning outputs''' !'''Details''' !'''Analysis instrument **''' |- |'''ProLearning app''' |''ProLearning'': [http://www.prolearning.realto.ch/ www.prolearning.realto.ch]https://www.epfl.ch/labs/chili/dualt/current-projects/realto/ |learning analytics |- |'''Engagement app''' |''ForgetNot'' (by EduLog): https://web.htk.tlu.ee/forgetnot |learning analytics |- |'''Self-Reflection Form/Compass''' |App under development, [https://docs.google.com/forms/d/1f4xNbQka73bfeDtwKCXTC5W5CoyhfVrtr4dwMXRNmWk/copy form] * (for copying and editing) |SOLO taxonomy, reflection levels, content criteria |- |'''Pre-post texts''' |Collect a short text (e.g. a response to a case or short essay) before the training and after the training |SOLO taxonomy, reflection levels, content criteria |- |'''Learning diaries''' |Ask learners keep a diary over a certain period, for each submission provide some guiding questions or topics |SOLO taxonomy, reflection levels, content criteria |- |'''Group reports''' |Ask groups working together to provide a (short) group report (or provide a template with points to work on) |SOLO taxonomy, content criteria |- |'''Group discussions''' |Monitor the group discussions to evaluate the level of understanding and content discussed (scaffold as appropriate) |SOLO taxonomy, content criteria |- |'''Group dynamics''' |''CoTrack'' application: https://www.cotrack.website/en/ |learning analytics |- |'''Online learning platform''' |Make use of accumulated authentic learning outputs in the learning platform |statistics, SOLO taxonomy, reflection scale, content criteria |- |'''Domain-specific/ domain-transcending measure''' |Use either of the two forms (WP4.2) measuring recognition and exemplifying of ethical issues |statistics, SOLO taxonomy, content criteria |- |'''Retention check''' |After a certain time (few weeks/months) ask learners to provide a short text (analysis of a case, short essay on an ethics topic/question). Compare the levels of understanding to another piece collected during or right after the training.[https://amsterdamumc-my.sharepoint.com/personal/g_inguaggiato_amsterdamumc_nl/Documents/!Oude%20N%20schijf/Documenten/Beyond/Training%20guide%20V7_GI.docx#_msocom_1 '"`UNIQ--nowiki-00000010-QINU`"'] [https://amsterdamumc-my.sharepoint.com/personal/g_inguaggiato_amsterdamumc_nl/Documents/!Oude%20N%20schijf/Documenten/Beyond/Training%20guide%20V7_GI.docx#_msocom_2 '"`UNIQ--nowiki-00000011-QINU`"'] |SOLO taxonomy, content criteria |} For instance, to measure participants’ reactions during or right after the training, ProLearning app or Self-Reflection Form can be used. In addition, if learners worked in groups so their group discussions can be monitored, and if they provided a group-report, the learning process can be evaluated based on the SOLO taxonomy to measure the levels of understanding. Moreover, if possible, a couple of months after the training an additional case study could be given to the same learners, and the content of their analysis could again be evaluated with the SOLO taxonomy. With this target group the domain-specific and domain-transcending measure could be implemented. This kind of effectiveness measure would give a possibility to triangulate the measurement in different time points. An example for implementation can be found here: [https://helsinkifi.sharepoint.com/:p:/r/sites/BEYONDHelsinkiteam/Shared%20Documents/ENERI%20CR%20material%20example%20for%20ba_ma%20students.pptx?d=wdfb62d452de140f6a5a8df15611b48e2&csf=1&web=1&e=z2OYsx ENERI CR material example for ba_ma students.pptx] '"`UNIQ--nowiki-00000012-QINU`"' The Self-Reflection Form link enables the facilitator to make a copy of the form, which they can then edit and the data will accumulate on the facilitator’s cloud service (Google or Microsoft). '"`UNIQ--nowiki-00000013-QINU`"' Analysis instruments are described in WP4.2, later available at the Embassy’s website.
The goal in training students and PhD candidates in RE/RI is to support ever-deepening understanding of ethical principles and practices in research. This includes exploring aspects of honesty, transparency, objectivity and accountability at all stages of the research process, from conception to dissemination of results. The goal is to equip students with the knowledge and skills to conduct research responsibly, avoid misconduct such as plagiarism or falsification, comply with relevant regulations and guidelines, and uphold the integrity of the scientific community. Although these examples can be used to train students and PhD candidates, they can be used in training more senior profiles to brush up competencies on the topic. Resources for PhD candidates include (please see the last section of the BEYOND trainer guide for an overview of materials divided by topics and target groups): *The [https://printeger.eu/upright/toc/obas-introduction/ introductory module][https://printeger.eu/upright/toc/ , the modules focusing specific RE and/or RI issues anf the dilemmas] ([https://printeger.eu/upright/toc/dilemma-with-a-little-help/ With a little help][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-mutual-favours/ Mutual favours][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-sharing-data/ Sharing data][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-so-close/ So close][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-different-results/ Different results][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-put-your-supervisor-first/ Put your supervisor first][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-flexible-scope/ Flexible scope][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-outliers/ Outliers][https://printeger.eu/upright/toc/ , and] [https://printeger.eu/upright/toc/dilemma-invalid-data/ Invalid data]) developed for [https://printeger.eu/upright/toc/ the Upright training] by [https://printeger.eu/ PRINTEGER]. *The e-learning modules on research integrity, virtue ethics relevant for RI, virtue ethics under current research conditions and the a series of introductory videos produced by the VIRT2UE. The Modified Dilemma Game developed by the VIRT2UE project. *The introductory videos and information on the teaching methodology and the learning cards ([https://www.path2integrity.eu/ri-materials Path2Integrity learning cards Y]) focusing on doctoral students, alongside a dedicated handbook ([https://www.path2integrity.eu/ri-materials Y-Series handbook]) developed by [https://www.path2integrity.eu/ri-materials Path2Integrity Training Programme] *The RID-SSISS training materials for ECRs and junior academics ([https://www.researchethicstraining.net/ advanced level]). *The [https://www.academicintegrity.eu/wp/bridge/ BRIDGE project] provides [https://www.academicintegrity.eu/wp/bridge-modules-2/ training modules] and [https://www.academicintegrity.eu/wp/vignettes_interactive/ vignettes] that can be inserted into research ethics and integrity courses. *The [https://h2020integrity.eu/toolkit/tools-phd-students/ modules and a full interactive training] developed by the [https://h2020integrity.eu/ INTEGRITY] project. *The scenarios produced by EnTIRE for The Embassy of Good Science ([https://embassy.science/wiki-wiki/index.php/Resource:F6100097-fddb-4c77-9098-1bc767c34a6a Research Procedures and Research Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:1d26fd13-1ced-44bc-8d19-e094b37f8f70 Collaborative Working Between Academia and Industry];[https://embassy.science/wiki-wiki/index.php/Resource:45a04c31-5a75-4816-8484-2dd9b71d1674 Data Practices, Data Management and FAIR Principles];[https://embassy.science/wiki-wiki/index.php/Resource:Aef6b98d-9cc5-4db0-bffd-4a3daa99a3f3 Publication, Dissemination and Research Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:C99f17ec-3d1e-4f7a-bfc7-3e3607934ead Research Environments and Research Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:7f7810d8-74a2-42ac-906c-7f6a73fcd183 Reviewing, Evaluating, Editing and Research Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:67caae86-68db-49ea-8305-2010fe701aa6 Training, Supervision and Mentoring with Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:E99e20d0-8116-4d77-84ec-7df396703bf4 Safeguards, Data-sharing and the Disclosure of Sensitive Results]). Trainers can select one or more of the following tools for evaluating training effectiveness for PhD candidates and early career researchers: {| class="wikitable" |+ Table 6: BEYOND Tools for evaluating training effectiveness for early-career researchers !'''Tool for collecting learning outputs''' !'''Details''' !'''Analysis instrument **''' |- |'''Self-Reflection Form/Compass''' |App under development, [https://forms.office.com/Pages/ShareFormPage.aspx?id=WXWumNwQiEKOLkWT5i_j7twYn7PlpvpDlgGDpz2LgIdUMk5XRTVYQTVKRFRDWDlHOUdGU1FHTUlFVi4u&sharetoken=03epmvYBRpmfXvpRg9os form] * (for copying and editing) |SOLO taxonomy, reflection levels, content criteria |- |'''Pre-post texts''' |Collect a short text (e.g. a response to a case or short essay) before the training and after the training |SOLO taxonomy, reflection levels, content criteria |- |'''Learning diaries''' |Ask learners keep a diary over a certain period, for each submission provide some guiding questions or topics |SOLO taxonomy, reflection levels, content criteria |- |'''Group reports''' |Ask groups working together to provide a (short) group report (or provide a template with points to work on) |SOLO taxonomy, content criteria |- |'''Group discussions''' |Monitor the group discussions to evaluate the level of understanding and content discussed (scaffold as appropriate) |SOLO taxonomy, content criteria |- |'''Group dynamics''' |''CoTrack'' application: https://www.cotrack.website/en/ |learning analytics |- |'''Online learning platform''' |Make use of accumulated authentic learning outputs in the learning platform. |statistics, SOLO taxonomy, reflection scale, content criteria |- |'''Domain-specific/ domain-transcending measure''' |Use either of the two forms (WP4.2) measuring recognition and exemplifying of ethical issues. |statistics, SOLO taxonomy, content criteria |- |'''Retention check''' |After a certain time (few weeks/months) ask learners to provide a short text (analysis of a case, short essay on an ethics topic/question). Compare the levels of understanding to another piece collected during or right after the training. |SOLO taxonomy, content criteria |- |'''Vignettes''' |This can be used for measuring ethical sensitivity in (non-)training context |statistics, EASM (based on the SOLO taxonomy), content criteria |- |'''National surveys''' |Can be used for analysing training-related content in reports and monitoring the display of REI leadership. |statistics, REI leadership framework |} For instance, to measure participants’ reactions during or right after the training, Self-Reflection Form can be used. In addition, if learners worked in groups so their group discussions can be monitored, and if they provided a group-report, the learning process can be evaluated based on the SOLO taxonomy to measure the levels of understanding. Moreover, if possible, a couple of months after the training an additional case study could be given to the same learners, and the content of their analysis could again be evaluated with the SOLO taxonomy. With this target group the domain-specific and domain-transcending measure could be implemented. This kind of effectiveness measure would give a possibility to triangulate the measurement in different time points. Examples for implementation can be found here: [https://helsinkifi.sharepoint.com/:p:/r/sites/BEYONDHelsinkiteam/Shared%20Documents/ENERI%20CR%20material%20example%20for%20ECRs.pptx?d=w10c8dc6f452042fdae8775faf52ca081&csf=1&web=1&e=NAqYpk ENERI CR material example for ECRs.pptx] '"`UNIQ--nowiki-00000014-QINU`"' The Self-Reflection Form link enables the facilitator to make a copy of the form, which they can then edit, and the data will accumulate on the facilitator’s cloud service (Google or Microsoft). '"`UNIQ--nowiki-00000015-QINU`"' Analysis instruments are described in WP4.2, later available at the Embassy’s website.
Doctoral Students and Early Career Researchers
When training professors, trainers, mentors, and supervisors in research integrity, the aim is to equip them with the knowledge, skills, and resources to effectively train, mentor and guide their students and junior colleagues in conducting research with integrity. Resources for professors and senior academics include (please see the last section of the BEYOND trainer guide for an overview of materials divided by topics and target groups): *The VIRT2UE train the trainer course on research integrity and the training module introducing supervision and mentorship practices. *The training materials to foster mentorship and supervision developed by [https://h2020integrity.eu/toolkit/tools-researchers-supervisors/ INTEGRITY]. *The training for supervisors and leaders ([https://www.researchethicstraining.net/leadershiplevel leadership level]) developed by the RID-SSISS project. *The [https://www.academicintegrity.eu/wp/bridge-module-for-supervisors/ module for supervisors] developed by [https://www.academicintegrity.eu/wp/bridge/ BRIDGE project]. Trainers can select one or more of the following tools for evaluating training effectiveness for Professors and Senior Academics: {| class="wikitable" |+Table 7: BEYOND Tools for evaluating training effectiveness for academics and experts !'''Tool for collecting learning outputs''' !'''Details''' !'''Analysis instrument **''' |- |'''Self-Reflection Form/Compass''' |App under development, [https://forms.office.com/Pages/ShareFormPage.aspx?id=WXWumNwQiEKOLkWT5i_j7twYn7PlpvpDlgGDpz2LgIdUMk5XRTVYQTVKRFRDWDlHOUdGU1FHTUlFVi4u&sharetoken=03epmvYBRpmfXvpRg9os form] * (for copying and editing) |SOLO taxonomy, reflection levels, content criteria |- |'''Pre-post texts''' |Collect a short text (e.g. a response to a case or short essay) before the training and after the training |SOLO taxonomy, reflection levels, content criteria |- |'''Learning diaries''' |Ask learners keep a diary over a certain period, for each submission provide some guiding questions or topics |SOLO taxonomy, reflection levels, content criteria |- |'''Group reports''' |Ask groups working together to provide a (short) group report (or provide a template with points to work on) |SOLO taxonomy, content criteria |- |'''Group discussions''' |Monitor the group discussions to evaluate the level of understanding and content discussed (scaffold as appropriate) |SOLO taxonomy, content criteria |- |'''Retention check''' |After a certain time (few weeks/months) ask learners to provide a short text (analysis of a case, short essay on an ethics topic/question). Compare the levels of understanding to another piece collected during or right after the training. |SOLO taxonomy, content criteria |- |'''Vignettes''' |This can be used for measuring ethical sensitivity in (non-)training context. |statistics, EASM (based on the SOLO taxonomy), content criteria |- |'''National surveys''' |Can be used for analysing training-related content in reports and monitoring the display of REI leadership. |statistics, REI leadership framework |} For instance, to measure participants’ reactions during or right after the training, Self-Reflection Form can be used. In addition, if learners worked in groups so their group discussions can be monitored, and if they provided a group-report or pre- and post-texts, the learning process can be evaluated based on the SOLO taxonomy to measure the levels of understanding. Moreover, if possible, a couple of months after the training an additional case study could be given to the same learners, and the content of their analysis could again be evaluated with the SOLO taxonomy. Analysing vignettes and participating in national REI surveys would be relevant for this target group. This kind of effectiveness measure would give a possibility to triangulate the measurement in different time points. An example for implementation can be found here: [https://helsinkifi.sharepoint.com/:p:/r/sites/BEYONDHelsinkiteam/Shared%20Documents/ENERI%20CR%20material%20example.pptx?d=w41ec1fe12bbb495781dbe81d55a09d71&csf=1&web=1&e=yxKgSC ENERI CR material example.pptx] '"`UNIQ--nowiki-00000016-QINU`"' The Self-Reflection Form link enables the facilitator to make a copy of the form, which they can then edit, and the data will accumulate on the facilitator’s cloud service (Google or Microsoft). '"`UNIQ--nowiki-00000017-QINU`"' Analysis instruments are described in WP4.2, later available at the Embassy’s website.
Training Materials for Professors and Senior Academics
The aim of training students in research ethics and integrity is to promote an understanding of ethical principles and practices in research. This includes exploring aspects of honesty, transparency, objectivity and accountability. Although at these levels, students are marginally involved in doing research, it is important to equip them with the knowledge and skills to conduct research responsibly, avoid misconduct such as plagiarism or falsification, comply with relevant regulations and guidelines, and uphold the integrity of the scientific community. Resources for Bachelor and master students include (please see the last section of the BEYOND trainer guide for an overview of materials divided by topics and target groups): *The [https://printeger.eu/upright/toc/obas-introduction/ introductory module][https://printeger.eu/upright/toc/ , and modules focusing specific RE and/or RI issues of the Upright training] developed by the [https://printeger.eu/ PRINTEGER] project *The e-learning modules on research integrity, virtue ethics relevant for RI, the series of introductory videos and The Modified Rotterdam Dilemma Game developed by the VIRT2UE project. *The introductory videos and [https://www.path2integrity.eu/ri-materials information on the teaching methodology discussing research integrity and its significance], the series of learning cards ([https://www.path2integrity.eu/ri-materials Path2Integrity learning cards S]) focusing on bachelor students, alongside a dedicated handbook ([https://www.path2integrity.eu/ri-materials S-Series handbook]), and a series of learning cards ([https://www.path2integrity.eu/ri-materials Path2Integrity learning cards M]) focusing on bachelor students, alongside a dedicated handbook [https://www.path2integrity.eu/ri-materials (M-Series handbook]) developed byt the Path2Integrity project. *The [https://en.researchethicscompass.net/ RID-SSISS training Foundation level] focuses on developing (but also helping learners to recall) central concepts of RE/RI *The [https://www.academicintegrity.eu/wp/bridge/ BRIDGE project] [https://www.academicintegrity.eu/wp/bridge-modules-2/ training modules] and [https://www.academicintegrity.eu/wp/vignettes_interactive/ vignettes] *The INTEGRITY project training tool for [https://h2020integrity.eu/toolkit/tools-undergraduate-students/integrity-games/ undergraduate]. *The [https://www.academicintegrity.eu/wp/bridge/ BRIDGE project] online games, board games, role-play games and scenarios game can be found [https://www.academicintegrity.eu/wp/bridge-games/ here]. *The [https://rosie-project.eu/ ROSiE] Training Materials for Responsible Open Science and the learning resources produced by the ROSIE project available on [https://rosie-project.eu/rosie-knowledge-hub/ ROSiE Knowledge Hub]. Trainers can select one or more of the following tools for evaluating training effectiveness for bachelor and master students. {| class="wikitable" |+Table 5: Tools for evaluating training effectiveness for bachelor and master students !'''Tool for collecting learning outputs''' !'''Details''' !'''Analysis instrument **''' |- |'''ProLearning app''' |''ProLearning'': https://www.epfl.ch/labs/chili/dualt/current-projects/realto/ |learning analytics |- |'''Engagement app''' |''ForgetNot'' (by EduLog): https://web.htk.tlu.ee/forgetnot |learning analytics |- |'''Self-Reflection Form/Compass''' |App under development, [https://docs.google.com/forms/d/1f4xNbQka73bfeDtwKCXTC5W5CoyhfVrtr4dwMXRNmWk/copy form] * (for copying and editing) |SOLO taxonomy, reflection levels, content criteria |- |'''Pre-post texts''' |Collect a short text (e.g. a response to a case or short essay) before the training and after the training |SOLO taxonomy, reflection levels, content criteria |- |'''Learning diaries''' |Ask learners keep a diary over a certain period, for each submission provide some guiding questions or topics |SOLO taxonomy, reflection levels, content criteria |- |'''Group reports''' |Ask groups working together to provide a (short) group report (or provide a template with points to work on) |SOLO taxonomy, content criteria |- |'''Group discussions''' |Monitor the group discussions to evaluate the level of understanding and content discussed (scaffold as appropriate) |SOLO taxonomy, content criteria |- |'''Group dynamics''' |''CoTrack'' application: https://www.cotrack.website/en/ |learning analytics |- |'''Online learning platform''' |Make use of accumulated authentic learning outputs in the learning platform |statistics, SOLO taxonomy, reflection scale, content criteria |- |'''Domain-specific/ domain-transcending measure''' |Use either of the two forms (WP4.2) measuring recognition and exemplifying of ethical issues |statistics, SOLO taxonomy, content criteria |- |'''Retention check''' |After a certain time (few weeks/months) ask learners to provide a short text (analysis of a case, short essay on an ethics topic/question). Compare the levels of understanding to another piece collected during or right after the training.[https://amsterdamumc-my.sharepoint.com/personal/g_inguaggiato_amsterdamumc_nl/Documents/!Oude%20N%20schijf/Documenten/Beyond/Training%20guide%20V7_GI.docx#_msocom_1 '"`UNIQ--nowiki-00000018-QINU`"'] [https://amsterdamumc-my.sharepoint.com/personal/g_inguaggiato_amsterdamumc_nl/Documents/!Oude%20N%20schijf/Documenten/Beyond/Training%20guide%20V7_GI.docx#_msocom_2 '"`UNIQ--nowiki-00000019-QINU`"'] |SOLO taxonomy, content criteria |} For instance, to measure participants’ reactions during or right after the training, ProLearning app or Self-Reflection Form can be used. In addition, if learners worked in groups so their group discussions can be monitored, and if they provided a group-report, the learning process can be evaluated based on the SOLO taxonomy to measure the levels of understanding. Moreover, if possible, a couple of months after the training an additional case study could be given to the same learners, and the content of their analysis could again be evaluated with the SOLO taxonomy. With this target group the domain-specific and domain-transcending measure could be implemented. This kind of effectiveness measure would give a possibility to triangulate the measurement in different time points. An example for implementation can be found here: [https://helsinkifi.sharepoint.com/:p:/r/sites/BEYONDHelsinkiteam/Shared%20Documents/ENERI%20CR%20material%20example%20for%20ba_ma%20students.pptx?d=wdfb62d452de140f6a5a8df15611b48e2&csf=1&web=1&e=z2OYsx ENERI CR material example for ba_ma students.pptx] '"`UNIQ--nowiki-0000001A-QINU`"' The Self-Reflection Form link enables the facilitator to make a copy of the form, which they can then edit and the data will accumulate on the facilitator’s cloud service (Google or Microsoft). '"`UNIQ--nowiki-0000001B-QINU`"' Analysis instruments are described in WP4.2, later available at the Embassy’s website.
The goal in training students and PhD candidates in RE/RI is to support ever-deepening understanding of ethical principles and practices in research. This includes exploring aspects of honesty, transparency, objectivity and accountability at all stages of the research process, from conception to dissemination of results. The goal is to equip students with the knowledge and skills to conduct research responsibly, avoid misconduct such as plagiarism or falsification, comply with relevant regulations and guidelines, and uphold the integrity of the scientific community. Although these examples can be used to train students and PhD candidates, they can be used in training more senior profiles to brush up competencies on the topic. Resources for PhD candidates include (please see the last section of the BEYOND trainer guide for an overview of materials divided by topics and target groups): *The [https://printeger.eu/upright/toc/obas-introduction/ introductory module][https://printeger.eu/upright/toc/ , the modules focusing specific RE and/or RI issues anf the dilemmas] ([https://printeger.eu/upright/toc/dilemma-with-a-little-help/ With a little help][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-mutual-favours/ Mutual favours][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-sharing-data/ Sharing data][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-so-close/ So close][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-different-results/ Different results][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-put-your-supervisor-first/ Put your supervisor first][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-flexible-scope/ Flexible scope][https://printeger.eu/upright/toc/ ,] [https://printeger.eu/upright/toc/dilemma-outliers/ Outliers][https://printeger.eu/upright/toc/ , and] [https://printeger.eu/upright/toc/dilemma-invalid-data/ Invalid data]) developed for [https://printeger.eu/upright/toc/ the Upright training] by [https://printeger.eu/ PRINTEGER]. *The e-learning modules on research integrity, virtue ethics relevant for RI, virtue ethics under current research conditions and the a series of introductory videos produced by the VIRT2UE. The Modified Dilemma Game developed by the VIRT2UE project. *The introductory videos and information on the teaching methodology and the learning cards ([https://www.path2integrity.eu/ri-materials Path2Integrity learning cards Y]) focusing on doctoral students, alongside a dedicated handbook ([https://www.path2integrity.eu/ri-materials Y-Series handbook]) developed by [https://www.path2integrity.eu/ri-materials Path2Integrity Training Programme] *The RID-SSISS training materials for ECRs and junior academics ([https://www.researchethicstraining.net/ advanced level]). *The [https://www.academicintegrity.eu/wp/bridge/ BRIDGE project] provides [https://www.academicintegrity.eu/wp/bridge-modules-2/ training modules] and [https://www.academicintegrity.eu/wp/vignettes_interactive/ vignettes] that can be inserted into research ethics and integrity courses. *The [https://h2020integrity.eu/toolkit/tools-phd-students/ modules and a full interactive training] developed by the [https://h2020integrity.eu/ INTEGRITY] project. *The scenarios produced by EnTIRE for The Embassy of Good Science ([https://embassy.science/wiki-wiki/index.php/Resource:F6100097-fddb-4c77-9098-1bc767c34a6a Research Procedures and Research Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:1d26fd13-1ced-44bc-8d19-e094b37f8f70 Collaborative Working Between Academia and Industry];[https://embassy.science/wiki-wiki/index.php/Resource:45a04c31-5a75-4816-8484-2dd9b71d1674 Data Practices, Data Management and FAIR Principles];[https://embassy.science/wiki-wiki/index.php/Resource:Aef6b98d-9cc5-4db0-bffd-4a3daa99a3f3 Publication, Dissemination and Research Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:C99f17ec-3d1e-4f7a-bfc7-3e3607934ead Research Environments and Research Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:7f7810d8-74a2-42ac-906c-7f6a73fcd183 Reviewing, Evaluating, Editing and Research Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:67caae86-68db-49ea-8305-2010fe701aa6 Training, Supervision and Mentoring with Integrity];[https://embassy.science/wiki-wiki/index.php/Resource:E99e20d0-8116-4d77-84ec-7df396703bf4 Safeguards, Data-sharing and the Disclosure of Sensitive Results]). Trainers can select one or more of the following tools for evaluating training effectiveness for PhD candidates and early career researchers: {| class="wikitable" |+ Table 6: BEYOND Tools for evaluating training effectiveness for early-career researchers !'''Tool for collecting learning outputs''' !'''Details''' !'''Analysis instrument **''' |- |'''Self-Reflection Form/Compass''' |App under development, [https://forms.office.com/Pages/ShareFormPage.aspx?id=WXWumNwQiEKOLkWT5i_j7twYn7PlpvpDlgGDpz2LgIdUMk5XRTVYQTVKRFRDWDlHOUdGU1FHTUlFVi4u&sharetoken=03epmvYBRpmfXvpRg9os form] * (for copying and editing) |SOLO taxonomy, reflection levels, content criteria |- |'''Pre-post texts''' |Collect a short text (e.g. a response to a case or short essay) before the training and after the training |SOLO taxonomy, reflection levels, content criteria |- |'''Learning diaries''' |Ask learners keep a diary over a certain period, for each submission provide some guiding questions or topics |SOLO taxonomy, reflection levels, content criteria |- |'''Group reports''' |Ask groups working together to provide a (short) group report (or provide a template with points to work on) |SOLO taxonomy, content criteria |- |'''Group discussions''' |Monitor the group discussions to evaluate the level of understanding and content discussed (scaffold as appropriate) |SOLO taxonomy, content criteria |- |'''Group dynamics''' |''CoTrack'' application: https://www.cotrack.website/en/ |learning analytics |- |'''Online learning platform''' |Make use of accumulated authentic learning outputs in the learning platform. |statistics, SOLO taxonomy, reflection scale, content criteria |- |'''Domain-specific/ domain-transcending measure''' |Use either of the two forms (WP4.2) measuring recognition and exemplifying of ethical issues. |statistics, SOLO taxonomy, content criteria |- |'''Retention check''' |After a certain time (few weeks/months) ask learners to provide a short text (analysis of a case, short essay on an ethics topic/question). Compare the levels of understanding to another piece collected during or right after the training. |SOLO taxonomy, content criteria |- |'''Vignettes''' |This can be used for measuring ethical sensitivity in (non-)training context |statistics, EASM (based on the SOLO taxonomy), content criteria |- |'''National surveys''' |Can be used for analysing training-related content in reports and monitoring the display of REI leadership. |statistics, REI leadership framework |} For instance, to measure participants’ reactions during or right after the training, Self-Reflection Form can be used. In addition, if learners worked in groups so their group discussions can be monitored, and if they provided a group-report, the learning process can be evaluated based on the SOLO taxonomy to measure the levels of understanding. Moreover, if possible, a couple of months after the training an additional case study could be given to the same learners, and the content of their analysis could again be evaluated with the SOLO taxonomy. With this target group the domain-specific and domain-transcending measure could be implemented. This kind of effectiveness measure would give a possibility to triangulate the measurement in different time points. Examples for implementation can be found here: [https://helsinkifi.sharepoint.com/:p:/r/sites/BEYONDHelsinkiteam/Shared%20Documents/ENERI%20CR%20material%20example%20for%20ECRs.pptx?d=w10c8dc6f452042fdae8775faf52ca081&csf=1&web=1&e=NAqYpk ENERI CR material example for ECRs.pptx] '"`UNIQ--nowiki-0000001C-QINU`"' The Self-Reflection Form link enables the facilitator to make a copy of the form, which they can then edit, and the data will accumulate on the facilitator’s cloud service (Google or Microsoft). '"`UNIQ--nowiki-0000001D-QINU`"' Analysis instruments are described in WP4.2, later available at the Embassy’s website.
When training professors, trainers, mentors, and supervisors in research integrity, the aim is to equip them with the knowledge, skills, and resources to effectively train, mentor and guide their students and junior colleagues in conducting research with integrity. Resources for professors and senior academics include (please see the last section of the BEYOND trainer guide for an overview of materials divided by topics and target groups): *The VIRT2UE train the trainer course on research integrity and the training module introducing supervision and mentorship practices. *The training materials to foster mentorship and supervision developed by [https://h2020integrity.eu/toolkit/tools-researchers-supervisors/ INTEGRITY]. *The training for supervisors and leaders ([https://www.researchethicstraining.net/leadershiplevel leadership level]) developed by the RID-SSISS project. *The [https://www.academicintegrity.eu/wp/bridge-module-for-supervisors/ module for supervisors] developed by [https://www.academicintegrity.eu/wp/bridge/ BRIDGE project]. Trainers can select one or more of the following tools for evaluating training effectiveness for Professors and Senior Academics: {| class="wikitable" |+Table 7: BEYOND Tools for evaluating training effectiveness for academics and experts !'''Tool for collecting learning outputs''' !'''Details''' !'''Analysis instrument **''' |- |'''Self-Reflection Form/Compass''' |App under development, [https://forms.office.com/Pages/ShareFormPage.aspx?id=WXWumNwQiEKOLkWT5i_j7twYn7PlpvpDlgGDpz2LgIdUMk5XRTVYQTVKRFRDWDlHOUdGU1FHTUlFVi4u&sharetoken=03epmvYBRpmfXvpRg9os form] * (for copying and editing) |SOLO taxonomy, reflection levels, content criteria |- |'''Pre-post texts''' |Collect a short text (e.g. a response to a case or short essay) before the training and after the training |SOLO taxonomy, reflection levels, content criteria |- |'''Learning diaries''' |Ask learners keep a diary over a certain period, for each submission provide some guiding questions or topics |SOLO taxonomy, reflection levels, content criteria |- |'''Group reports''' |Ask groups working together to provide a (short) group report (or provide a template with points to work on) |SOLO taxonomy, content criteria |- |'''Group discussions''' |Monitor the group discussions to evaluate the level of understanding and content discussed (scaffold as appropriate) |SOLO taxonomy, content criteria |- |'''Retention check''' |After a certain time (few weeks/months) ask learners to provide a short text (analysis of a case, short essay on an ethics topic/question). Compare the levels of understanding to another piece collected during or right after the training. |SOLO taxonomy, content criteria |- |'''Vignettes''' |This can be used for measuring ethical sensitivity in (non-)training context. |statistics, EASM (based on the SOLO taxonomy), content criteria |- |'''National surveys''' |Can be used for analysing training-related content in reports and monitoring the display of REI leadership. |statistics, REI leadership framework |} For instance, to measure participants’ reactions during or right after the training, Self-Reflection Form can be used. In addition, if learners worked in groups so their group discussions can be monitored, and if they provided a group-report or pre- and post-texts, the learning process can be evaluated based on the SOLO taxonomy to measure the levels of understanding. Moreover, if possible, a couple of months after the training an additional case study could be given to the same learners, and the content of their analysis could again be evaluated with the SOLO taxonomy. Analysing vignettes and participating in national REI surveys would be relevant for this target group. This kind of effectiveness measure would give a possibility to triangulate the measurement in different time points. An example for implementation can be found here: [https://helsinkifi.sharepoint.com/:p:/r/sites/BEYONDHelsinkiteam/Shared%20Documents/ENERI%20CR%20material%20example.pptx?d=w41ec1fe12bbb495781dbe81d55a09d71&csf=1&web=1&e=yxKgSC ENERI CR material example.pptx] '"`UNIQ--nowiki-0000001E-QINU`"' The Self-Reflection Form link enables the facilitator to make a copy of the form, which they can then edit, and the data will accumulate on the facilitator’s cloud service (Google or Microsoft). '"`UNIQ--nowiki-0000001F-QINU`"' Analysis instruments are described in WP4.2, later available at the Embassy’s website.
Exploring Specific Research Ethics and Integrity Topics
Exploring Training Materials on Open Science
Exploring Training Materials on the Responsible Use of AI
Introducing Research Ethics and Integrity
Introduction to the BEYOND Trainer Guide
Learning about Research Ethics and Integrity Games
Learning about Supervision and Mentorship
Teaching methods for strengthening research ethics and integrity
Training Initiatives for Research Ethics and Integrity
Using Learning Cards to teach Research Ethics and Integrity