The Embassy of Good Science - User contributions [en]

Instruction:4afd5ea7-0308-4e80-9dfc-6b66daf2ac81

2025-01-31T12:41:23Z

0000-0001-7581-7849:

{{Instruction
|Title=Using Different Learning Taxonomies
|Has Related Initiative=Initiative:667449ef-1c0a-4e06-8374-830c8ad68cb8
|Instruction Goal=This module provides guidance on how to use the '''Bloom''', '''SOLO''' and '''Significant Learning (Fink's)''' taxonomies for the creation of effective and impactful learning objectives.
|Has Duration=2
|Important For=Academic staff; Prospective Research Integrity Trainers; Research integrity trainers; Senior researchers; Teachers
}}
{{Custom TabContent Trainee Open}}
{{Instruction Steps Foldout Trainee}}
{{Instruction Perspective Trainee}}
{{Instruction Remarks Trainee}}
{{Custom TabContent Close Trainee}}
{{Custom TabContent Trainer Open}}
{{Instruction Steps Foldout Trainer}}
{{Instruction Perspective Trainer
|Is About=Defining learning objectives serves as a cornerstone for creating successful and impactful learning environments. Learning objectives, i.e. what the learners should learn by taking part in the training, guides the choice of the learning process, the content, activities and assessment. Learning objectives can be defined by using different taxonomies. [https://poorvucenter.yale.edu/BloomsTaxonomy Bloom's taxonomy] categorises educational goals into a hierarchical model, from simple recall of facts to complex evaluation and creation tasks. The [https://www.johnbiggs.com.au/academic/solo-taxonomy/ SOLO] (Structure of Observed Learning Outcomes) taxonomy, on the other hand, describes levels of increasing complexity in a learner's understanding of subjects, ranging from unistructural to extended abstract levels.
|Important Because=Learning is what the learner does, but it can be facilitated through what trainers do and through appropriate teaching activities.<ref>Biggs, J. (1999). What the Student Does: teaching for enhanced learning. Higher Education Research & Development/Higher Education Research and Development, 18(1), 57–75. <nowiki>https://doi.org/10.1080/0729436990180105</nowiki></ref> The Taxonomy of Significant Learning (sometimes also referred as the [https://www.buffalo.edu/catt/teach/develop/design/learning-outcomes/finks.html Fink’s taxonomy]) is not hierarchical in the same way as the other two, however, it builds on Blooms’ taxonomy by including a long-forgotten affective component into the discussion (namely caring).<ref name=":0">Fink, L. D. (2013). Creating significant learning experiences: an integrated approach to designing college courses (Revised and updated edition). San Francisco: Jossey-Bass.</ref> It encourages to include into the learning outcomes the objectives foundational knowledge, application, integration, a human dimension, caring, and learning to acquire competencies, thus providing a holistic approach to learning.<ref name=":0" /> However, since the existing material aligns with Bloom's and SOLO frameworks, this module will primarily describe these two to ensure coherence and consistency in training delivery. Nevertheless, we encourage trainers to also consider the more effective type of learning objectives proposed in the Taxonomy of Significant Learning.
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Use the Bloom’s taxonomy
|Instruction Step Text=Bloom's Taxonomy is a well-known educational framework that offers a methodical way to classify learning objectives according to cognitive difficulty. (e.g., Adams, 2015).<ref>Adams, N. (2015). Bloom’s taxonomy of cognitive learning objectives. Journal of the Medical Library Association, 103(3), 152–153. <nowiki>https://doi.org/10.3163/1536-5050.103.3.010</nowiki> </ref> It is a hierarchical framework that uses cognitive complexity to classify learning objectives. Benjamin Bloom created it in the 1950s, and it is now a vital instrument in educational theory and practice. The taxonomy is divided into six stages: remembering, understanding, applying, analysing, evaluating, and creating. The levels are arranged from lower to higher order cognitive skills. Fundamentally, remembering entails recollecting words, information, and fundamental ideas. Understanding is more than just remembering concepts; it also involves understanding meanings. Applying necessitates applying knowledge to novel contexts or problem-solving. Analysing means dissecting data into its constituent elements and identifying connections between them. Making decisions based on standards and criteria is the process of evaluating. Creating, in the end, involves coming up with original concepts and/or interpretations. The goal of applying Bloom's Taxonomy to training aims and results is to enhance comprehension by considering the knowledge, skills, and competencies that the specific training programmes were created to impart. The Remembering, Understanding, Applying, Analysing, Evaluating, and Creating domains of Bloom's Taxonomy each reflect a different cognitive process and the depth and complexity of learning.

[[File:BloomsTaxonomy.jpg|alt=|center|frame|Fig 21. Bloom’s Taxonomy (taken from the Centre for teaching, Vanderbilt University. <nowiki>https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/</nowiki>) ]]

All taxonomic levels are relevant irrespective of the study or career level. However, the taxonomic levels may mean different things for different individuals. For example, application of knowledge may mean engaging with research designs, but senior researchers often use more complex designs than students still learning how to do research. Nevertheless, it is essential that the learning extends beyond remembering and understanding, and that the complexity of activities at all levels gradually grow as the individual gains experience, knowledge and confidence.

''Remembering and understanding:''

Here, the focus is on memorising key facts, concepts and theories relevant to the field of research and innovation. Understanding these foundational elements is critical to moving forward. For example, undergraduate students need to master the basic principles and terminology related to ethics and integrity to effectively navigate through more complex topics later. Similarly, individuals pursuing a PhD or who are new to academia need a solid understanding of basic concepts before they can conduct more in-depth analyses and applications, such as mastering the ethics of their own PhD research. Moreover, senior researchers may need to understand the basic concept of supervision and mentoring practices when it comes to supervising a team and PhD candidates.

''Apply and analyse:''

Learning should always be an active endeavour irrespective of career or studies applying and analysing knowledge. This is where the emphasis shifts to practical application and critical thinking. Early career researchers, junior professors and academics need competencies for applying the ethics and integrity concepts they have learnt to real-life scenarios in connection to conducting experiments, collecting data and critically analysing the results to gain meaningful insights. Through these activities, participants develop the skills necessary to contribute to the advancement of their field and address research questions with greater depth and sophistication. In terms of research ethics and integrity, this involves applying such knowledge and values to every step of the research.

''Evaluate and create:''

The highest level in Bloom’s Taxonomy involves evaluating existing knowledge and creating new knowledge. All researchers play a critical role in shaping the direction of research and innovation. They are responsible for assessing the validity and significance of research findings and identifying areas for further investigation and innovation. By synthesising existing knowledge and developing new ideas, theories or methods, researchers develop their field forward and inspire the next generation of researchers and innovators. All RE/RI training should include components, which encourage learners to extend their thinking to evaluation and creation. In practice, this involves having such a robust knowledge base and values so that even when encountering new ethical dilemmas or being posed with a novel potentially integrity-threatening situation, they can rely on having the ‘tools’ to handle the situation.
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Use the SOLO taxonomy
|Instruction Step Text=Besides using Bloom’s Taxonomy to define learning objectives, the [https://www.johnbiggs.com.au/academic/solo-taxonomy/ SOLO Taxonomy] can be used .<ref name=":0">Biggs, J. (1999). What the Student Does: teaching for enhanced learning. Higher Education Research & Development/Higher Education Research and Development, 18(1), 57–75. <nowiki>https://doi.org/10.1080/0729436990180105</nowiki></ref><ref name=":1">Biggs, J., & Tang, C. (2007). Using Constructive Alignment in Outcomes-Based Teaching and Learning Teaching for Quality Learning at University (3rd ed., pp. 50-63). Maidenhead: Open University Press. </ref> The Structure of the Observed Learning Outcome, or SOLO, is a way to set the learning outcomes according to how complicated they are.<ref name=":0" /><ref name=":1" /> In this way, the students' work can be assessed according to its quality and not according to how many parts they have understood correctly: initially, we learn one or a few aspects of the task (unistructural), then multiple aspects that are unrelated to each other (multistructural), then we learn how to integrate them into a whole (relational), and lastly, we can generalise that whole to still-untaught applications (extended abstract).

[[File:SOLOtaxonomy.png|alt=|center|frame|Fig 32. SOLO taxonomy (taken from Tammeleht & Löfström, 2023)]]

The SOLO taxonomy has been used to evaluate effectiveness of trainings as well as the development of ethical sensitivity and has been proven to be effective in the context of research ethics and integrity training.<ref>Löfström, E. (2012). Students’ ethical awareness and conceptions of Research Ethics. Ethics & Behavior, 22(5), 349–361. <nowiki>https://doi.org/10.1080/10508422.2012.679136</nowiki> </ref><ref>Tammeleht, A., & Löfström, E. (2023). Learners’ self-assessment as a measure to evaluate the effectiveness of research ethics and integrity training: can we rely on self-reports? Ethics & Behavior, 1–22. <nowiki>https://doi.org/10.1080/10508422.2023.2266073</nowiki> </ref><ref>Tammeleht, A., Löfström, E. & Rodríguez-Triana, M. J. (2022). Facilitating development of research ethics and integrity leadership competencies. International Journal of Educational Integrity, 18, 11, 1-23. <nowiki>https://doi.org/10.1007/s40979-022-00102-3</nowiki> </ref><ref>Tammeleht, A., Löfström, E., & Rajando, K. (Accepted 2024) Effectiveness of research ethics and integrity competence development – what do learning diaries tell us about learning? International Journal of Ethics Education. <div></div></ref> The verbs emphasised in the descriptions below can be used as indicators of the appropriate levels in learning objectives.

''Pre-structural level (0)''

At the pre-structural level the learner fails to identify or approach topics in a meaningful way, but simply repeats the words in the question without understanding them.

''Unistructural level (1)''

At the unistructural level, the learner has sufficient knowledge to identify, recognise, count, find, label, match, name, and perform follow simple procedures. The learner identifies one relevant aspect displaying some familiarity with relevant concepts, but failing to outline multiple dimensions of it but failing to outline multiple dimensions of it. In the context of research ethics and integrity, this may mean identifying certain, perhaps common concepts, but having a limited view of them. For example, the learner may be able to identify some of the things that ought to be mentioned in an information letter to research participants but fails to understand all aspects of ensuring voluntary participation in research.

''Multi-structural level (2)''

At the multistructural level, the learner can enumerate, describe, illustrate, list, sequence, select, combine, and follow procedures, but struggles to make connections between concepts or draw conclusions based on interrelations. For example, the learner may understand that informed consent is necessary in research but fails to understand that this is so because of the need to respect people’s autonomy and right to make decisions that concern themselves.

''Relational level (3)''

At the relational level, the learner displays an ability to address the most relevant aspects of the concept and provide explanations pointing out interrelations and providing examples demonstrating their own reasoning. Corresponding action verbs include; analyse, apply, argue, compare, contrast, critique, explain causes, relate and justify. For example, the learner understands at least the main mechanisms and connections between FFP and the detrimental effects to science.

''Extended abstract level (4)''

At the extended abstract level, the coherent whole is generalised or re-conceptualised at a higher level of abstraction. The learner grasps a more abstract version of the concept, and recognises other domains to which the concept might be applied by displaying the ability to theorise, generate, generalise, hypothesise, create or reflect, formulate and reflect. For example, the learner is able to use knowledge about ethical analysis and ethical principles to solve a novel integrity-related dilemma, which the learner recognises is affecting a research group, but to which the learner has not been exposed before.
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Use The Taxonomy of Significant Learning
|Instruction Step Text=The taxonomy of Significant Learning or the Fink’s taxonomy is a non-hierarchical system that helps trainers devise learning outcomes to support deep learning. No dimension is considered more important than the other and within the course various aspects should be present.<ref name=":0">Fink, L. D. (2013). Creating significant learning experiences: an integrated approach to designing college courses (Revised and updated edition). San Francisco: Jossey-Bass.</ref> Thus, this taxonomy provides an alternative frame for devising learning objectives for training. The fact that this taxonomy emphasises care and empathy makes it very suitable for training on ethics and integrity. The following is a short overview of the various dimensions:

''Foundational knowledge''

This dimension focuses on content knowledge and includes recalling and understanding of information and ideas.

''Application''

Here the learner demonstrates skills – they can be related to the use of knowledge or include skills necessary to interact in the subject, e.g. critical and creative thinking, decision-making, solving problems etc. For example, using the steps of ethical analysis to solve a situation involving an integrity-related challenge.

''Integration''

In this dimension the learner perceives connections between various ideas, disciplines, and experiences. It includes relating various ideas to each other, comparing, contrasting ideas and examples, and so on. For example, in solving an ethical issue, different ethical theoretical viewpoints may lead to diverse actions and solutions. Recognising how for example a virtue ethical approach may lead to a different solution than reasoning based on utilitarianism may be an expression of integration.

''Human dimension''

Learners learn with others, and they gain new understanding of themselves as well as others and alsoin the learning process. They recognise how people influence each other. Understanding how to respectfully work together for the greater good is an example of how the human dimension materialises positively in practice.

''Caring''

The caring dimension includes an affective stance and involves change in a learner. The learners start to see the reason to care about a topic, they gain new interests, feelings and values about the subject. Empathy and an ethics of care are values compatible with caring.

''Learning to learn''

In this dimension the learner understands that it is not only the outcome of learning that matters but also the process is important. This dimension includes guiding one’s learning for instance by inquiry, reflection and self-assessment. The role of reflection has been emphasised as a key activity in learning and individual development.<ref name=":0" />
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Apply and Compare the Taxonomies
|Instruction Step Text=To illustrate how the taxonomies can be applied in creating learning objectives for training, we provide an example of an analysis of learning outcomes in RID-SSISS training material based on the three taxonomies (Table 1).
{{{!}} class="wikitable"
{{!}}+
Table 1: Example of an analysis of learning outcomes in RID-SSISS training material based on three taxonomies
!Level of training material
!Learning outcomes
!Bloom
!SOLO
!Fink
{{!}}-
{{!}} rowspan="3"{{!}}Foundational level
{{!}}Raise awareness of ethical issues during the research process;
{{!}}remember, understand
{{!}}unistructural, multistructural, relational
{{!}}foundational knowledge
{{!}}-
{{!}}Practice utilising the codes of conduct, being familiar with central topics;
{{!}}understand, apply, analyse
{{!}}relational, extended abstract
{{!}}application, integration
{{!}}-
{{!}}Learn together as a team, collaborate with peers.
{{!}}evaluate
{{!}}relational
{{!}}human dimension, learning to learn, caring
{{!}}-
{{!}} rowspan="3"{{!}}Advanced level
{{!}}Develop one’s research ethics competencies by combining previous knowledge and implementing new tools;
{{!}}understand, apply
{{!}}unistructural, multistructural, relational
{{!}}foundational knowledge, application, integration, learning to learn
{{!}}-
{{!}}Identify ethical issues by determining which ethical principle might be at stake.<ref name=":0">Kitchener, K. S. (1985). Ethical principles and ethical decisions in student affairs. ''New Directions for Student Services, 30,'' 17–29. [https://psycnet.apa.org/doi/10.1002/ss.37119853004 https://doi.org/10.1002/ss.37119853004]</ref>
{{!}}analyse, evaluate
{{!}}unistructural, multistructural, relational
{{!}}application, integration
{{!}}-
{{!}}Utilise the ethical analysis steps to provide solutions to ethical dilemmas (in groups).<ref name=":1">Mustajoki, A. S., & Mustajoki, H. (2017). ''A new approach to research ethics: Using guided dialogue to strengthen research communities''. Routledge. <nowiki>https://doi.org/10.4324/9781315545318</nowiki></ref>
{{!}}apply, analyse, evaluate, create
{{!}}unistructural, multistructural, relational, extended abstract
{{!}}foundational knowledge, application, integration,
human dimension,

caring, learning to learn
{{!}}-
{{!}} rowspan="5"{{!}}Leadership level
{{!}}Develop research ethics competencies by combining previous knowledge and implementing new tools;
{{!}}understand, apply
{{!}}unistructural, multistructural, relational
{{!}}foundational knowledge, application, integration, learning to learn
{{!}}-
{{!}}Identify which ethical principles might be at stake in a case.<ref name=":0" />
{{!}}analyse, evaluate
{{!}}unistructural, multistructural, relational
{{!}}application, integration
{{!}}-
{{!}}Utilise the ethical analysis steps to provide solutions to ethical dilemmas.<ref name=":1" />
{{!}}apply, analyse, evaluate, create
{{!}}unistructural, multistructural, relational, extended abstract
{{!}}foundational knowledge, application, integration,
human dimension,

caring, learning to learn
{{!}}-
{{!}}Implement different ethical approaches to the possible courses of action;
{{!}}analyse, evaluate
{{!}}unistructural, multistructural, relational
{{!}}foundational knowledge, application, integration,
{{!}}-
{{!}}Take the role of a REI leader and display (some) REI leadership competencies during their group work.
{{!}}apply, analyse, evaluate, create
{{!}}relational, extended abstract
{{!}}human dimension, caring, learning to learn
{{!}}}
Learning taxonomies are helpful not only in the design of learning goals for teaching, but also in devising appropriate targets of assessment. The coordination between learning goals or objectives, the methods used for teaching and the activities chosen to support learning, as well as the assessment of the learning are referred to as constructive alignment.<ref>Biggs, J. (1996). Enhancing teaching through constructive alignment. ''Higher Education'', 32, 347–364. <nowiki>https://doi.org/10.1007/bf00138871</nowiki></ref><ref>Biggs, J. (1999). What the Student Does: teaching for enhanced learning. Higher Education Research & Development/Higher Education Research and Development, 18(1), 57–75. <nowiki>https://doi.org/10.1080/0729436990180105</nowiki> </ref><ref>Biggs, J., & Tang, C. (2007). Using Constructive Alignment in Outcomes-Based Teaching and Learning Teaching for Quality Learning at University (3rd ed., pp. 50-63). Maidenhead: Open University Press. </ref> This means that the core components of a teaching should be geared towards supporting the same aim; namely learning. Learning taxonomies are helpful not only in the design of learning goals for teaching, but also in devising appropriate targets of assessment.
<references />
}}
{{Instruction Remarks Trainer
|Has Remarks=This project has received funding from the European Union's Horizon Europe research and innovation programme under GA No 101094714 (University of Oslo). UK participants in BEYOND are supported by UKRI grant number 10062742 (Trilateral Research) and by UKRI grant number 10067440 (Heriot-Watt University). Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union, Research Executive Agency, or UKRI. Neither the European Union nor the granting authority nor UKRI can be held responsible for them.)
}}
{{Custom TabContent Close Trainer}}
{{Related To
|Related To Instruction=Instruction:02a025d1-fb65-40b4-914f-5efed84ee112; Instruction:238dfd77-9a59-4dec-b498-3e2214ddc1fe; Instruction:55649946-f15f-4c94-89e1-b79dff231c0e; Instruction:30f33e85-09ed-4ea7-9ac2-7fb26d27ae55; Instruction:1c5bdda7-6444-497b-aaf2-53852a0fd8ee; Instruction:126b8806-0cc1-4625-b11b-f60f43ad9f6f; Instruction:Bcec25e1-c0d9-4e1d-b5c3-eb9c13ae3618; Instruction:204c2375-0867-4302-845f-cdc99e3d38bc; Instruction:Eeee007a-14ec-4ffa-b62e-514aafe4bd1a; Instruction:44eb9553-4c0a-40c9-9ed1-50b4f591e3fb; Instruction:4afd5ea7-0308-4e80-9dfc-6b66daf2ac81; Instruction:Ee61dfb7-737f-40ff-9568-aa669cd6865c
}}
{{Tags}}

Instruction:44eb9553-4c0a-40c9-9ed1-50b4f591e3fb

2025-01-31T12:33:16Z

0000-0001-7581-7849:

{{Instruction
|Title=Training Initiatives for Research Ethics and Integrity
|Has Related Initiative=Initiative:667449ef-1c0a-4e06-8374-830c8ad68cb8
|Instruction Goal=This module introduces different '''trainings''' on '''research ethics''' and '''research integrity''' developed by EU funded initiatives for a range of different target groups and topics.
|Has Duration=1
|Important For=Bachelor students; Citizen Scientists; Doctoral students; Early career researchers; Graduate students; Master students; Prospective Research Integrity Trainers; Research integrity trainers; Senior researchers; Undergraduate students
}}
{{Custom TabContent Trainee Open}}
{{Instruction Steps Foldout Trainee}}
{{Instruction Perspective Trainee}}
{{Instruction Remarks Trainee}}
{{Custom TabContent Close Trainee}}
{{Custom TabContent Trainer Open}}
{{Instruction Steps Foldout Trainer}}
{{Instruction Perspective Trainer
|Is About=This module introduces a collection of training developed by EU funded initiatives. The content of each of these trainings is presented in the last section of the BEYOND Trainer Guide divided by topics and target audience.
}}
{{Instruction Step Trainer
|Instruction Step Title=PRINTEGER
|Instruction Step Text=[https://printeger.eu/ PRINTEGER] project, aims to strengthen research integrity by fostering a culture where integrity is integral to excellent research, beyond just external regulation. The project seeks to improve integrity governance by focusing on researchers’ everyday practices within a complex research environment
}}
{{Instruction Step Trainer
|Instruction Step Title=PATH2INTEGRITY
|Instruction Step Text=The [https://www.path2integrity.eu/ Path2Integrity] project introduces educators to innovative teaching methods that cover topics in research integrity and ethics. The project provides introductory videos and information on the teaching methodology used, discussing research integrity and its significance.
}}
{{Instruction Step Trainer
|Instruction Step Title=INTEGRITY
|Instruction Step Text=The goal of [https://h2020integrity.eu/ INTEGRITY] is to empower students and early career researchers. Rather than focusing on compliance, the project's approach is to develop the capacity of participants to identify, consider, and address integrity concerns in research procedures.
}}
{{Instruction Step Trainer
|Instruction Step Title=ENERI Classroom
|Instruction Step Text=[https://eneri.eu/eneri-classroom/ ENERI Classroom] contains training material exploring research ethics and integrity issues focusing on academics, and research ethics and integrity experts. The research integrity advisory boards, committees handling allegations or working with research integrity policy development, research integrity officers and advisors, research integrity ombudspersons (RIOs), research ethics committees including their members and their secretariats (RECs), and experts and officers in EU-bodies are the main target groups of the training materials and curricula tool.
}}
{{Instruction Step Trainer
|Instruction Step Title=VIRT2UE
|Instruction Step Text=[[Guide:Bbe860a3-56a9-45f7-b787-031689729e52|The VIRT2UE Train the Trainer program]] is designed for researchers and educators across various disciplines who wish to become Research Integrity trainers. It adopts a virtue-based approach, encouraging participants to reflect on their own perspectives and understanding of research integrity. The program emphasises personal case reflection and practical experiences, aiming to create a strong link between theoretical knowledge and real-world application.
}}
{{Instruction Step Trainer
|Instruction Step Title=RID-SSISS
|Instruction Step Text=[https://www.helsinki.fi/en/researchgroups/diversity-multilingualism-and-social-justice-in-education/projects-1/ended-projects/researcher-identity-development-strengthening-science-in-society-strategies-rid-ssiss The RID-SSISS] training aims to help beginner and more experienced researchers develop their research ethics competencies in HE institutions. A CSCL (Computer-Supported Collaborative Learning) ethics resource was designed that utilised cases, collaboration, and structural scaffolding (see Table 1 for an overview).
}}
{{Instruction Step Trainer
|Instruction Step Title=EnTIRE
|Instruction Step Text=EnTIRE is the project behind [[Main Page|The Embassy of Good Science]], a Wiki based community driven platform on research ethics and integrity which provides resources such as guidelines, cases and training material for and by researchers and research stakeholders who want to support good science.
}}
{{Instruction Step Trainer
|Instruction Step Title=BRIDGE
|Instruction Step Text=The Bridging Integrity in Higher Education, Business, and Society ([https://www.academicintegrity.eu/wp/bridge/ BRIDGE]) project aimed at connecting integrity practices across academia, research, business, and society. BRIDGE targets early-career researchers (master’s and PhD students) and their supervisors, with the first step focused on analysing integrity practices across these fields. The project will develop checklists, open educational resources (including gamified tools), and training to bridge gaps between academic and research integrity.
}}
{{Instruction Step Trainer
|Instruction Step Title=ROSiE
|Instruction Step Text=The [https://rosie-project.eu/ ROSiE] training materials are aimed at the following groups of trainees: (i) students, (ii) early career researchers, (iii) experienced researchers, and (iv) citizen scientists. The ROSiE consortium developed a didactic framework which is learner-centred; based on interests, backgrounds, and learning styles of trainees; and ensures active and collaborative involvement in the learning process.
}}
{{Instruction Remarks Trainer
|Has Remarks=This project has received funding from the European Union's Horizon Europe research and innovation programme under GA No 101094714 (University of Oslo). UK participants in BEYOND are supported by UKRI grant number 10062742 (Trilateral Research) and by UKRI grant number 10067440 (Heriot-Watt University). Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union, Research Executive Agency, or UKRI. Neither the European Union nor the granting authority nor UKRI can be held responsible for them.)
}}
{{Custom TabContent Close Trainer}}
{{Related To
|Related To Resource=Resource:1d26fd13-1ced-44bc-8d19-e094b37f8f70; Resource:45a04c31-5a75-4816-8484-2dd9b71d1674; Resource:Aef6b98d-9cc5-4db0-bffd-4a3daa99a3f3; Resource:H5P-26; Resource:H5P-39; Resource:H5P-41; Resource:H5P-28; Resource:H5P-29; Resource:H5P-38; Resource:H5P-40; Resource:C99f17ec-3d1e-4f7a-bfc7-3e3607934ead; Resource:F6100097-fddb-4c77-9098-1bc767c34a6a; Resource:7f7810d8-74a2-42ac-906c-7f6a73fcd183; Resource:E99e20d0-8116-4d77-84ec-7df396703bf4; Resource:67caae86-68db-49ea-8305-2010fe701aa6
|Related To Instruction=Instruction:02a025d1-fb65-40b4-914f-5efed84ee112; Instruction:238dfd77-9a59-4dec-b498-3e2214ddc1fe; Instruction:55649946-f15f-4c94-89e1-b79dff231c0e; Instruction:30f33e85-09ed-4ea7-9ac2-7fb26d27ae55; Instruction:1c5bdda7-6444-497b-aaf2-53852a0fd8ee; Instruction:126b8806-0cc1-4625-b11b-f60f43ad9f6f; Instruction:Bcec25e1-c0d9-4e1d-b5c3-eb9c13ae3618; Instruction:204c2375-0867-4302-845f-cdc99e3d38bc; Instruction:Eeee007a-14ec-4ffa-b62e-514aafe4bd1a; Instruction:44eb9553-4c0a-40c9-9ed1-50b4f591e3fb; Instruction:4afd5ea7-0308-4e80-9dfc-6b66daf2ac81
}}
{{Tags}}

Instruction:Eeee007a-14ec-4ffa-b62e-514aafe4bd1a

2025-01-31T12:17:03Z

0000-0001-7581-7849:

{{Instruction
|Title=Teaching methods for strengthening research ethics and integrity
|Has Related Initiative=Initiative:667449ef-1c0a-4e06-8374-830c8ad68cb8
|Instruction Goal=This module provides a state-of-the art overview of '''methods''' and '''approaches''' to teaching, which according to extant research, are appropriate to support learning in the context of research ethics and integrity.
|Has Duration=1
|Important For=Academic staff; Prospective Research Integrity Trainers; Research integrity trainers; Senior researchers; Teachers
}}
{{Custom TabContent Trainee Open}}
{{Instruction Steps Foldout Trainee}}
{{Instruction Perspective Trainee}}
{{Instruction Remarks Trainee}}
{{Custom TabContent Close Trainee}}
{{Custom TabContent Trainer Open}}
{{Instruction Steps Foldout Trainer}}
{{Instruction Perspective Trainer
|Is About=Effective RE/RI training benefits from active learning, reflective practices, experiential learning, and ongoing feedback.<ref>Löfström, E., & Tammeleht, A. (2023). A pedagogy for teaching research ethics and integrity in the social sciences: Case-based and collaborative learning. In G. J. Curtis (Ed.), ''Academic Integrity in the Social Sciences: Perspectives on Pedagogy and Practice'' (Vol. 6, pp. 127-145). (Ethics and Integrity in Educational Contexts; Vol. 6). Springer. <nowiki>https://doi.org/10.1007/978-3-031-43292-7_9</nowiki><div></div></ref><ref>Bagdasarov, Z., Harkrider, L. N., Johnson, J. F., MacDougal A. E., Devenport, L. D., Connelly, S., Mumford, M. D., Peacock, J., & Thiel, C. E. (2012). An investigation of case-based instructional strategies on learning, retention, and ethical decision-making. Journal of Empirical Research on Human Research Ethics, 7(4), 79–86. <nowiki>https://doi.org/10.1525/jer.2012.7.4.79</nowiki></ref> <ref>Johnson, J. F., Bagdasarov, Z., Connelly, S., Harkrider, L., Devenport, L. D., Mumford, M. D., & Thiel, C. E. (2012). Case-based ethics education: the impact of cause complexity and outcome favorability on ethicality. Journal of Empirical Research on Human Research Ethics, 7(3), 63– 77. <nowiki>https://doi.org/10.1525/jer.2012.7.3.63</nowiki></ref><ref>Quintana, Chris & Reiser, Brian & Davis, Elizabeth & Krajcik, Joseph & Fretz, Eric & Duncan, Ravit & Kyza, Eleni & Edelson, Daniel & Soloway, Elliot. (2004). A Scaffolding Design Framework for Software to Support Science Inquiry. Journal of the Learning Sciences. 13. 337-386. 10.1207/s15327809jls1303_4. </ref><ref>Reiser, B. J. (2004). Scaffolding Complex Learning: The Mechanisms of Structuring and Problematizing Student Work. ''Journal of the Learning Sciences, 13''(3), 273–304. [https://psycnet.apa.org/doi/10.1207/s15327809jls1303_2 https://doi.org/10.1207/s15327809jls1303_2]</ref><ref>Tammeleht, Anu & Rodríguez-Triana, María & Koort, Kairi & Löfström, Erika. (2021). Scaffolding Collaborative Case-Based Learning during Research Ethics Training. Journal of Academic Ethics. 19. 10.1007/s10805-020-09378-x. </ref> This module provides an overview of methods and approaches to teaching, which according to extant research, are the best ways to support learning in the context of research ethics and integrity. These methods and approaches are compatible with the training materials and resources produced within the selected EU-funded projects and presented in the BEYOND trainer guide. Indeed, many of the activities described in the project materials draw on a case-based approach, scaffolding and collaborative learning. To underpin the use of these methods and approaches, this guide provides an overview of why and how they support RE/RI learning, so that trainers may make their teaching choice based on research evidence.
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Learn about the case based approach
|Instruction Step Text=Across disciplines, case-based learning (CBL) is a well-established method that encourages higher levels of cognition by having students apply their knowledge to real-world or fictional situations (see Bloom's Taxonomy or Relational/extended abstract levels of learning in SOLO taxonomy).

Learners usually '''work in groups''' on case studies, which are narratives with one or more characters and/or scenarios. The cases pose a disciplinary issue or issues, to which learners come up with remedies while working with an instructor.<ref>Case-Based learning. (2024, January 25). Poorvu Center for Teaching and Learning. <nowiki>https://poorvucenter.yale.edu/strategic-resources-digital-publications/strategies-teaching/case-based-learning</nowiki> </ref><ref>Löfström, E. & Tammeleht, A. (2023). A pedagogy for teaching research ethics and integrity in the social sciences: Case-based and collaborative learning. In G. Curtis (Ed.) Academic Integrity in the Social Sciences. Perspectives on Pedagogy and Practice (127-145). Springer. https://doi.org/10.1007/978-3-031-43292-7 </ref><ref>Tammeleht, A., Rodríguez‐Triana, M. J., Koort, K., & Löfström, E. (2019). Collaborative case-based learning process in research ethics. International Journal for Educational Integrity, 15(1). <nowiki>https://doi.org/10.1007/s40979-019-0043-3</nowiki> </ref> Case studies are an effective teaching tool that engages learners, stimulates critical thinking, and enables a deeper understanding of real-life situations. The use of case studies is a deliberate process designed to promote active engagement, critical thinking, and deeper understanding among students. Prior research (e.g., Löfström & Tammeleht, 2023 citing: Bagdasarov et al., 2012; Johnson et al., 2012; McWilliams & Nahavandi, 2006; Nonis & Swift, 2001; O'Leary & Cotter, 2000) has identified the use of cases to be beneficial in RE/RI teaching/learning.<ref name=":0">Löfström, E., & Tammeleht, A. (2023). A pedagogy for teaching research ethics and integrity in the social sciences: Case-based and collaborative learning. In G. J. Curtis (Ed.), ''Academic Integrity in the Social Sciences: Perspectives on Pedagogy and Practice'' (Vol. 6, pp. 127-145). (Ethics and Integrity in Educational Contexts; Vol. 6). Springer. <nowiki>https://doi.org/10.1007/978-3-031-43292-7_9</nowiki><div></div></ref> Understanding why and how learning occurs is essential for improving teaching, and as a result, understanding how learners learn can be accessed through an awareness of learning within the framework of research ethics and integrity.<ref name=":0" />

'''It begins with the careful selection''' of relevant case studies that align with the learning objectives of the course. These cases should not only be current and authentic but should also reflect the students' interests and experiences and provide them with a tangible connection to the course material. RE/RI case-studies can be easily found on [https://embassy.science/wiki-wiki/index.php/Main_Page the Embassy of Good Science].

'''Once a suitable case study has been selected''', the teaching process usually begins with an introduction to the case (e.g., providing context and background information. This first step is crucial to ensuring that students understand the importance of the case study and its relevance to the wider course material. This practice will help students to get acquainted with the topic. In addition to case-studies, also vignettes have been used in RE/RI education to reflect on real-life situations including an explicit or implicit conflict. Trainers may identify a specific ethical/integrity issue on which learners are asked to reflect on.<ref name=":0" />

'''As learners delve into the case study''', they are asked to actively engage with the material. This means more than just passive reading; learners are encouraged to take notes, ask questions, and identify important themes or patterns contained in the case study. By promoting active reading practises, instructors aim to encourage the development of deeper understanding of the complexity of real-world problems and the various factors at play.

'''The initial reading''' is often followed by common discussion and analysis. With the guidance of the trainer, learners are encouraged to share their interpretations of the case study and explore different perspectives. Discussions can be structured around questions, which encourage critical thinking, consider alternative viewpoints and evaluate the implications of different approaches in order to move from uni- and multistructural levels to relational and extended abstract levels.

Central to the case study approach is the opportunity for learners to '''apply theoretical concepts and principles to real-life situations'''. Instructors help learners make connections between the case study and the course material by encouraging them to analyse the case through the lens of relevant theories, models or frameworks. This process not only deepens students' understanding of theoretical concepts, but also enhances their ability to apply these concepts in practical contexts.

In addition, case studies provide a platform to foster problem-solving skills. Learners are tasked with finding creative solutions to the challenges presented in the case, evaluating the feasibility of various options, and developing a reasoned plan of action. Through this process, learners learn to deal with complex problems, weigh competing interests, and make informed decisions based on facts and analyses.

'''Finally''', case studies can serve as a valuable assessment tool, allowing instructors to evaluate learners' mastery of the content of the course and their ability to apply theoretical concepts to real-world scenarios. Assignments may include written reflections, group presentations, or class discussions based on the case study so that students can demonstrate their learning and receive constructive feedback from fellow students and instructors.

Case-based approaches are utilised in [[Initiative:639c9790-bf80-4f21-9fc0-8027b4c0cfe0|ENERI]], [[Initiative:Fa186292-623f-4b6f-a21e-44250c057f15|RID-SSISS]], [[Initiative:0582c7af-35eb-4def-b74e-c884f29965da|Path2Integrity]], [[Initiative:F9656f91-a514-44ff-9264-d6b3414fdddc|INTEGRITY]] and [[Initiative:8eed30fd-c2ed-44d1-9752-753092bd350e|VIRT2UE]].
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Learn about collaborative learning
|Instruction Step Text=Collaborative learning is a pedagogical approach that emphasises active participation, shared responsibility and mutual support among students. Collaborative learning is based on the idea that the production and internalisation of the knowledge is established by collaboration. Moreover, learning is usually best supported through social negotiation rarther than competition. Furthermore, team learning has been demonstrated to significantly enhance ethical practice. Research indicates that students primarily interpret their socialisation into academia and their field by the ethical standards and practices that they observe.<ref name=":0">Löfström, E., & Tammeleht, A. (2023). A pedagogy for teaching research ethics and integrity in the social sciences: Case-based and collaborative learning. In G. J. Curtis (Ed.), ''Academic Integrity in the Social Sciences: Perspectives on Pedagogy and Practice'' (Vol. 6, pp. 127-145). (Ethics and Integrity in Educational Contexts; Vol. 6). Springer. <nowiki>https://doi.org/10.1007/978-3-031-43292-7_9</nowiki><div></div></ref><ref name=":1">Tammeleht, A., Koort, K., Rodríguez-Triana, M. J. & Löfström, E. (2022). Knowledge building process during collaborative research ethics training for researchers: experiences from one university. International Journal of Ethics Education, 7, 147–170. <nowiki>https://doi.org/10.1007/s40889-021-00138-y</nowiki> </ref><ref name=":2">Tammeleht, A., Rodríguez‐Triana, M. J., Koort, K., & Löfström, E. (2019). Collaborative case-based learning process in research ethics. International Journal for Educational Integrity, 15(1). <nowiki>https://doi.org/10.1007/s40979-019-0043-3</nowiki> </ref>

When teaching research ethics and integrity, collaborative learning can be particularly effective as it can promote deeper understanding, critical thinking and ethical reasoning skills. In collaborative learning environments, students are actively engaged in the learning process rather than passively receiving information. They participate in discussions, debates and hands-on activities that require them to grapple with ethical dilemmas, analyse complex issues and apply ethical principles to real-world scenarios. This active engagement promotes deeper learning and retention of ethical concepts and principles. Collaborative learning encourages students to critically evaluate information, perspectives and arguments related to research ethics and integrity. Through discussions with peers, analysing case studies and reflecting on their own ethical beliefs and values, learners develop the ability to identify ethical issues, consider alternative viewpoints and make informed decisions.<ref name=":0" /><ref name=":1" /><ref name=":2" />

Collaborative learning environments provide opportunities for learners to challenge assumptions, explore ethical complexity and develop reasoned arguments based on evidence and ethical principles. Peer interaction is a central component of collaborative learning that allows learners to learn from each other's experiences, perspectives, and insights. By participating in discussions, debates, and collaborative projects with their peers, learners learn about various viewpoints, cultural perspectives, and disciplinary approaches to research ethics and integrity. Peer interaction also fosters collaboration, communication skills and teamwork, which are essential for addressing ethical challenges in research environments where collaboration and interdisciplinary cooperation are increasingly common.<ref name=":0" /><ref name=":1" /><ref name=":2" />

During collaborating trainings, a variety of teaching methods can be used. Prior research has addressed collaborative learning with the use of case-based approaches, storytelling, flipped classroom, and role play and games (e.g., [https://www.erim.eur.nl/research-integrity/training-and-education/dilemma-game/ Rotterdam dilemma game]) .<ref>Koterwas, A., Dwojak-Matras, A., & Kalinowska, K. (2021). Dialogical teaching of research integrity: an overview of selected methods. Facets, 6, 2138–2154. <nowiki>https://doi.org/10.1139/facets-2021-0045</nowiki> </ref><ref>Löfström, E. (2016). Role-playing institutional academic integrity policy-making: using researched perspectives to develop pedagogy. International Journal for Educational Integrity, 12:5, 1-14. DOI 10.1007/s40979-016-0011-0, <nowiki>http://rdcu.be/mVXE</nowiki>, <nowiki>http://link.springer.com/article/10.1007/s40979-016-0011-0</nowiki> </ref><ref name=":1" /><ref name=":2" />

Collaborative approaches are utilised in [[Initiative:0582c7af-35eb-4def-b74e-c884f29965da|Path2Integrity]], [[Initiative:F9656f91-a514-44ff-9264-d6b3414fdddc|INTEGRITY]] and [[Initiative:8eed30fd-c2ed-44d1-9752-753092bd350e|VIRT2UE]].
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Learn about scaffolding and feedback
|Instruction Step Text=[https://www.structural-learning.com/post/scaffolding-in-education-a-teachers-guide Scaffolding] is a teaching technique, which involves providing tailored support to learners based on their current expertise and gradually withdrawing support as they become more proficient. This approach, researched in the context of research ethics and integrity, is used in both face-to-face and online learning environments, including problem-based and inquiry learning.<ref>Quintana, Chris & Reiser, Brian & Davis, Elizabeth & Krajcik, Joseph & Fretz, Eric & Duncan, Ravit & Kyza, Eleni & Edelson, Daniel & Soloway, Elliot. (2004). A Scaffolding Design Framework for Software to Support Science Inquiry. Journal of the Learning Sciences. 13. 337-386. 10.1207/s15327809jls1303_4.</ref><ref>Reiser, B. J. (2004). Scaffolding complex learning: The mechanisms of structuring and problematizing student work. The Journal of the Learning Sciences, 13, 273–304. </ref><ref>Tammeleht, Anu & Rodríguez-Triana, María & Koort, Kairi & Löfström, Erika. (2021). Scaffolding Collaborative Case-Based Learning during Research Ethics Training. Journal of Academic Ethics. 19. 10.1007/s10805-020-09378-x.</ref> Conceptual scaffolding is crucial in problem-based and inquiry learning, helping learners navigate complex concepts and considering various learning styles. Teachers can adjust academic content to suit learners' abilities. Online courses benefit from personalised learning paths that can be adjusted in real-time. Scaffolding aligns with the idea of Vygotsky's zone of proximal development, that is instruction and facilitation should target the domain where the learner can function with assistance. It is unnecessary to provide scaffolding in domains which the learner already masters, or which are still beyond the learners reach.

When teaching research ethics and integrity, scaffolding entails dissecting difficult ideas into smaller, more digestible chunks and supporting students as they gain knowledge. Scaffolding can be planned into the design of the course teaching/learning activities and the instructions of these, but often opportunities to incorporate scaffolding techniques present themselves ad hoc. In this case, it is important that the trainer is aware of a variety of techniques and recognises situations in which they can be beneficially used to support learning. The steps for incorporating scaffolding include:

#Identifying what the learner already knows, that is, what is their current level and where is the zone of proximal development;
#Setting goals, which reflect the learning objectives, for the learner in line with what was determined to be within reach in the prior step;
#Planning a suitable breakdown of goals and activities in support of the goals;
#Carrying out the training with scaffolding, monitoring of learning and provision of feedback during the learning;
#Adjusting the support and gradually decreasing it as the learner progresses towards the goal;
#New goals and planning activities as the prior goals are reached.

For example the website [https://www.buffalo.edu/catt.html Scaffolding Over Time] prepared by the Office of Curriculum and Teaching Transformation, University of Buffalo provides more information on Scaffolding for the trainer interested in using this technique.

Table 3 presents a scaffolding framework utilised in research ethics training.<ref>Tammeleht, Anu & Rodríguez-Triana, María & Koort, Kairi & Löfström, Erika. (2021). Scaffolding Collaborative Case-Based Learning during Research Ethics Training. Journal of Academic Ethics. 19. 10.1007/s10805-020-09378-x.</ref>

{{{!}} class="wikitable"
{{!}}+
!'''Scaffolding process'''
!'''Scaffolding mechanism'''
!'''Scaffolding technique'''
!'''Scaffolding purpose'''
!'''Illustrative example'''
{{!}}-
{{!}} rowspan="6"{{!}}SENSE-MAKING
{{!}}Problematising
{{!}}Hinting
{{!}}Give an indirect suggestion or piece of evidence that leads toward a problem solution (Merriam-Webster)
{{!}}‘So, the documents pertaining to research ethics and integrity to consult would be…’ (implying that some documents should be consulted)
{{!}}-
{{!}}Problematising
{{!}}Describing the problem to direct focus
{{!}}Orient to the important features (Tambaum, 2017)
{{!}}‘Indeed, when you have to get an informed consent you should consider various aspects, for example …’
{{!}}-
{{!}}Structural/ Problematising
{{!}}Making fill-in-the-blank kinds of requests
{{!}}A statement or a question with a missing component (but some info is given)
{{!}}‘A good way to highlight the importance of research integrity would be to ….?’
{{!}}-
{{!}}Structural/ Problematising
{{!}}Asking a leading question (P – why?)
{{!}}A question that prompts or encourages the answer wanted
{{!}}‘Where can you find the information pertaining to the codes of conduct?’
{{!}}-
{{!}}Structural
{{!}}Providing an example
{{!}}Giving an example to illustrate a point
{{!}}‘For example, in Europe it is a common practice to consult a research integrity advisor.’
{{!}}-
{{!}}Structural
{{!}}Providing physical props
{{!}}Helping to understand by mimicking or showing a visual aid
{{!}}‘What do you think are the ethical aspects of designing this item?’ (show e.g. a fork)
{{!}}-
{{!}} rowspan="6"{{!}}PROCESS
MANAGEMENT
{{!}}Structural
{{!}}Pumping
{{!}}Simulating to go further without specific instructions (Tambaum, 2017)
{{!}}‘OK, what else?’
{{!}}-
{{!}}Structural
{{!}}Redirecting the learner
{{!}}Showing which direction to go, which aspect should be tackled next (especially when seeing that the direction is lost/off)
{{!}}‘All right, let’s get back to the track and discuss the next steps of the ethics review process’
{{!}}-
{{!}}Structural
{{!}}Decomposing the task
{{!}}Making the bigger task into smaller components
{{!}}‘First, think what you know about the ethics review process, then, read the paragraph and finally…’ (give instructions one task after another)
{{!}}-
{{!}}Structural
{{!}}Initiating the reasoning step
{{!}}Use the faculty of reason so as to arrive at conclusions
{{!}}‘First, think what you know about the ethics review process, then, read the paragraph and finally…’ (give instructions one task after another)
{{!}}-
{{!}}Structural
{{!}}Completing the learners’ reasoning
{{!}}‘Splicing in’ the correct answer (Tambaum, 2017)
{{!}}(the learner cannot end the thought) ‘... you mean a code of conduct should be consulted?’
{{!}}-
{{!}}Structural
{{!}}Executing parts of the skill
{{!}}Do parts of the task for the learner to give an example
{{!}}‘OK, so first you consult the ALLEA code of conduct and then …’
{{!}}-
{{!}} rowspan="3"{{!}}ARTICULATION
&

REFLECTION
{{!}}Structural
{{!}}Maintaining goal orientation
{{!}}Reminding the learner of some aspect of the task
{{!}}‘Have you also had time to think about …?’
{{!}}-
{{!}}Structural/ Problematising
{{!}}Highlighting critical features
{{!}}Drawing attention to the most important aspects of the problem; highlighting ‘discrepancies’ (Reiser, 2004)
{{!}}‘Do you remember you mentioned the ethics review, what other purposes might it have?’
{{!}}-
{{!}}Problematising
{{!}}Comparing the current problem with a previously solved one
{{!}}Showing similarities between solutions
{{!}}‘Do you recall the situation that happened to Dr Smith when he invited participants into his survey? This situation may actually have similar implications.’
{{!}}}

Offering feedback and facilitating reflection are critical components when using scaffolding in research ethics and integrity training.

Feedback is more effective:

*when given as soon as possible after the session,
*when it is focused,
*when considered as a process, not a one-time shot,
*when receivers participate in the feedback process freely or when doing so is required by regular professional standards,
*when it has a restricted and chosen number of negative comments mixed in with a decent number of positive and encouraging remarks.<ref name=":0">Brinko, K. T. (1993). The practice of giving feedback to improve teaching: What is effective? Journal of Higher Education/the Journal of Higher Education, 64(5), 574. <nowiki>https://doi.org/10.2307/2959994</nowiki> </ref>
*when negative information is "sandwiched" between positive information,
*when it allows the receiver to respond and interact,
*when given frequently, but not excessively,
*when it creates cognitive [https://www.psychologytoday.com/us/basics/cognitive-dissonance dissonance].<ref>Van Lange, P. a. M., Kruglanski, A. W., & Higgins, E. (2012). Handbook of Theories of Social Psychology. In SAGE Publications Ltd eBooks. <nowiki>https://doi.org/10.4135/9781446249222</nowiki> </ref>

Feedback can be provided by the trainer as well as by peers. Peer feedback is effective:

*when information is gathered from different people,
*when it is believed that the information's source is reliable, informed, and has a good intention,
*when the status or career level of the feedback provider and its’ recipient are the same.<ref name=":0" />

Scaffolding techniques are utilised in [[Initiative:Fa186292-623f-4b6f-a21e-44250c057f15|RID-SSISS]], [[Initiative:76ef100a-e459-4942-bd1f-701f747e8906|ROSiE]], [[Initiative:8eed30fd-c2ed-44d1-9752-753092bd350e|VIRT2UE]], [[Initiative:F9656f91-a514-44ff-9264-d6b3414fdddc|INTEGRITY]], [[Initiative:0582c7af-35eb-4def-b74e-c884f29965da|Path2Integrity]], and in the Case studies section of [https://classroom.eneri.eu/ ENERI Classroom].
<references />
}}
{{Instruction Remarks Trainer
|Has Remarks=This project has received funding from the European Union's Horizon Europe research and innovation programme under GA No 101094714 (University of Oslo). UK participants in BEYOND are supported by UKRI grant number 10062742 (Trilateral Research) and by UKRI grant number 10067440 (Heriot-Watt University). Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union, Research Executive Agency, or UKRI. Neither the European Union nor the granting authority nor UKRI can be held responsible for them.)
}}
{{Custom TabContent Close Trainer}}
{{Related To
|Related To Resource=Resource:1d26fd13-1ced-44bc-8d19-e094b37f8f70; Resource:45a04c31-5a75-4816-8484-2dd9b71d1674; Resource:C386dbba-2f69-4257-89c2-903898cf1f12; Resource:Aef6b98d-9cc5-4db0-bffd-4a3daa99a3f3; Resource:C99f17ec-3d1e-4f7a-bfc7-3e3607934ead; Resource:F6100097-fddb-4c77-9098-1bc767c34a6a; Resource:7f7810d8-74a2-42ac-906c-7f6a73fcd183; Resource:E99e20d0-8116-4d77-84ec-7df396703bf4; Resource:67caae86-68db-49ea-8305-2010fe701aa6
|Related To Instruction=Instruction:02a025d1-fb65-40b4-914f-5efed84ee112; Instruction:238dfd77-9a59-4dec-b498-3e2214ddc1fe; Instruction:55649946-f15f-4c94-89e1-b79dff231c0e; Instruction:30f33e85-09ed-4ea7-9ac2-7fb26d27ae55; Instruction:1c5bdda7-6444-497b-aaf2-53852a0fd8ee; Instruction:126b8806-0cc1-4625-b11b-f60f43ad9f6f; Instruction:Bcec25e1-c0d9-4e1d-b5c3-eb9c13ae3618; Instruction:204c2375-0867-4302-845f-cdc99e3d38bc; Instruction:Eeee007a-14ec-4ffa-b62e-514aafe4bd1a; Instruction:44eb9553-4c0a-40c9-9ed1-50b4f591e3fb; Instruction:4afd5ea7-0308-4e80-9dfc-6b66daf2ac81; Instruction:Ee61dfb7-737f-40ff-9568-aa669cd6865c
}}
{{Tags
|Involves=ENERI; Integrity; Path2Integrity; RID-SSISS; ROSiE; VIRT2UE
}}

Instruction:4afd5ea7-0308-4e80-9dfc-6b66daf2ac81

2025-01-31T11:37:21Z

0000-0001-7581-7849:

{{Instruction
|Title=Using Different Learning Taxonomies
|Has Related Initiative=Initiative:667449ef-1c0a-4e06-8374-830c8ad68cb8
|Instruction Goal=This module provides guidance on how to use the '''Bloom''', '''SOLO''' and '''Significant Learning (Fink's)''' taxonomies for the creation of effective and impactful learning objectives.
|Has Duration=2
|Important For=Academic staff; Prospective Research Integrity Trainers; Research integrity trainers; Senior researchers; Teachers
}}
{{Custom TabContent Trainee Open}}
{{Instruction Steps Foldout Trainee}}
{{Instruction Perspective Trainee}}
{{Instruction Remarks Trainee}}
{{Custom TabContent Close Trainee}}
{{Custom TabContent Trainer Open}}
{{Instruction Steps Foldout Trainer}}
{{Instruction Perspective Trainer
|Is About=Defining learning objectives serves as a cornerstone for creating successful and impactful learning environments. Learning objectives, i.e. what the learners should learn by taking part in the training, guides the choice of the learning process, the content, activities and assessment. Learning objectives can be defined by using different taxonomies. [https://poorvucenter.yale.edu/BloomsTaxonomy Bloom's taxonomy] categorises educational goals into a hierarchical model, from simple recall of facts to complex evaluation and creation tasks. The [https://www.johnbiggs.com.au/academic/solo-taxonomy/ SOLO] (Structure of Observed Learning Outcomes) taxonomy, on the other hand, describes levels of increasing complexity in a learner's understanding of subjects, ranging from unistructural to extended abstract levels.
|Important Because=Learning is what the learner does, but it can be facilitated through what trainers do and through appropriate teaching activities.<ref>Biggs, J. (1999). What the Student Does: teaching for enhanced learning. Higher Education Research & Development/Higher Education Research and Development, 18(1), 57–75. <nowiki>https://doi.org/10.1080/0729436990180105</nowiki></ref> The Taxonomy of Significant Learning (sometimes also referred as the [https://www.buffalo.edu/catt/teach/develop/design/learning-outcomes/finks.html Fink’s taxonomy]) is not hierarchical in the same way as the other two, however, it builds on Blooms’ taxonomy by including a long-forgotten affective component into the discussion (namely caring).<ref name=":0">Fink, L. D. (2013). Creating significant learning experiences: an integrated approach to designing college courses (Revised and updated edition). San Francisco: Jossey-Bass.</ref> It encourages to include into the learning outcomes the objectives foundational knowledge, application, integration, a human dimension, caring, and learning to acquire competencies, thus providing a holistic approach to learning.<ref name=":0" /> However, since the existing material aligns with Bloom's and SOLO frameworks, this module will primarily describe these two to ensure coherence and consistency in training delivery. Nevertheless, we encourage trainers to also consider the more effective type of learning objectives proposed in the Taxonomy of Significant Learning.
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Use the Bloom’s taxonomy
|Instruction Step Text=Bloom's Taxonomy is a well-known educational framework that offers a methodical way to classify learning objectives according to cognitive difficulty. (e.g., Adams, 2015).<ref>Adams, N. (2015). Bloom’s taxonomy of cognitive learning objectives. Journal of the Medical Library Association, 103(3), 152–153. <nowiki>https://doi.org/10.3163/1536-5050.103.3.010</nowiki> </ref> It is a hierarchical framework that uses cognitive complexity to classify learning objectives. Benjamin Bloom created it in the 1950s, and it is now a vital instrument in educational theory and practice. The taxonomy is divided into six stages: remembering, understanding, applying, analysing, evaluating, and creating. The levels are arranged from lower to higher order cognitive skills. Fundamentally, remembering entails recollecting words, information, and fundamental ideas. Understanding is more than just remembering concepts; it also involves understanding meanings. Applying necessitates applying knowledge to novel contexts or problem-solving. Analysing means dissecting data into its constituent elements and identifying connections between them. Making decisions based on standards and criteria is the process of evaluating. Creating, in the end, involves coming up with original concepts and/or interpretations. The goal of applying Bloom's Taxonomy to training aims and results is to enhance comprehension by considering the knowledge, skills, and competencies that the specific training programmes were created to impart. The Remembering, Understanding, Applying, Analysing, Evaluating, and Creating domains of Bloom's Taxonomy each reflect a different cognitive process and the depth and complexity of learning.

[[File:BloomsTaxonomy.jpg|alt=|center|frame|Fig 21. Bloom’s Taxonomy (taken from the Centre for teaching, Vanderbilt University. <nowiki>https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/</nowiki>) ]]

All taxonomic levels are relevant irrespective of the study or career level. However, the taxonomic levels may mean different things for different individuals. For example, application of knowledge may mean engaging with research designs, but senior researchers often use more complex designs than students still learning how to do research. Nevertheless, it is essential that the learning extends beyond remembering and understanding, and that the complexity of activities at all levels gradually grow as the individual gains experience, knowledge and confidence.

''Remembering and understanding:''

Here, the focus is on memorising key facts, concepts and theories relevant to the field of research and innovation. Understanding these foundational elements is critical to moving forward. For example, undergraduate students need to master the basic principles and terminology related to ethics and integrity to effectively navigate through more complex topics later. Similarly, individuals pursuing a PhD or who are new to academia need a solid understanding of basic concepts before they can conduct more in-depth analyses and applications., such as mastering the ethics of their own PhD research. Moreover, senior researchers may need to understand the basic concept of supervision and mentoring practices when it comes to supervising a team and PhD candidates.

''Apply and analyse:''

Learning should always be an active endeavour irrespective of career or studies applying and analysing knowledge. This is where the emphasis shifts to practical application and critical thinking. Early career researchers, junior professors and academics need competencies for applying the ethics and integrity concepts they have learnt to real-life scenarios in connection to conducting experiments, collecting data and critically analysing the results to gain meaningful insights. Through these activities, participants develop the skills necessary to contribute to the advancement of their field and address research questions with greater depth and sophistication. In terms of research ethics and integrity, this involves applying such knowledge and values to every step of the research.

''Evaluate and Create:''

The highest level in Bloom’s Taxonomy involves evaluating existing knowledge and creating new knowledge. All researchers play a critical role in shaping the direction of research and innovation. They are responsible for assessing the validity and significance of research findings and identifying areas for further investigation and innovation. By synthesising existing knowledge and developing new ideas, theories or methods, researchers develop their field forward and inspire the next generation of researchers and innovators. All RE/RI training should include components, which encourage learners to extend their thinking to evaluation and creation. In practice, this involves having such a robust knowledge base and values so that even when encountering new ethical dilemmas or being posed with a novel potentially integrity-threatening situation, they can rely on having the ‘tools’ to handle the situation.
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Use the SOLO taxonomy
|Instruction Step Text=Besides using Bloom’s Taxonomy to define learning objectives, the [https://www.johnbiggs.com.au/academic/solo-taxonomy/ SOLO Taxonomy] can be used .<ref name=":0">Biggs, J. (1999). What the Student Does: teaching for enhanced learning. Higher Education Research & Development/Higher Education Research and Development, 18(1), 57–75. <nowiki>https://doi.org/10.1080/0729436990180105</nowiki></ref><ref name=":1">Biggs, J., & Tang, C. (2007). Using Constructive Alignment in Outcomes-Based Teaching and Learning Teaching for Quality Learning at University (3rd ed., pp. 50-63). Maidenhead: Open University Press. </ref> The Structure of the Observed Learning Outcome, or SOLO, is a way to set the learning outcomes according to how complicated they are.<ref name=":0" /><ref name=":1" /> In this way, the students' work can be assessed according to its quality and not according to how many parts they have understood correctly: initially, we learn one or a few aspects of the task (unistructural), then multiple aspects that are unrelated to each other (multistructural), then we learn how to integrate them into a whole (relational), and lastly, we can generalise that whole to still-untaught applications (extended abstract).

[[File:SOLOtaxonomy.png|alt=|center|frame|Fig 32. SOLO taxonomy (taken from Tammeleht & Löfström, 2023)]]

The SOLO taxonomy has been used to evaluate effectiveness of trainings as well as the development of ethical sensitivity and has been proven to be effective in the context of research ethics and integrity training.<ref>Löfström, E. (2012). Students’ ethical awareness and conceptions of Research Ethics. Ethics & Behavior, 22(5), 349–361. <nowiki>https://doi.org/10.1080/10508422.2012.679136</nowiki> </ref><ref>Tammeleht, A., & Löfström, E. (2023). Learners’ self-assessment as a measure to evaluate the effectiveness of research ethics and integrity training: can we rely on self-reports? Ethics & Behavior, 1–22. <nowiki>https://doi.org/10.1080/10508422.2023.2266073</nowiki> </ref><ref>Tammeleht, A., Löfström, E. & Rodríguez-Triana, M. J. (2022). Facilitating development of research ethics and integrity leadership competencies. International Journal of Educational Integrity, 18, 11, 1-23. <nowiki>https://doi.org/10.1007/s40979-022-00102-3</nowiki> </ref><ref>Tammeleht, A., Löfström, E., & Rajando, K. (Accepted 2024) Effectiveness of research ethics and integrity competence development – what do learning diaries tell us about learning? International Journal of Ethics Education. <div></div></ref> The verbs emphasised in the descriptions below can be used as indicators of the appropriate levels in learning objectives.

''Pre-structural level (0)''

At the pre-structural level the learner fails to identify or approach topics in a meaningful way, but simply repeats the words in the question without understanding them.

''Unistructural level (1)''

At the unistructural level, the learner has sufficient knowledge to identify, recognise, count, find, label, match, name, and perform follow simple procedures. The learner identifies one relevant aspect displaying some familiarity with relevant concepts, but failing to outline multiple dimensions of it but failing to outline multiple dimensions of it. In the context of research ethics and integrity, this may mean identifying certain, perhaps common concepts, but having a limited view of them. For example, the learner may be able to identify some of the things that ought to be mentioned in an information letter to research participants but fails to understand all aspects of ensuring voluntary participation in research.

''Multi-structural level (2)''

At the multistructural level, the learner can enumerate, describe, illustrate, list, sequence, select, combine, and follow procedures, but struggles to make connections between concepts or draw conclusions based on interrelations. For example, the learner may understand that informed consent is necessary in research but fails to understand that this is so because of the need to respect people’s autonomy and right to make decisions that concern themselves.

''Relational level (3)''

At the relational level, the learner displays an ability to address the most relevant aspects of the concept and provide explanations pointing out interrelations and providing examples demonstrating their own reasoning. Corresponding action verbs include; analyse, apply, argue, compare, contrast, critique, explain causes, relate and justify. For example, the learner understands at least the main mechanisms and connections between FFP and the detrimental effects to science.

''Extended abstract level (4)''

At the extended abstract level, the coherent whole is generalised or re-conceptualised at a higher level of abstraction. The learner grasps a more abstract version of the concept, and recognises other domains to which the concept might be applied by displaying the ability to theorise, generate, generalise, hypothesise, create or reflect, formulate and reflect. For example, the learner is able to use knowledge about ethical analysis and ethical principles to solve a novel integrity-related dilemma, which the learner recognises is affecting a research group, but to which the learner has not been exposed before.
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Use The Taxonomy of Significant Learning
|Instruction Step Text=The taxonomy of Significant Learning or the Fink’s taxonomy is a non-hierarchical system that helps trainers devise learning outcomes to support deep learning. No dimension is considered more important than the other and within the course various aspects should be present.<ref name=":0">Fink, L. D. (2013). Creating significant learning experiences: an integrated approach to designing college courses (Revised and updated edition). San Francisco: Jossey-Bass.</ref> Thus, this taxonomy provides an alternative frame for devising learning objectives for training. The fact that this taxonomy emphasises care and empathy makes it very suitable for training on ethics and integrity. The following is a short overview of the various dimensions:

''Foundational knowledge''

This dimension focuses on content knowledge and includes recalling and understanding of information and ideas.

''Application''

Here the learner demonstrates skills – they can be related to the use of knowledge or include skills necessary to interact in the subject, e.g. critical and creative thinking, decision-making, solving problems etc. For example, using the steps of ethical analysis to solve a situation involving an integrity-related challenge.

''Integration''

In this dimension the learner perceives connections between various ideas, disciplines, and experiences. It includes relating various ideas to each other, comparing, contrasting ideas and examples, and so on. For example, in solving an ethical issue, different ethical theoretical viewpoints may lead to diverse actions and solutions. Recognising how for example a virtue ethical approach may lead to a different solution than reasoning based on utilitarianism may be an expression of integration.

''Human dimension''

Learners learn with others, and they gain new understanding of themselves as well as others and alsoin the learning process. They recognise how people influence each other. Understanding how to respectfully work together for the greater good is an example of how the human dimension materialises positively in practice.

''Caring''

The caring dimension includes an affective stance and involves change in a learner. The learners start to see the reason to care about a topic, they gain new interests, feelings and values about the subject. Empathy and an ethics of care are values compatible with caring.

''Learning to learn''

In this dimension the learner understands that it is not only the outcome of learning that matters but also the process is important. This dimension includes guiding one’s learning for instance by inquiry, reflection and self-assessment. The role of reflection has been emphasised as a key activity in learning and individual development.<ref name=":0" />
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Apply and Compare the Taxonomies
|Instruction Step Text=To illustrate how the taxonomies can be applied in creating learning objectives for training, we provide an example of an analysis of learning outcomes in RID-SSISS training material based on the three taxonomies (Table 1).
{{{!}} class="wikitable"
{{!}}+
Table 1: Example of an analysis of learning outcomes in RID-SSISS training material based on three taxonomies
!Level of training material
!Learning outcomes
!Bloom
!SOLO
!Fink
{{!}}-
{{!}} rowspan="3"{{!}}Foundational level
{{!}}Raise awareness of ethical issues during the research process;
{{!}}remember, understand
{{!}}unistructural, multistructural, relational
{{!}}foundational knowledge
{{!}}-
{{!}}Practice utilising the codes of conduct, being familiar with central topics;
{{!}}understand, apply, analyse
{{!}}relational, extended abstract
{{!}}application, integration
{{!}}-
{{!}}Learn together as a team, collaborate with peers.
{{!}}evaluate
{{!}}relational
{{!}}human dimension, learning to learn, caring
{{!}}-
{{!}} rowspan="3"{{!}}Advanced level
{{!}}Develop one’s research ethics competencies by combining previous knowledge and implementing new tools;
{{!}}understand, apply
{{!}}unistructural, multistructural, relational
{{!}}foundational knowledge, application, integration, learning to learn
{{!}}-
{{!}}Identify ethical issues by determining which ethical principle might be at stake.<ref name=":0">Kitchener, K. S. (1985). Ethical principles and ethical decisions in student affairs. ''New Directions for Student Services, 30,'' 17–29. [https://psycnet.apa.org/doi/10.1002/ss.37119853004 https://doi.org/10.1002/ss.37119853004]</ref>
{{!}}analyse, evaluate
{{!}}unistructural, multistructural, relational
{{!}}application, integration
{{!}}-
{{!}}Utilise the ethical analysis steps to provide solutions to ethical dilemmas (in groups).<ref name=":1">Mustajoki, A. S., & Mustajoki, H. (2017). ''A new approach to research ethics: Using guided dialogue to strengthen research communities''. Routledge. <nowiki>https://doi.org/10.4324/9781315545318</nowiki></ref>
{{!}}apply, analyse, evaluate, create
{{!}}unistructural, multistructural, relational, extended abstract
{{!}}foundational knowledge, application, integration,
human dimension,

caring, learning to learn
{{!}}-
{{!}} rowspan="5"{{!}}Leadership level
{{!}}Develop research ethics competencies by combining previous knowledge and implementing new tools;
{{!}}understand, apply
{{!}}unistructural, multistructural, relational
{{!}}foundational knowledge, application, integration, learning to learn
{{!}}-
{{!}}Identify which ethical principles might be at stake in a case.<ref name=":0" />
{{!}}analyse, evaluate
{{!}}unistructural, multistructural, relational
{{!}}application, integration
{{!}}-
{{!}}Utilise the ethical analysis steps to provide solutions to ethical dilemmas.<ref name=":1" />
{{!}}apply, analyse, evaluate, create
{{!}}unistructural, multistructural, relational, extended abstract
{{!}}foundational knowledge, application, integration,
human dimension,

caring, learning to learn
{{!}}-
{{!}}Implement different ethical approaches to the possible courses of action;
{{!}}analyse, evaluate
{{!}}unistructural, multistructural, relational
{{!}}foundational knowledge, application, integration,
{{!}}-
{{!}}Take the role of a REI leader and display (some) REI leadership competencies during their group work.
{{!}}apply, analyse, evaluate, create
{{!}}relational, extended abstract
{{!}}human dimension, caring, learning to learn
{{!}}}
Learning taxonomies are helpful not only in the design of learning goals for teaching, but also in devising appropriate targets of assessment. The coordination between learning goals or objectives, the methods used for teaching and the activities chosen to support learning, as well as the assessment of the learning are referred to as constructive alignment.<ref>Biggs, J. (1996). Enhancing teaching through constructive alignment. ''Higher Education'', 32, 347–364. <nowiki>https://doi.org/10.1007/bf00138871</nowiki></ref><ref>Biggs, J. (1999). What the Student Does: teaching for enhanced learning. Higher Education Research & Development/Higher Education Research and Development, 18(1), 57–75. <nowiki>https://doi.org/10.1080/0729436990180105</nowiki> </ref><ref>Biggs, J., & Tang, C. (2007). Using Constructive Alignment in Outcomes-Based Teaching and Learning Teaching for Quality Learning at University (3rd ed., pp. 50-63). Maidenhead: Open University Press. </ref> This means that the core components of a teaching should be geared towards supporting the same aim; namely learning. Learning taxonomies are helpful not only in the design of learning goals for teaching, but also in devising appropriate targets of assessment.
<references />
}}
{{Instruction Remarks Trainer
|Has Remarks=This project has received funding from the European Union's Horizon Europe research and innovation programme under GA No 101094714 (University of Oslo). UK participants in BEYOND are supported by UKRI grant number 10062742 (Trilateral Research) and by UKRI grant number 10067440 (Heriot-Watt University). Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union, Research Executive Agency, or UKRI. Neither the European Union nor the granting authority nor UKRI can be held responsible for them.)
}}
{{Custom TabContent Close Trainer}}
{{Related To
|Related To Instruction=Instruction:02a025d1-fb65-40b4-914f-5efed84ee112; Instruction:238dfd77-9a59-4dec-b498-3e2214ddc1fe; Instruction:55649946-f15f-4c94-89e1-b79dff231c0e; Instruction:30f33e85-09ed-4ea7-9ac2-7fb26d27ae55; Instruction:1c5bdda7-6444-497b-aaf2-53852a0fd8ee; Instruction:126b8806-0cc1-4625-b11b-f60f43ad9f6f; Instruction:Bcec25e1-c0d9-4e1d-b5c3-eb9c13ae3618; Instruction:204c2375-0867-4302-845f-cdc99e3d38bc; Instruction:Eeee007a-14ec-4ffa-b62e-514aafe4bd1a; Instruction:44eb9553-4c0a-40c9-9ed1-50b4f591e3fb; Instruction:4afd5ea7-0308-4e80-9dfc-6b66daf2ac81; Instruction:Ee61dfb7-737f-40ff-9568-aa669cd6865c
}}
{{Tags}}

Instruction:4afd5ea7-0308-4e80-9dfc-6b66daf2ac81

2025-01-31T11:23:46Z

0000-0001-7581-7849:

{{Instruction
|Title=Using Different Learning Taxonomies
|Has Related Initiative=Initiative:667449ef-1c0a-4e06-8374-830c8ad68cb8
|Instruction Goal=This module provides guidance on how to use the '''Bloom''', '''SOLO''' and '''Significant Learning (Fink's)''' taxonomies for the creation of effective and impactful learning objectives.
|Has Duration=2
|Important For=Academic staff; Prospective Research Integrity Trainers; Research integrity trainers; Senior researchers; Teachers
}}
{{Custom TabContent Trainee Open}}
{{Instruction Steps Foldout Trainee}}
{{Instruction Perspective Trainee}}
{{Instruction Remarks Trainee}}
{{Custom TabContent Close Trainee}}
{{Custom TabContent Trainer Open}}
{{Instruction Steps Foldout Trainer}}
{{Instruction Perspective Trainer
|Is About=Defining learning objectives serves as a cornerstone for creating successful and impactful learning environments. Learning objectives, i.e. what the learners should learn by taking part in the training, guides the choice of the learning process, the content, activities and assessment. Learning objectives can be defined by using different taxonomies. [https://poorvucenter.yale.edu/BloomsTaxonomy Bloom's taxonomy] categorises educational goals into a hierarchical model, from simple recall of facts to complex evaluation and creation tasks. The [https://www.johnbiggs.com.au/academic/solo-taxonomy/ SOLO] (Structure of Observed Learning Outcomes) taxonomy, on the other hand, describes levels of increasing complexity in a learner's understanding of subjects, ranging from unstructured to extended abstract levels.
|Important Because=Learning is what the learner does, but it can be facilitated through what trainers do and through appropriate teaching activities.<ref>Biggs, J. (1999). What the Student Does: teaching for enhanced learning. Higher Education Research & Development/Higher Education Research and Development, 18(1), 57–75. <nowiki>https://doi.org/10.1080/0729436990180105</nowiki></ref> The Taxonomy of Significant Learning (sometimes also referred as the [https://www.buffalo.edu/catt/teach/develop/design/learning-outcomes/finks.html Fink’s taxonomy]) is not hierarchical in the same way as the other two, however, it builds on Blooms’ taxonomy by including a long-forgotten affective component into the discussion (namely caring).<ref name=":0">Fink, L. D. (2013). Creating significant learning experiences: an integrated approach to designing college courses (Revised and updated edition). San Francisco: Jossey-Bass.</ref> It encourages to include into the learning outcomes the objectives foundational knowledge, application, integration, a human dimension, caring, and learning to acquire competencies, thus providing a holistic approach to learning.<ref name=":0" /> However, since the existing material aligns with Bloom's and SOLO frameworks, this module will primarily describe these two to ensure coherence and consistency in training delivery. Nevertheless, we encourage trainers to also consider the more effective type of learning objectives proposed in the Taxonomy of Significant Learning.
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Use the Bloom’s taxonomy
|Instruction Step Text=Bloom's Taxonomy is a well-known educational framework that offers a methodical way to classify learning objectives according to cognitive difficulty. (e.g., Adams, 2015).<ref>Adams, N. (2015). Bloom’s taxonomy of cognitive learning objectives. Journal of the Medical Library Association, 103(3), 152–153. <nowiki>https://doi.org/10.3163/1536-5050.103.3.010</nowiki> </ref> It is a hierarchical framework that uses cognitive complexity to classify learning objectives. Benjamin Bloom created it in the 1950s, and it is now a vital instrument in educational theory and practice. The taxonomy is divided into six stages: remembering, understanding, applying, analysing, evaluating, and creating. The levels are arranged from lower to higher order cognitive skills. Fundamentally, remembering entails recollecting words, information, and fundamental ideas. Understanding is more than just remembering concepts; it also involves understanding meanings. Applying necessitates applying knowledge to novel contexts or problem-solving. Analysing means dissecting data into its constituent elements and identifying connections between them. Making decisions based on standards and criteria is the process of evaluating. Creating, in the end, involves coming up with original concepts and/or interpretations. The goal of applying Bloom's Taxonomy to training aims and results is to enhance comprehension by considering the knowledge, skills, and competencies that the specific training programmes were created to impart. The Remembering, Understanding, Applying, Analysing, Evaluating, and Creating domains of Bloom's Taxonomy each reflect a different cognitive process and the depth and complexity of learning.

[[File:BloomsTaxonomy.jpg|alt=|center|frame|Fig 21. Bloom’s Taxonomy (taken from the Centre for teaching, Vanderbilt University. <nowiki>https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/</nowiki>) ]]

All taxonomic levels are relevant irrespective of the study or career level. However, the taxonomic levels may mean different things for different individuals. For example, application of knowledge may mean engaging with research designs, but senior researchers often use more complex designs than students still learning how to do research. Nevertheless, it is essential that the learning extends beyond remembering and understanding, and that the complexity of activities at all levels gradually grow as the individual gains experience, knowledge and confidence.

''Remembering and understanding:''

Here, the focus is on memorising key facts, concepts and theories relevant to the field of research and innovation. Understanding these foundational elements is critical to moving forward. For example, undergraduate students need to master the basic principles and terminology related to ethics and integrity to effectively navigate through more complex topics later. Similarly, individuals pursuing a PhD or who are new to academia need a solid understanding of basic concepts before they can conduct more in-depth analyses and applications., such as mastering the ethics of their own PhD research. Moreover, senior researchers may need to understand the basic concept of supervision and mentoring practices when it comes to supervising a team and PhD candidates.

''Apply and analyse:''

Learning should always be an active endeavour irrespective of career or studies applying and analysing knowledge. This is where the emphasis shifts to practical application and critical thinking. Early career researchers, junior professors and academics need competencies for applying the ethics and integrity concepts they have learnt to real-life scenarios in connection to conducting experiments, collecting data and critically analysing the results to gain meaningful insights. Through these activities, participants develop the skills necessary to contribute to the advancement of their field and address research questions with greater depth and sophistication. In terms of research ethics and integrity, this involves applying such knowledge and values to every step of the research.

''Evaluate and Create:''

The highest level in Bloom’s Taxonomy involves evaluating existing knowledge and creating new knowledge. All researchers play a critical role in shaping the direction of research and innovation. They are responsible for assessing the validity and significance of research findings and identifying areas for further investigation and innovation. By synthesising existing knowledge and developing new ideas, theories or methods, researchers develop their field forward and inspire the next generation of researchers and innovators. All RE/RI training should include components, which encourage learners to extend their thinking to evaluation and creation. In practice, this involves having such a robust knowledge base and values so that even when encountering new ethical dilemmas or being posed with a novel potentially integrity-threatening situation, they can rely on having the ‘tools’ to handle the situation.
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Use the SOLO taxonomy
|Instruction Step Text=Besides using Bloom’s Taxonomy to define learning objectives, the [https://www.johnbiggs.com.au/academic/solo-taxonomy/ SOLO Taxonomy] can be used .<ref name=":0">Biggs, J. (1999). What the Student Does: teaching for enhanced learning. Higher Education Research & Development/Higher Education Research and Development, 18(1), 57–75. <nowiki>https://doi.org/10.1080/0729436990180105</nowiki></ref><ref name=":1">Biggs, J., & Tang, C. (2007). Using Constructive Alignment in Outcomes-Based Teaching and Learning Teaching for Quality Learning at University (3rd ed., pp. 50-63). Maidenhead: Open University Press. </ref> The Structure of the Observed Learning Outcome, or SOLO, is a way to set the learning outcomes according to how complicated they are.<ref name=":0" /><ref name=":1" /> In this way, the students' work can be assessed according to its quality and not according to how many parts they have understood correctly: initially, we learn one or a few aspects of the task (unistructural), then multiple aspects that are unrelated to each other (multistructural), then we learn how to integrate them into a whole (relational), and lastly, we can generalise that whole to still-untaught applications (extended abstract).

[[File:SOLOtaxonomy.png|alt=|center|frame|Fig 32. SOLO taxonomy (taken from Tammeleht & Löfström, 2023)]]

The SOLO taxonomy has been used to evaluate effectiveness of trainings as well as the development of ethical sensitivity and has been proven to be effective in the context of research ethics and integrity training.<ref>Löfström, E. (2012). Students’ ethical awareness and conceptions of Research Ethics. Ethics & Behavior, 22(5), 349–361. <nowiki>https://doi.org/10.1080/10508422.2012.679136</nowiki> </ref><ref>Tammeleht, A., & Löfström, E. (2023). Learners’ self-assessment as a measure to evaluate the effectiveness of research ethics and integrity training: can we rely on self-reports? Ethics & Behavior, 1–22. <nowiki>https://doi.org/10.1080/10508422.2023.2266073</nowiki> </ref><ref>Tammeleht, A., Löfström, E. & Rodríguez-Triana, M. J. (2022). Facilitating development of research ethics and integrity leadership competencies. International Journal of Educational Integrity, 18, 11, 1-23. <nowiki>https://doi.org/10.1007/s40979-022-00102-3</nowiki> </ref><ref>Tammeleht, A., Löfström, E., & Rajando, K. (Accepted 2024) Effectiveness of research ethics and integrity competence development – what do learning diaries tell us about learning? International Journal of Ethics Education. <div></div></ref> The verbs emphasised in the descriptions below can be used as indicators of the appropriate levels in learning objectives.

''Pre-structural level (0)''

At the pre-structural level the learner fails to identify or approach topics in a meaningful way, but simply repeats the words in the question without understanding them.

''Unistructural level (1)''

At the unistructural level, the learner has sufficient knowledge to identify, recognise, count, find, label, match, name, and perform follow simple procedures. The learner identifies one relevant aspect displaying some familiarity with relevant concepts, but failing to outline multiple dimensions of it but failing to outline multiple dimensions of it. In the context of research ethics and integrity, this may mean identifying certain, perhaps common concepts, but having a limited view of them. For example, the learner may be able to identify some of the things that ought to be mentioned in an information letter to research participants but fails to understand all aspects of ensuring voluntary participation in research.

''Multi-structural level (2)''

At the multistructural level, the learner can enumerate, describe, illustrate, list, sequence, select, combine, and follow procedures, but struggles to make connections between concepts or draw conclusions based on interrelations. For example, the learner may understand that informed consent is necessary in research but fails to understand that this is so because of the need to respect people’s autonomy and right to make decisions that concern themselves.

''Relational level (3)''

At the relational level, the learner displays an ability to address the most relevant aspects of the concept and provide explanations pointing out interrelations and providing examples demonstrating their own reasoning. Corresponding action verbs include; analyse, apply, argue, compare, contrast, critique, explain causes, relate and justify. For example, the learner understands at least the main mechanisms and connections between FFP and the detrimental effects to science.

''Extended abstract level (4)''

At the extended abstract level, the coherent whole is generalised or re-conceptualised at a higher level of abstraction. The learner grasps a more abstract version of the concept, and recognises other domains to which the concept might be applied by displaying the ability to theorise, generate, generalise, hypothesise, create or reflect, formulate and reflect. For example, the learner is able to use knowledge about ethical analysis and ethical principles to solve a novel integrity-related dilemma, which the learner recognises is affecting a research group, but to which the learner has not been exposed before.
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Use The Taxonomy of Significant Learning
|Instruction Step Text=The taxonomy of Significant Learning or the Fink’s taxonomy is a non-hierarchical system that helps trainers devise learning outcomes to support deep learning. No dimension is considered more important than the other and within the course various aspects should be present.<ref name=":0">Fink, L. D. (2013). Creating significant learning experiences: an integrated approach to designing college courses (Revised and updated edition). San Francisco: Jossey-Bass.</ref> Thus, this taxonomy provides an alternative frame for devising learning objectives for training. The fact that this taxonomy emphasises care and empathy makes it very suitable for training on ethics and integrity. The following is a short overview of the various dimensions:

''Foundational knowledge''

This dimension focuses on content knowledge and includes recalling and understanding of information and ideas.

''Application''

Here the learner demonstrates skills – they can be related to the use of knowledge or include skills necessary to interact in the subject, e.g. critical and creative thinking, decision-making, solving problems etc. For example, using the steps of ethical analysis to solve a situation involving an integrity-related challenge.

''Integration''

In this dimension the learner perceives connections between various ideas, disciplines, and experiences. It includes relating various ideas to each other, comparing, contrasting ideas and examples, and so on. For example, in solving an ethical issue, different ethical theoretical viewpoints may lead to diverse actions and solutions. Recognising how for example a virtue ethical approach may lead to a different solution than reasoning based on utilitarianism may be an expression of integration.

''Human dimension''

Learners learn with others, and they gain new understanding of themselves as well as others and alsoin the learning process. They recognise how people influence each other. Understanding how to respectfully work together for the greater good is an example of how the human dimension materialises positively in practice.

''Caring''

The caring dimension includes an affective stance and involves change in a learner. The learners start to see the reason to care about a topic, they gain new interests, feelings and values about the subject. Empathy and an ethics of care are values compatible with caring.

''Learning to learn''

In this dimension the learner understands that it is not only the outcome of learning that matters but also the process is important. This dimension includes guiding one’s learning for instance by inquiry, reflection and self-assessment. The role of reflection has been emphasised as a key activity in learning and individual development.<ref name=":0" />
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Apply and Compare the Taxonomies
|Instruction Step Text=To illustrate how the taxonomies can be applied in creating learning objectives for training, we provide an example of an analysis of learning outcomes in RID-SSISS training material based on the three taxonomies (Table 1).
{{{!}} class="wikitable"
{{!}}+
Table 1: Example of an analysis of learning outcomes in RID-SSISS training material based on three taxonomies
!Level of training material
!Learning outcomes
!Bloom
!SOLO
!Fink
{{!}}-
{{!}} rowspan="3"{{!}}Foundational level
{{!}}Raise awareness of ethical issues during the research process;
{{!}}remember, understand
{{!}}unistructural, multistructural, relational
{{!}}foundational knowledge
{{!}}-
{{!}}Practice utilising the codes of conduct, being familiar with central topics;
{{!}}understand, apply, analyse
{{!}}relational, extended abstract
{{!}}application, integration
{{!}}-
{{!}}Learn together as a team, collaborate with peers.
{{!}}evaluate
{{!}}relational
{{!}}human dimension, learning to learn, caring
{{!}}-
{{!}} rowspan="3"{{!}}Advanced level
{{!}}Develop one’s research ethics competencies by combining previous knowledge and implementing new tools;
{{!}}understand, apply
{{!}}unistructural, multistructural, relational
{{!}}foundational knowledge, application, integration, learning to learn
{{!}}-
{{!}}Identify ethical issues by determining which ethical principle might be at stake.<ref name=":0">Kitchener, K. S. (1985). Ethical principles and ethical decisions in student affairs. ''New Directions for Student Services, 30,'' 17–29. [https://psycnet.apa.org/doi/10.1002/ss.37119853004 https://doi.org/10.1002/ss.37119853004]</ref>
{{!}}analyse, evaluate
{{!}}unistructural, multistructural, relational
{{!}}application, integration
{{!}}-
{{!}}Utilise the ethical analysis steps to provide solutions to ethical dilemmas (in groups).<ref name=":1">Mustajoki, A. S., & Mustajoki, H. (2017). ''A new approach to research ethics: Using guided dialogue to strengthen research communities''. Routledge. <nowiki>https://doi.org/10.4324/9781315545318</nowiki></ref>
{{!}}apply, analyse, evaluate, create
{{!}}unistructural, multistructural, relational, extended abstract
{{!}}foundational knowledge, application, integration,
human dimension,

caring, learning to learn
{{!}}-
{{!}} rowspan="5"{{!}}Leadership level
{{!}}Develop research ethics competencies by combining previous knowledge and implementing new tools;
{{!}}understand, apply
{{!}}unistructural, multistructural, relational
{{!}}foundational knowledge, application, integration, learning to learn
{{!}}-
{{!}}Identify which ethical principles might be at stake in a case.<ref name=":0" />
{{!}}analyse, evaluate
{{!}}unistructural, multistructural, relational
{{!}}application, integration
{{!}}-
{{!}}Utilise the ethical analysis steps to provide solutions to ethical dilemmas.<ref name=":1" />
{{!}}apply, analyse, evaluate, create
{{!}}unistructural, multistructural, relational, extended abstract
{{!}}foundational knowledge, application, integration,
human dimension,

caring, learning to learn
{{!}}-
{{!}}Implement different ethical approaches to the possible courses of action;
{{!}}analyse, evaluate
{{!}}unistructural, multistructural, relational
{{!}}foundational knowledge, application, integration,
{{!}}-
{{!}}Take the role of a REI leader and display (some) REI leadership competencies during their group work.
{{!}}apply, analyse, evaluate, create
{{!}}relational, extended abstract
{{!}}human dimension, caring, learning to learn
{{!}}}
Learning taxonomies are helpful not only in the design of learning goals for teaching, but also in devising appropriate targets of assessment. The coordination between learning goals or objectives, the methods used for teaching and the activities chosen to support learning, as well as the assessment of the learning are referred to as constructive alignment.<ref>Biggs, J. (1996). Enhancing teaching through constructive alignment. ''Higher Education'', 32, 347–364. <nowiki>https://doi.org/10.1007/bf00138871</nowiki></ref><ref>Biggs, J. (1999). What the Student Does: teaching for enhanced learning. Higher Education Research & Development/Higher Education Research and Development, 18(1), 57–75. <nowiki>https://doi.org/10.1080/0729436990180105</nowiki> </ref><ref>Biggs, J., & Tang, C. (2007). Using Constructive Alignment in Outcomes-Based Teaching and Learning Teaching for Quality Learning at University (3rd ed., pp. 50-63). Maidenhead: Open University Press. </ref> This means that the core components of a teaching should be geared towards supporting the same aim; namely learning. Learning taxonomies are helpful not only in the design of learning goals for teaching, but also in devising appropriate targets of assessment.
<references />
}}
{{Instruction Remarks Trainer
|Has Remarks=This project has received funding from the European Union's Horizon Europe research and innovation programme under GA No 101094714 (University of Oslo). UK participants in BEYOND are supported by UKRI grant number 10062742 (Trilateral Research) and by UKRI grant number 10067440 (Heriot-Watt University). Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union, Research Executive Agency, or UKRI. Neither the European Union nor the granting authority nor UKRI can be held responsible for them.)
}}
{{Custom TabContent Close Trainer}}
{{Related To
|Related To Instruction=Instruction:02a025d1-fb65-40b4-914f-5efed84ee112; Instruction:238dfd77-9a59-4dec-b498-3e2214ddc1fe; Instruction:55649946-f15f-4c94-89e1-b79dff231c0e; Instruction:30f33e85-09ed-4ea7-9ac2-7fb26d27ae55; Instruction:1c5bdda7-6444-497b-aaf2-53852a0fd8ee; Instruction:126b8806-0cc1-4625-b11b-f60f43ad9f6f; Instruction:Bcec25e1-c0d9-4e1d-b5c3-eb9c13ae3618; Instruction:204c2375-0867-4302-845f-cdc99e3d38bc; Instruction:Eeee007a-14ec-4ffa-b62e-514aafe4bd1a; Instruction:44eb9553-4c0a-40c9-9ed1-50b4f591e3fb; Instruction:4afd5ea7-0308-4e80-9dfc-6b66daf2ac81; Instruction:Ee61dfb7-737f-40ff-9568-aa669cd6865c
}}
{{Tags}}

Instruction:1c5bdda7-6444-497b-aaf2-53852a0fd8ee

2025-01-31T10:35:10Z

0000-0001-7581-7849:

{{Instruction
|Title=Introduction to the BEYOND Trainer Guide
|Has Related Initiative=Initiative:667449ef-1c0a-4e06-8374-830c8ad68cb8
|Instruction Goal=This module introduces the content and '''approach''' of the BEYOND Trainer Guide and provides guidance for developing '''learning goals''' and '''evaluation''' strategies.
|Has Duration=1
|Important For=Early career researchers; Graduate students; Prospective Research Integrity Trainers; Research Ethics Committees; Research integrity trainers; Senior researchers; Supervisors; Undergraduate students
}}
{{Custom TabContent Trainee Open}}
{{Instruction Steps Foldout Trainee}}
{{Instruction Perspective Trainee
|Is About=The BEYOND Trainer Guide serves as a comprehensive resource for educators seeking to equip and support researchers and research communities with the knowledge, skills, and attitudes necessary to conduct research ethically and with integrity. By fostering critical reflection, ethical decision-making, and responsible behaviour, educators play a critical role in training the next generation of researchers who are committed to the highest ethical standards in their work.

The BEYOND Trainer Guide provides a research-based overview of state-of-the art in teaching research ethics and integrity. According to the extant knowledge base, case-based and collaborative teaching and learning activities, which make use of scaffolding techniques are the best ways to support learning in the context of research ethics and integrity.

This guide provides guidance on how to effectively design teaching activities to foster responsible conduct of research to different target groups, such as students, early career researchers, experienced researchers and supervisors. It draws on prior research on teaching and learning research ethics and integrity and makes use of the vast training resources produced through selected EU-funded projects targeting ethics and integrity. It is a comprehensive resource equipped to empower trainers in delivering impactful training sessions. Trainers can adapt and personalise it based on their specific audience, training context, and personal style. But its core purpose remains clear: to equip trainers with the knowledge, strategies, and resources needed to deliver engaging, informative, and ultimately impactful training sessions. by bringing together research-based knowledge about research ethics and integrity teaching and learning, trainings produced in several EU-funded projects, and templates and activities for adapting materials to various target groups.
|Important Because=Teaching research ethics and research integrity is important for several key reasons including fostering skills such as critical thinking, problem-solving, and effective communication.<ref>ALLEA (2023) The European Code of Conduct for Research Integrity – Revised Edition 2023. Berlin. DOI 10.26356/ECOC</ref><ref>Resnik, D. B., & Shamoo, A. E. (2011). The singapore statement on research integrity. Accountability in research, 18(2), 71–75. <nowiki>https://doi.org/10.1080/08989621.2011.557296</nowiki> </ref><ref>Science Europe. (2016). Research Integrity Practices in Science Europe member Organisations. <nowiki>https://scienceeurope.org/our-resources/research-integrity-practices-in-science-europe-member-organisations/</nowiki> </ref> These skills are valuable not only for researchers but also for individuals in various professions and aspects of life.
<references />
}}
{{Instruction Step Trainee
|Instruction Step Title=Learn about the approach of this guide
|Instruction Step Text=The BEYOND approach - ‘it’s not the apple, but the orchard’ - reflects the idea that integrity is upheld as a collaborative effort. This is why it is important that training also models the collaborative way. Cases have the capacity to open up discussion space for the complexities of integrity and ethics in research, again, guiding learners to think of the full complexity, not just individuals, but also other systemic levels, including meso and macro levels, that is organisation, research community, and national, international and global context. Scaffolding provides a technique acknowledging where the individual or even a team or research community is at and designing the next steps to facilitate learning and development eventually leading to better alignment with the highest ethical and integrity standards. The point of departure is that there is always room for improvement, even in the strongest of research communities and the work starts with acknowledging status quo and identifying the next goals, which are within reach, irrespective of whether we envision the learning of individuals or communities. With these approaches; case-based and collaborative learning and scaffolding we believe training is well geared towards nurturing the orchard.

The BEYOND Trainer Guide goes beyond simply listing training materials; it adds value by explaining various pedagogical approaches that can be applied to enhance the use of different materials. It shows how learning taxonomies can be applied to create learning-focused training (as opposed to mere information transmission) irrespective of which materials produced in EU-funded projects that are implemented. We have structured the material according to target group, so that trainers can easily identify materials that are suitable for the target group they are training.

Additionally, the content is also structured according to the type of learning activities to support those trainers who wish to work using specific activities but may hesitate whether they are suitable for a particular target group, or simply would like to know more about the activity itself.

To summarise, the BEYOND approach is manifested in the Trainer Guide as:

- A proposal for a research-based approach to an ‘orchard pedagogy’

- Suggestions for measuring training effect to gain an indication of the preparedness of the research community to develop a culture of integrity

Facilitation for using existing research ethics and integrity training resources by providing two alternative structures for trainers, including one, which addresses various actors in ‘the orchard’ through a career-level approach. We wish trainers and other readers, as well as learners taking part in trainings and learning activities utilising the resources referred to in the BEYOND Trainer Guide, a joyful journey through the orchard!
}}
{{Instruction Step Trainee
|Instruction Step Title=Reflect on evaluating effectiveness
|Instruction Step Text=The BEYOND Trainer Guide introduces effectiveness measures to help trainers assess whether the training provided is impactful and beneficial. The versatile evaluation tools are designed to be applicable to various target groups and compatible with a variety of training activities and resources. Such evaluation measures are often absent in training resources, yet they provide trainers with a valuable mechanism to ensure how effectively training supports learning. Understanding how training facilitates learning and development is necessary in the process of fostering and strengthening integrity in the research community. Provision of training is a necessary component of the overall building of a culture of integrity. Yet training, the effects of which are not monitored, falls short of its potential to mirror the change it contributes to the research community. Therefore, in the orchard approach, learning and development provides important information about the readiness of the community to build a culture of integrity. Evaluating training effectiveness to ensure training programmes achieve their intended outcomes is crucial because it connects training investments to tangible results, ensuring that the effort put into developing and delivering training is worthwhile, and for pinpointing further development needs.

Effectiveness of research ethics and integrity (REI) training can be viewed through an established effectiveness framework, which identifies four outcome domains, namely:

1. reactions (participants’ self-assessment),

2. learning (knowledge, content),

3. behaviour (acting in the research community),

4. results (e.g. institutional outcomes).<ref>Kirkpatrick, D. L. (1959). Techniques for Evaluation Training Programs. Journal of the American Society of Training Directors, 13, 21-26.</ref><ref>Praslova, L. (2010). Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in higher education. ''Educational Assessment, Evaluation and Accountability, 22''(3), 215–225. [https://psycnet.apa.org/doi/10.1007/s11092-010-9098-7 https://doi.org/10.1007/s11092-010-9098-7]</ref>

Evaluating development of ethical competencies should be determined through done as a system to get a more holistic picture. To do this, one can combine different forms of measurement, such as self-assessment and facilitator feedback as well as attitudes and behaviour treats (in tasks that display REI competencies in the research community, like research proposals, ethics sections of theses, articles, etc.). Furthermore, measurement could take place at different times to gain insight into the learning process, learning outcome, and long-term implications, namely:

• during the training (learning process),

• right after the training – students' and facilitator’s self-reports,

• later as part of another event or course where the display of REI competencies is expected (like RE section in theses and articles, research proposal, evaluation of RE situation in the department, etc.)

It is also important to consider what to do with the results, that is what kind of changes are necessary to improve teaching and/or the environment to build a culture of integrity.

Different tools can be used to collect various learning outputs and analysis instruments can be implemented to analyse the information that has been collected (Table 2). By analysis instruments we mean the taxonomies of learning and application of theoretical models, such as levels of reflection, ethical principles and so on (if data available are mainly in a qualitative format) or statistics and learning analytics (if the data are mainly in quantitative format).
{{{!}} class="wikitable"
{{!}}+Table 1: Tools and analytical instruments for collecting learning outputs in research ethics and integrity training
!Tool for collecting learning outputs
!Details
!Analysis instrument
{{!}}-
{{!}}'''''ProLearning'' app'''
{{!}}''ProLearning'': [http://www.prolearning.realto.ch/ www.prolearning.realto.ch]
{{!}}learning analytics
{{!}}-
{{!}}'''Engagement app'''
{{!}}App under development, [https://forms.office.com/Pages/ShareFormPage.aspx?id=WXWumNwQiEKOLkWT5i_j7twYn7PlpvpDlgGDpz2LgIdUMk5XRTVYQTVKRFRDWDlHOUdGU1FHTUlFVi4u&sharetoken=03epmvYBRpmfXvpRg9os form] (for copying and editing)
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Self-Reflection Form/Compass'''
{{!}}App under development, [https://forms.office.com/Pages/ShareFormPage.aspx?id=WXWumNwQiEKOLkWT5i_j7twYn7PlpvpDlgGDpz2LgIdUMk5XRTVYQTVKRFRDWDlHOUdGU1FHTUlFVi4u&sharetoken=03epmvYBRpmfXvpRg9os form] (for copying and editing)
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Pre-post texts'''
{{!}}Collect a short text (e.g. a response to a case or short essay) before the training and after the training
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Learning diaries'''
{{!}}Ask learners keep a diary over a certain period, for each submission provide some guiding questions or topics
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Group reports'''
{{!}}Ask groups working together to provide a (short) group report (or provide a template with points to work on)
{{!}}SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Group discussions'''
{{!}}Monitor the group discussions to evaluate the level of understanding and content discussed (scaffold as appropriate)
{{!}}SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Group dynamics'''
{{!}}''CoTrack'' application: https://www.cotrack.website/en/
{{!}}learning analytics
{{!}}-
{{!}}'''Online learning platform'''
{{!}}Make use of accumulated authentic learning outputs in the learning platform.
{{!}}statistics, SOLO taxonomy, reflection scale, content criteria
{{!}}-
{{!}}'''Domain-specific/ domain-transcending measure'''
{{!}}Use either of the two forms measuring recognition and exemplifying of ethical issues.
{{!}}statistics, SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Retention check'''
{{!}}After a certain time (few weeks/months) ask learners to provide a short text (analysis of a case, short essay on an ethics topic/question). Compare the levels of understanding to another piece collected during or right after the training.
{{!}}SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Vignettes'''
{{!}}This can be used for measuring ethical sensitivity in (non-)training context
{{!}}statistics, EASM (based on the SOLO taxonomy), content criteria
{{!}}-
{{!}}'''National surveys'''
{{!}}Can be used for analysing training-related content in reports and monitoring the display of REI leadership.
{{!}}statistics, REI leadership framework
{{!}}}
Evaluation tools can give further insight into the effectiveness of the training and materials proposed. This will help trainers to adjust training content and delivery methods to improve trainees’ learning experience and outcomes. We propose mixing various tools for collecting learning outputs and adjusting them to the intended target groups (throughout the training guide suggestions are provided on which tools would be most suitable for various target groups).
<references />
}}
{{Instruction Remarks Trainee
|Has Remarks=This project has received funding from the European Union's Horizon Europe research and innovation programme under GA No 101094714 (University of Oslo). UK participants in BEYOND are supported by UKRI grant number 10062742 (Trilateral Research) and by UKRI grant number 10067440 (Heriot-Watt University). Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union, Research Executive Agency, or UKRI. Neither the European Union nor the granting authority nor UKRI can be held responsible for them.)
}}
{{Custom TabContent Close Trainee}}
{{Custom TabContent Trainer Open}}
{{Instruction Steps Foldout Trainer}}
{{Instruction Perspective Trainer
|Is About=The BEYOND Trainer Guide serves as a comprehensive resource for educators seeking to equip and support researchers and research communities with the knowledge, skills, and attitudes necessary to conduct research ethically and with integrity. By fostering critical reflection, ethical decision-making, and responsible behaviour, educators play a critical role in training the next generation of researchers who are committed to the highest ethical standards in their work.

The BEYOND Trainer Guide provides a research-based overview of state-of-the art in teaching research ethics and integrity. According to the extant knowledge base, case-based and collaborative teaching and learning activities, which make use of scaffolding techniques are the best ways to support learning in the context of research ethics and integrity.

This guide provides guidance on how to effectively design teaching activities to foster responsible conduct of research to different target groups, such as students, early career researchers, experienced researchers and supervisors. It draws on prior research on teaching and learning research ethics and integrity and makes use of the vast training resources produced through selected EU-funded projects targeting ethics and integrity. It is a comprehensive resource equipped to empower trainers in delivering impactful training sessions. Trainers can adapt and personalize it based on their specific audience, training context, and personal style. But its core purpose remains clear: to equip trainers with the knowledge, strategies, and resources needed to deliver engaging, informative, and ultimately impactful training sessions. by bringing together research-based knowledge about research ethics and integrity teaching and learning, trainings produced in several EU-funded projects, and templates and activities for adapting materials to various target groups.
|Important Because=Teaching research ethics and research integrity is incredibly important for several key reasons including fostering important skills such as critical thinking, problem-solving, and effective communication.<ref>ALLEA (2023) The European Code of Conduct for Research Integrity – Revised Edition 2023. Berlin. DOI 10.26356/ECOC</ref><ref>Resnik, D. B., & Shamoo, A. E. (2011). The singapore statement on research integrity. Accountability in research, 18(2), 71–75. <nowiki>https://doi.org/10.1080/08989621.2011.557296</nowiki> </ref><ref>Science Europe. (2016). Research Integrity Practices in Science Europe member Organisations. <nowiki>https://scienceeurope.org/our-resources/research-integrity-practices-in-science-europe-member-organisations/</nowiki> </ref> These skills are valuable not only for researchers but also for individuals in various professions and aspects of life.
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Reflect on the goals of RCR education
|Instruction Step Text=The ethical conduct of research is crucial for maintaining the integrity of science. [https://oir.nih.gov/sourcebook/ethical-conduct/responsible-conduct-research-training Responsible Conduct of Research] (RCR) advances scientific goals, fosters a collaborative research environment, and builds public trust in scientific advances that benefit society. Conversely, unethical research practices such as data fabrication and falsification lead to the dissemination of false hypotheses and unreliable data, which harms the search for valid knowledge. Similarly, plagiarism and harassment undermine respect and trust among researchers, while fraudulent or socially irresponsible research weakens public trust and support for science. [https://oir.nih.gov/sourcebook/ethical-conduct/responsible-conduct-research-training The goals of RCR training] include developing a culture of integrity in science and improving knowledge and awareness about the conduct of research.

RCR training and education should be continuous and extend beyond the academic programme throughout a scientist's career. This education can take place in a variety of contexts, such as seminars, workshops, conferences on research ethics and informal mentoring sessions, training courses and laboratory meetings where ethical behaviour and practises are discussed.

As described by van den Hoven and colleagues, multiple factors influence research integrity (RI) training (learning objectives), RI learning (learning outcomes), and changes in RI behaviour (learning outcomes).<ref name=":0">Van Den Hoven, M., Lindemann, T., Zollitsch, L., & Prieß-Buchheit, J. (2023). A Taxonomy for Research Integrity Training: Design, conduct, and improvements in research integrity courses. Science and Engineering Ethics, 29(3). <nowiki>https://doi.org/10.1007/s11948-022-00425-x</nowiki> </ref> Through these, it is possible to promote trustworthy science, responsible research practices, and high integrity/ethical standards. "Training effects" Can be conceptualised through the (intended) impacts of RI training on various performance levels, including individual, institutional, and societal levels.<ref name=":0" /> Furthermore, the authors distinguish among intended training effects (for example changes in behaviour); training input and output (that is training focus/content and learners orientating themselves towards this content); outcomes (that is, learners change their behaviour); and training impact (manifestation of the outcome, such as decreases in misconduct).

[https://nap.nationalacademies.org/read/21896/chapter/15 Effective education in research ethics and integrity aims to achieve several crucial goals.]Defining goals for teaching research integrity and research ethics is crucial to promote and foster responsible research practices and a trustworthy research ecosystem. The main goals to achieve in training RE/RI are related to promoting knowledge (in relation to responsible research practices, norms, and guidelines), skills (in relation to ethical decision-making, problem solving and critical thinking), ‘theoretical’ attitude (in relation to what should be done to foster responsible research) and ‘practical’ behaviour (in relation to how researchers behave in their daily practice).<ref>Kalichman, M. W., & Plemmons, D. K. (2007). Reported goals for responsible conduct of research courses. ''Academic Medicine, 82''(9), 846–852. [https://psycnet.apa.org/doi/10.1097/ACM.0b013e31812f78bf https://doi.org/10.1097/ACM.0b013e31812f78bf]</ref>
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Learn about the approach of this guide
|Instruction Step Text=The BEYOND approach - ‘it’s not the apple, but the orchard’ - reflects the idea that integrity is upheld as a collaborative effort. This is why it is important that training also models the collaborative way. Cases have the capacity to open up discussion space for the complexities of integrity and ethics in research, again, guiding learners to think of the full complexity, not just individuals, but also other systemic levels, including meso and macro levels, that is organisation, research community, and national, international and global context. Scaffolding provides a technique acknowledging where the individual or even a team or research community is at and designing the next steps to facilitate learning and development eventually leading to better alignment with the highest ethical and integrity standards. The point of departure is that there is always room for improvement, even in the strongest of research communities and the work starts with acknowledging status quo and identifying the next goals, which are within reach, irrespective of whether we envision the learning of individuals or communities. With these approaches; case-based and collaborative learning and scaffolding we believe training is well geared towards nurturing the orchard.

The BEYOND Trainer Guide goes beyond simply listing training materials; it adds value by explaining various pedagogical approaches that can be applied to enhance the use of different materials. It shows how learning taxonomies can be applied to create learning-focused training (as opposed to mere information transmission) irrespective of which materials produced in EU-funded projects that are implemented. We have structured the material according to target group, so that trainers can easily identify materials that are suitable for the target group they are training.

Additionally, the content is also structured according to the type of learning activities to support those trainers who wish to work using specific activities but may hesitate whether they are suitable for a particular target group, or simply would like to know more about the activity itself.

To summarise, the BEYOND approach is manifested in the Trainer Guide as:

- A proposal for a research-based approach to an ‘orchard pedagogy’

- Suggestions for measuring training effect to gain an indication of the preparedness of the research community to develop a culture of integrity

Facilitation for using existing RE/RI training resources by providing two alternative structures for trainers, including one, which addresses various actors in ‘the orchard’ through a career-level approach. We wish trainers and other readers, as well as learners taking part in trainings and learning activities utilising the resources referred to in the BEYOND Trainer Guide, a joyful journey through the orchard!
}}
{{Instruction Step Trainer
|Instruction Step Title=Reflect on the goals of RCR education
|Instruction Step Text=The ethical conduct of research is crucial for maintaining the integrity of science. [https://oir.nih.gov/sourcebook/ethical-conduct/responsible-conduct-research-training Responsible Conduct of Research] (RCR) advances scientific goals, fosters a collaborative research environment, and builds public trust in scientific advances that benefit society. Conversely, unethical research practices such as data fabrication and falsification lead to the dissemination of false hypotheses and unreliable data, which harms the search for valid knowledge. Similarly, plagiarism and harassment undermine respect and trust among researchers, while fraudulent or socially irresponsible research weakens public trust and support for science. [https://oir.nih.gov/sourcebook/ethical-conduct/responsible-conduct-research-training The goals of RCR training] include developing a culture of integrity in science and improving knowledge and awareness about the conduct of research.

RCR training and education should be continuous and extend beyond the academic programme throughout a scientist's career. This education can take place in a variety of contexts, such as seminars, workshops, conferences on research ethics and informal mentoring sessions, training courses and laboratory meetings where ethical behaviour and practises are discussed.

As described by van den Hoven and colleagues, multiple factors influence research integrity (RI) training (learning objectives), RI learning (learning outcomes), and changes in RI behaviour (learning outcomes).<ref name=":0">Van Den Hoven, M., Lindemann, T., Zollitsch, L., & Prieß-Buchheit, J. (2023). A Taxonomy for Research Integrity Training: Design, conduct, and improvements in research integrity courses. Science and Engineering Ethics, 29(3). <nowiki>https://doi.org/10.1007/s11948-022-00425-x</nowiki> </ref> Through these, it is possible to promote trustworthy science, responsible research practices, and high integrity/ethical standards. "Training effects" Can be conceptualised through the (intended) impacts of RI training on various performance levels, including individual, institutional, and societal levels.<ref name=":0" /> Furthermore, the authors distinguish among intended training effects (for example changes in behaviour); training input and output (that is training focus/content and learners orientating themselves towards this content); outcomes (that is, learners change their behaviour); and training impact (manifestation of the outcome, such as decreases in misconduct).

[https://nap.nationalacademies.org/read/21896/chapter/15 Effective education in research ethics and integrity aims to achieve several crucial goals.]Defining goals for teaching research integrity and research ethics is crucial to promote and foster responsible research practices and a trustworthy research ecosystem. The main goals to achieve in training RE/RI are related to promoting knowledge (in relation to responsible research practices, norms, and guidelines), skills (in relation to ethical decision-making, problem solving and critical thinking), ‘theoretical’ attitude (in relation to what should be done to foster responsible research) and ‘practical’ behaviour (in relation to how researchers behave in their daily practice).<ref>Kalichman, M. W., & Plemmons, D. K. (2007). Reported goals for responsible conduct of research courses. ''Academic Medicine, 82''(9), 846–852. [https://psycnet.apa.org/doi/10.1097/ACM.0b013e31812f78bf https://doi.org/10.1097/ACM.0b013e31812f78bf]</ref>
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Reflect on evaluating effectiveness
|Instruction Step Text=The BEYOND Trainer Guide introduces effectiveness measures to help trainers assess whether the training provided is impactful and beneficial. The versatile evaluation tools (developed in WP4) are designed to be applicable to various target groups and compatible with a variety of training activities and resources. Such evaluation measures are often absent in training resources, yet they provide trainers with a valuable mechanism to ensure how effectively training supports learning. Understanding how training facilitates learning and development is necessary in the process of fostering and strengthening integrity in the research community. Provision of training is a necessary component of the overall building of a culture of integrity. Yet training, the effects of which are not monitored, falls short of its potential to mirror the change it contributes to the research community. Therefore, in the orchard approach, learning and development provides important information about the readiness of the community to build a culture of integrity. Evaluating training effectiveness to ensure training programs achieve their intended outcomes is crucial because it connects training investments to tangible results, ensuring that the effort put into developing and delivering training is worthwhile, and for pinpointing further development needs.

Effectiveness of research ethics and integrity (REI) training can be viewed through an established effectiveness framework, which identifies four outcome domains, namely:

1. reactions (participants’ self-assessment),

2. learning (knowledge, content),

3. behaviour (acting in the research community),

4. results (e.g. institutional outcomes).<ref>Kirkpatrick, D. L. (1959). Techniques for Evaluation Training Programs. Journal of the American Society of Training Directors, 13, 21-26.</ref><ref>Praslova, L. (2010). Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in higher education. ''Educational Assessment, Evaluation and Accountability, 22''(3), 215–225. [https://psycnet.apa.org/doi/10.1007/s11092-010-9098-7 https://doi.org/10.1007/s11092-010-9098-7]</ref>

Evaluating development of ethical competencies should be determined through done as a system to get a more holistic picture. To do this, one can combine different forms of measurement, such as self-assessment and facilitator feedback as well as attitudes and behaviour treats (in tasks that display REI competencies in the research community, like research proposals, ethics sections of theses, articles, etc.). Furthermore, measurement could take place at different time points to gain insight into the learning process, learning outcome, and long-term implications, namely:

• during the training (learning process),

• right after the training – students' and facilitator’s self-reports,

• later as part of another event or course where the display of REI competencies is expected (like RE section in theses and articles, research proposal, evaluation of RE situation in the department, etc.)

It is also important to consider what to do with the results, that is what kind of changes are necessary to improve teaching and/or the environment to build the culture of integrity.

Different tools can be used to collect various learning outputs and analysis instruments can be implemented to analyse the information that has been collected (Table 2) By analysis instruments we mean the taxonomies of learning and application of theoretical models, such as levels of reflection, ethical principles and so on (if data available are mainly in a qualitative format) or statistics and learning analytics (if the data are mainly in quantitative format).
{{{!}} class="wikitable"
{{!}}+Table 1: Tools and analytical instruments for collecting learning outputs in research ethics and integrity training
!Tool for collecting learning outputs
!Details
!Analysis instrument
{{!}}-
{{!}}'''''ProLearning'' app'''
{{!}}''ProLearning'': [http://www.prolearning.realto.ch/ www.prolearning.realto.ch]
{{!}}learning analytics
{{!}}-
{{!}}'''Engagement app'''
{{!}}App under development, [https://forms.office.com/Pages/ShareFormPage.aspx?id=WXWumNwQiEKOLkWT5i_j7twYn7PlpvpDlgGDpz2LgIdUMk5XRTVYQTVKRFRDWDlHOUdGU1FHTUlFVi4u&sharetoken=03epmvYBRpmfXvpRg9os form] (for copying and editing)
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Self-Reflection Form/Compass'''
{{!}}App under development, [https://forms.office.com/Pages/ShareFormPage.aspx?id=WXWumNwQiEKOLkWT5i_j7twYn7PlpvpDlgGDpz2LgIdUMk5XRTVYQTVKRFRDWDlHOUdGU1FHTUlFVi4u&sharetoken=03epmvYBRpmfXvpRg9os form] (for copying and editing)
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Pre-post texts'''
{{!}}Collect a short text (e.g. a response to a case or short essay) before the training and after the training
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Learning diaries'''
{{!}}Ask learners keep a diary over a certain period, for each submission provide some guiding questions or topics
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Group reports'''
{{!}}Ask groups working together to provide a (short) group report (or provide a template with points to work on)
{{!}}SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Group discussions'''
{{!}}Monitor the group discussions to evaluate the level of understanding and content discussed (scaffold as appropriate)
{{!}}SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Group dynamics'''
{{!}}''CoTrack'' application: https://www.cotrack.website/en/
{{!}}learning analytics
{{!}}-
{{!}}'''Online learning platform'''
{{!}}Make use of accumulated authentic learning outputs in the learning platform.
{{!}}statistics, SOLO taxonomy, reflection scale, content criteria
{{!}}-
{{!}}'''Domain-specific/ domain-transcending measure'''
{{!}}Use either of the two forms (WP4.2) measuring recognition and exemplifying of ethical issues.
{{!}}statistics, SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Retention check'''
{{!}}After a certain time (few weeks/months) ask learners to provide a short text (analysis of a case, short essay on an ethics topic/question). Compare the levels of understanding to another piece collected during or right after the training.
{{!}}SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Vignettes'''
{{!}}This can be used for measuring ethical sensitivity in (non-)training context
{{!}}statistics, EASM (based on the SOLO taxonomy), content criteria
{{!}}-
{{!}}'''National surveys'''
{{!}}Can be used for analysing training-related content in reports and monitoring the display of REI leadership.
{{!}}statistics, REI leadership framework
{{!}}}
Evaluation tools can give further insight into the effectiveness of the training and materials proposed. This will help trainers to adjust training content and delivery methods to improve trainees’ learning experience and outcomes. We propose mixing various tools for collecting learning outputs and adjusting them to the intended target groups (thoughout the trainig guide suggestions are provided on which tools would be most suitable for various target groups).
<references />
}}
{{Instruction Remarks Trainer
|Has Remarks=This project has received funding from the European Union's Horizon Europe research and innovation programme under GA No 101094714 (University of Oslo). UK participants in BEYOND are supported by UKRI grant number 10062742 (Trilateral Research) and by UKRI grant number 10067440 (Heriot-Watt University). Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union, Research Executive Agency, or UKRI. Neither the European Union nor the granting authority nor UKRI can be held responsible for them.)
}}
{{Custom TabContent Close Trainer}}
{{Related To
|Related To Instruction=Instruction:02a025d1-fb65-40b4-914f-5efed84ee112; Instruction:238dfd77-9a59-4dec-b498-3e2214ddc1fe; Instruction:55649946-f15f-4c94-89e1-b79dff231c0e; Instruction:30f33e85-09ed-4ea7-9ac2-7fb26d27ae55; Instruction:126b8806-0cc1-4625-b11b-f60f43ad9f6f; Instruction:Bcec25e1-c0d9-4e1d-b5c3-eb9c13ae3618; Instruction:204c2375-0867-4302-845f-cdc99e3d38bc; Instruction:Eeee007a-14ec-4ffa-b62e-514aafe4bd1a; Instruction:44eb9553-4c0a-40c9-9ed1-50b4f591e3fb; Instruction:4afd5ea7-0308-4e80-9dfc-6b66daf2ac81; Instruction:Ee61dfb7-737f-40ff-9568-aa669cd6865c
}}
{{Tags}}

Instruction:1c5bdda7-6444-497b-aaf2-53852a0fd8ee

2025-01-31T10:33:13Z

0000-0001-7581-7849:

{{Instruction
|Title=Introduction to the BEYOND Trainer Guide
|Has Related Initiative=Initiative:667449ef-1c0a-4e06-8374-830c8ad68cb8
|Instruction Goal=This module introduces the content and '''approach''' of the BEYOND Trainer Guide and provides guidance for developing '''learning goals''' and '''evaluation''' strategies.
|Has Duration=1
|Important For=Early career researchers; Graduate students; Prospective Research Integrity Trainers; Research Ethics Committees; Research integrity trainers; Senior researchers; Supervisors; Undergraduate students
}}
{{Custom TabContent Trainee Open}}
{{Instruction Steps Foldout Trainee}}
{{Instruction Perspective Trainee
|Is About=The BEYOND Trainer Guide serves as a comprehensive resource for educators seeking to equip and support researchers and research communities with the knowledge, skills, and attitudes necessary to conduct research ethically and with integrity. By fostering critical reflection, ethical decision-making, and responsible behaviour, educators play a critical role in training the next generation of researchers who are committed to the highest ethical standards in their work.

The BEYOND Trainer Guide provides a research-based overview of state-of-the art in teaching research ethics and integrity. According to the extant knowledge base, case-based and collaborative teaching and learning activities, which make use of scaffolding techniques are the best ways to support learning in the context of research ethics and integrity.

This guide provides guidance on how to effectively design teaching activities to foster responsible conduct of research to different target groups, such as students, early career researchers, experienced researchers and supervisors. It draws on prior research on teaching and learning research ethics and integrity and makes use of the vast training resources produced through selected EU-funded projects targeting ethics and integrity. It is a comprehensive resource equipped to empower trainers in delivering impactful training sessions. Trainers can adapt and personalise it based on their specific audience, training context, and personal style. But its core purpose remains clear: to equip trainers with the knowledge, strategies, and resources needed to deliver engaging, informative, and ultimately impactful training sessions. by bringing together research-based knowledge about research ethics and integrity teaching and learning, trainings produced in several EU-funded projects, and templates and activities for adapting materials to various target groups.
|Important Because=Teaching research ethics and research integrity is important for several key reasons including fostering skills such as critical thinking, problem-solving, and effective communication.<ref>ALLEA (2023) The European Code of Conduct for Research Integrity – Revised Edition 2023. Berlin. DOI 10.26356/ECOC</ref><ref>Resnik, D. B., & Shamoo, A. E. (2011). The singapore statement on research integrity. Accountability in research, 18(2), 71–75. <nowiki>https://doi.org/10.1080/08989621.2011.557296</nowiki> </ref><ref>Science Europe. (2016). Research Integrity Practices in Science Europe member Organisations. <nowiki>https://scienceeurope.org/our-resources/research-integrity-practices-in-science-europe-member-organisations/</nowiki> </ref> These skills are valuable not only for researchers but also for individuals in various professions and aspects of life.
<references />
}}
{{Instruction Step Trainee
|Instruction Step Title=Learn about the approach of this guide
|Instruction Step Text=The BEYOND approach - ‘it’s not the apple, but the orchard’ - reflects the idea that integrity is upheld as a collaborative effort. This is why it is important that training also models the collaborative way. Cases have the capacity to open up discussion space for the complexities of integrity and ethics in research, again, guiding learners to think of the full complexity, not just individuals, but also other systemic levels, including meso and macro levels, that is organisation, research community, and national, international and global context. Scaffolding provides a technique acknowledging where the individual or even a team or research community is at and designing the next steps to facilitate learning and development eventually leading to better alignment with the highest ethical and integrity standards. The point of departure is that there is always room for improvement, even in the strongest of research communities and the work starts with acknowledging status quo and identifying the next goals, which are within reach, irrespective of whether we envision the learning of individuals or communities. With these approaches; case-based and collaborative learning and scaffolding we believe training is well geared towards nurturing the orchard.

The BEYOND Trainer Guide goes beyond simply listing training materials; it adds value by explaining various pedagogical approaches that can be applied to enhance the use of different materials. It shows how learning taxonomies can be applied to create learning-focused training (as opposed to mere information transmission) irrespective of which materials produced in EU-funded projects that are implemented. We have structured the material according to target group, so that trainers can easily identify materials that are suitable for the target group they are training.

Additionally, the content is also structured according to the type of learning activities to support those trainers who wish to work using specific activities but may hesitate whether they are suitable for a particular target group, or simply would like to know more about the activity itself.

To summarise, the BEYOND approach is manifested in the Trainer Guide as:

- A proposal for a research-based approach to an ‘orchard pedagogy’

- Suggestions for measuring training effect to gain an indication of the preparedness of the research community to develop a culture of integrity

Facilitation for using existing research ethics and integrity training resources by providing two alternative structures for trainers, including one, which addresses various actors in ‘the orchard’ through a career-level approach. We wish trainers and other readers, as well as learners taking part in trainings and learning activities utilising the resources referred to in the BEYOND Trainer Guide, a joyful journey through the orchard!
}}
{{Instruction Step Trainee
|Instruction Step Title=Reflect on evaluating effectiveness
|Instruction Step Text=The BEYOND Trainer Guide introduces effectiveness measures to help trainers assess whether the training provided is impactful and beneficial. The versatile evaluation tools are designed to be applicable to various target groups and compatible with a variety of training activities and resources. Such evaluation measures are often absent in training resources, yet they provide trainers with a valuable mechanism to ensure how effectively training supports learning. Understanding how training facilitates learning and development is necessary in the process of fostering and strengthening integrity in the research community. Provision of training is a necessary component of the overall building of a culture of integrity. Yet training, the effects of which are not monitored, falls short of its potential to mirror the change it contributes to the research community. Therefore, in the orchard approach, learning and development provides important information about the readiness of the community to build a culture of integrity. Evaluating training effectiveness to ensure training programmes achieve their intended outcomes is crucial because it connects training investments to tangible results, ensuring that the effort put into developing and delivering training is worthwhile, and for pinpointing further development needs.

Effectiveness of research ethics and integrity (REI) training can be viewed through an established effectiveness framework, which identifies four outcome domains, namely:

1. reactions (participants’ self-assessment),

2. learning (knowledge, content),

3. behaviour (acting in the research community),

4. results (e.g. institutional outcomes).<ref>Kirkpatrick, D. L. (1959). Techniques for Evaluation Training Programs. Journal of the American Society of Training Directors, 13, 21-26.</ref><ref>Praslova, L. (2010). Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in higher education. ''Educational Assessment, Evaluation and Accountability, 22''(3), 215–225. [https://psycnet.apa.org/doi/10.1007/s11092-010-9098-7 https://doi.org/10.1007/s11092-010-9098-7]</ref>

Evaluating development of ethical competencies should be determined through done as a system to get a more holistic picture. To do this, one can combine different forms of measurement, such as self-assessment and facilitator feedback as well as attitudes and behaviour treats (in tasks that display REI competencies in the research community, like research proposals, ethics sections of theses, articles, etc.). Furthermore, measurement could take place at different times to gain insight into the learning process, learning outcome, and long-term implications, namely:

• during the training (learning process),

• right after the training – students' and facilitator’s self-reports,

• later as part of another event or course where the display of REI competencies is expected (like RE section in theses and articles, research proposal, evaluation of RE situation in the department, etc.)

It is also important to consider what to do with the results, that is what kind of changes are necessary to improve teaching and/or the environment to build a culture of integrity.

Different tools can be used to collect various learning outputs and analysis instruments can be implemented to analyse the information that has been collected (Table 2). By analysis instruments we mean the taxonomies of learning and application of theoretical models, such as levels of reflection, ethical principles and so on (if data available are mainly in a qualitative format) or statistics and learning analytics (if the data are mainly in quantitative format).
{{{!}} class="wikitable"
{{!}}+Table 1: Tools and analytical instruments for collecting learning outputs in research ethics and integrity training
!Tool for collecting learning outputs
!Details
!Analysis instrument
{{!}}-
{{!}}'''''ProLearning'' app'''
{{!}}''ProLearning'': [http://www.prolearning.realto.ch/ www.prolearning.realto.ch]
{{!}}learning analytics
{{!}}-
{{!}}'''Engagement app'''
{{!}}App under development, [https://forms.office.com/Pages/ShareFormPage.aspx?id=WXWumNwQiEKOLkWT5i_j7twYn7PlpvpDlgGDpz2LgIdUMk5XRTVYQTVKRFRDWDlHOUdGU1FHTUlFVi4u&sharetoken=03epmvYBRpmfXvpRg9os form] (for copying and editing)
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Self-Reflection Form/Compass'''
{{!}}App under development, [https://forms.office.com/Pages/ShareFormPage.aspx?id=WXWumNwQiEKOLkWT5i_j7twYn7PlpvpDlgGDpz2LgIdUMk5XRTVYQTVKRFRDWDlHOUdGU1FHTUlFVi4u&sharetoken=03epmvYBRpmfXvpRg9os form] (for copying and editing)
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Pre-post texts'''
{{!}}Collect a short text (e.g. a response to a case or short essay) before the training and after the training
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Learning diaries'''
{{!}}Ask learners keep a diary over a certain period, for each submission provide some guiding questions or topics
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Group reports'''
{{!}}Ask groups working together to provide a (short) group report (or provide a template with points to work on)
{{!}}SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Group discussions'''
{{!}}Monitor the group discussions to evaluate the level of understanding and content discussed (scaffold as appropriate)
{{!}}SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Group dynamics'''
{{!}}''CoTrack'' application: https://www.cotrack.website/en/
{{!}}learning analytics
{{!}}-
{{!}}'''Online learning platform'''
{{!}}Make use of accumulated authentic learning outputs in the learning platform.
{{!}}statistics, SOLO taxonomy, reflection scale, content criteria
{{!}}-
{{!}}'''Domain-specific/ domain-transcending measure'''
{{!}}Use either of the two forms (WP4.2) measuring recognition and exemplifying of ethical issues.
{{!}}statistics, SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Retention check'''
{{!}}After a certain time (few weeks/months) ask learners to provide a short text (analysis of a case, short essay on an ethics topic/question). Compare the levels of understanding to another piece collected during or right after the training.
{{!}}SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Vignettes'''
{{!}}This can be used for measuring ethical sensitivity in (non-)training context
{{!}}statistics, EASM (based on the SOLO taxonomy), content criteria
{{!}}-
{{!}}'''National surveys'''
{{!}}Can be used for analysing training-related content in reports and monitoring the display of REI leadership.
{{!}}statistics, REI leadership framework
{{!}}}
Evaluation tools can give further insight into the effectiveness of the training and materials proposed. This will help trainers to adjust training content and delivery methods to improve trainees’ learning experience and outcomes. We propose mixing various tools for collecting learning outputs and adjusting them to the intended target groups (throughout the training guide suggestions are provided on which tools would be most suitable for various target groups).
<references />
}}
{{Instruction Remarks Trainee
|Has Remarks=This project has received funding from the European Union's Horizon Europe research and innovation programme under GA No 101094714 (University of Oslo). UK participants in BEYOND are supported by UKRI grant number 10062742 (Trilateral Research) and by UKRI grant number 10067440 (Heriot-Watt University). Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union, Research Executive Agency, or UKRI. Neither the European Union nor the granting authority nor UKRI can be held responsible for them.)
}}
{{Custom TabContent Close Trainee}}
{{Custom TabContent Trainer Open}}
{{Instruction Steps Foldout Trainer}}
{{Instruction Perspective Trainer
|Is About=The BEYOND Trainer Guide serves as a comprehensive resource for educators seeking to equip and support researchers and research communities with the knowledge, skills, and attitudes necessary to conduct research ethically and with integrity. By fostering critical reflection, ethical decision-making, and responsible behaviour, educators play a critical role in training the next generation of researchers who are committed to the highest ethical standards in their work.

The BEYOND Trainer Guide provides a research-based overview of state-of-the art in teaching research ethics and integrity. According to the extant knowledge base, case-based and collaborative teaching and learning activities, which make use of scaffolding techniques are the best ways to support learning in the context of research ethics and integrity.

This guide provides guidance on how to effectively design teaching activities to foster responsible conduct of research to different target groups, such as students, early career researchers, experienced researchers and supervisors. It draws on prior research on teaching and learning research ethics and integrity and makes use of the vast training resources produced through selected EU-funded projects targeting ethics and integrity. It is a comprehensive resource equipped to empower trainers in delivering impactful training sessions. Trainers can adapt and personalize it based on their specific audience, training context, and personal style. But its core purpose remains clear: to equip trainers with the knowledge, strategies, and resources needed to deliver engaging, informative, and ultimately impactful training sessions. by bringing together research-based knowledge about research ethics and integrity teaching and learning, trainings produced in several EU-funded projects, and templates and activities for adapting materials to various target groups.
|Important Because=Teaching research ethics and research integrity is incredibly important for several key reasons including fostering important skills such as critical thinking, problem-solving, and effective communication.<ref>ALLEA (2023) The European Code of Conduct for Research Integrity – Revised Edition 2023. Berlin. DOI 10.26356/ECOC</ref><ref>Resnik, D. B., & Shamoo, A. E. (2011). The singapore statement on research integrity. Accountability in research, 18(2), 71–75. <nowiki>https://doi.org/10.1080/08989621.2011.557296</nowiki> </ref><ref>Science Europe. (2016). Research Integrity Practices in Science Europe member Organisations. <nowiki>https://scienceeurope.org/our-resources/research-integrity-practices-in-science-europe-member-organisations/</nowiki> </ref> These skills are valuable not only for researchers but also for individuals in various professions and aspects of life.
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Reflect on the goals of RCR education
|Instruction Step Text=The ethical conduct of research is crucial for maintaining the integrity of science. [https://oir.nih.gov/sourcebook/ethical-conduct/responsible-conduct-research-training Responsible Conduct of Research] (RCR) advances scientific goals, fosters a collaborative research environment, and builds public trust in scientific advances that benefit society. Conversely, unethical research practices such as data fabrication and falsification lead to the dissemination of false hypotheses and unreliable data, which harms the search for valid knowledge. Similarly, plagiarism and harassment undermine respect and trust among researchers, while fraudulent or socially irresponsible research weakens public trust and support for science. [https://oir.nih.gov/sourcebook/ethical-conduct/responsible-conduct-research-training The goals of RCR training] include developing a culture of integrity in science and improving knowledge and awareness about the conduct of research.

RCR training and education should be continuous and extend beyond the academic programme throughout a scientist's career. This education can take place in a variety of contexts, such as seminars, workshops, conferences on research ethics and informal mentoring sessions, training courses and laboratory meetings where ethical behaviour and practises are discussed.

As described by van den Hoven and colleagues, multiple factors influence research integrity (RI) training (learning objectives), RI learning (learning outcomes), and changes in RI behaviour (learning outcomes).<ref name=":0">Van Den Hoven, M., Lindemann, T., Zollitsch, L., & Prieß-Buchheit, J. (2023). A Taxonomy for Research Integrity Training: Design, conduct, and improvements in research integrity courses. Science and Engineering Ethics, 29(3). <nowiki>https://doi.org/10.1007/s11948-022-00425-x</nowiki> </ref> Through these, it is possible to promote trustworthy science, responsible research practices, and high integrity/ethical standards. "Training effects" Can be conceptualised through the (intended) impacts of RI training on various performance levels, including individual, institutional, and societal levels.<ref name=":0" /> Furthermore, the authors distinguish among intended training effects (for example changes in behaviour); training input and output (that is training focus/content and learners orientating themselves towards this content); outcomes (that is, learners change their behaviour); and training impact (manifestation of the outcome, such as decreases in misconduct).

[https://nap.nationalacademies.org/read/21896/chapter/15 Effective education in research ethics and integrity aims to achieve several crucial goals.]Defining goals for teaching research integrity and research ethics is crucial to promote and foster responsible research practices and a trustworthy research ecosystem. The main goals to achieve in training RE/RI are related to promoting knowledge (in relation to responsible research practices, norms, and guidelines), skills (in relation to ethical decision-making, problem solving and critical thinking), ‘theoretical’ attitude (in relation to what should be done to foster responsible research) and ‘practical’ behaviour (in relation to how researchers behave in their daily practice).<ref>Kalichman, M. W., & Plemmons, D. K. (2007). Reported goals for responsible conduct of research courses. ''Academic Medicine, 82''(9), 846–852. [https://psycnet.apa.org/doi/10.1097/ACM.0b013e31812f78bf https://doi.org/10.1097/ACM.0b013e31812f78bf]</ref>
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Learn about the approach of this guide
|Instruction Step Text=The BEYOND approach - ‘it’s not the apple, but the orchard’ - reflects the idea that integrity is upheld as a collaborative effort. This is why it is important that training also models the collaborative way. Cases have the capacity to open up discussion space for the complexities of integrity and ethics in research, again, guiding learners to think of the full complexity, not just individuals, but also other systemic levels, including meso and macro levels, that is organisation, research community, and national, international and global context. Scaffolding provides a technique acknowledging where the individual or even a team or research community is at and designing the next steps to facilitate learning and development eventually leading to better alignment with the highest ethical and integrity standards. The point of departure is that there is always room for improvement, even in the strongest of research communities and the work starts with acknowledging status quo and identifying the next goals, which are within reach, irrespective of whether we envision the learning of individuals or communities. With these approaches; case-based and collaborative learning and scaffolding we believe training is well geared towards nurturing the orchard.

The BEYOND Trainer Guide goes beyond simply listing training materials; it adds value by explaining various pedagogical approaches that can be applied to enhance the use of different materials. It shows how learning taxonomies can be applied to create learning-focused training (as opposed to mere information transmission) irrespective of which materials produced in EU-funded projects that are implemented. We have structured the material according to target group, so that trainers can easily identify materials that are suitable for the target group they are training.

Additionally, the content is also structured according to the type of learning activities to support those trainers who wish to work using specific activities but may hesitate whether they are suitable for a particular target group, or simply would like to know more about the activity itself.

To summarise, the BEYOND approach is manifested in the Trainer Guide as:

- A proposal for a research-based approach to an ‘orchard pedagogy’

- Suggestions for measuring training effect to gain an indication of the preparedness of the research community to develop a culture of integrity

Facilitation for using existing RE/RI training resources by providing two alternative structures for trainers, including one, which addresses various actors in ‘the orchard’ through a career-level approach. We wish trainers and other readers, as well as learners taking part in trainings and learning activities utilising the resources referred to in the BEYOND Trainer Guide, a joyful journey through the orchard!
}}
{{Instruction Step Trainer
|Instruction Step Title=Reflect on the goals of RCR education
|Instruction Step Text=The ethical conduct of research is crucial for maintaining the integrity of science. [https://oir.nih.gov/sourcebook/ethical-conduct/responsible-conduct-research-training Responsible Conduct of Research] (RCR) advances scientific goals, fosters a collaborative research environment, and builds public trust in scientific advances that benefit society. Conversely, unethical research practices such as data fabrication and falsification lead to the dissemination of false hypotheses and unreliable data, which harms the search for valid knowledge. Similarly, plagiarism and harassment undermine respect and trust among researchers, while fraudulent or socially irresponsible research weakens public trust and support for science. [https://oir.nih.gov/sourcebook/ethical-conduct/responsible-conduct-research-training The goals of RCR training] include developing a culture of integrity in science and improving knowledge and awareness about the conduct of research.

RCR training and education should be continuous and extend beyond the academic programme throughout a scientist's career. This education can take place in a variety of contexts, such as seminars, workshops, conferences on research ethics and informal mentoring sessions, training courses and laboratory meetings where ethical behaviour and practises are discussed.

As described by van den Hoven and colleagues, multiple factors influence research integrity (RI) training (learning objectives), RI learning (learning outcomes), and changes in RI behaviour (learning outcomes).<ref name=":0">Van Den Hoven, M., Lindemann, T., Zollitsch, L., & Prieß-Buchheit, J. (2023). A Taxonomy for Research Integrity Training: Design, conduct, and improvements in research integrity courses. Science and Engineering Ethics, 29(3). <nowiki>https://doi.org/10.1007/s11948-022-00425-x</nowiki> </ref> Through these, it is possible to promote trustworthy science, responsible research practices, and high integrity/ethical standards. "Training effects" Can be conceptualised through the (intended) impacts of RI training on various performance levels, including individual, institutional, and societal levels.<ref name=":0" /> Furthermore, the authors distinguish among intended training effects (for example changes in behaviour); training input and output (that is training focus/content and learners orientating themselves towards this content); outcomes (that is, learners change their behaviour); and training impact (manifestation of the outcome, such as decreases in misconduct).

[https://nap.nationalacademies.org/read/21896/chapter/15 Effective education in research ethics and integrity aims to achieve several crucial goals.]Defining goals for teaching research integrity and research ethics is crucial to promote and foster responsible research practices and a trustworthy research ecosystem. The main goals to achieve in training RE/RI are related to promoting knowledge (in relation to responsible research practices, norms, and guidelines), skills (in relation to ethical decision-making, problem solving and critical thinking), ‘theoretical’ attitude (in relation to what should be done to foster responsible research) and ‘practical’ behaviour (in relation to how researchers behave in their daily practice).<ref>Kalichman, M. W., & Plemmons, D. K. (2007). Reported goals for responsible conduct of research courses. ''Academic Medicine, 82''(9), 846–852. [https://psycnet.apa.org/doi/10.1097/ACM.0b013e31812f78bf https://doi.org/10.1097/ACM.0b013e31812f78bf]</ref>
<references />
}}
{{Instruction Step Trainer
|Instruction Step Title=Reflect on evaluating effectiveness
|Instruction Step Text=The BEYOND Trainer Guide introduces effectiveness measures to help trainers assess whether the training provided is impactful and beneficial. The versatile evaluation tools (developed in WP4) are designed to be applicable to various target groups and compatible with a variety of training activities and resources. Such evaluation measures are often absent in training resources, yet they provide trainers with a valuable mechanism to ensure how effectively training supports learning. Understanding how training facilitates learning and development is necessary in the process of fostering and strengthening integrity in the research community. Provision of training is a necessary component of the overall building of a culture of integrity. Yet training, the effects of which are not monitored, falls short of its potential to mirror the change it contributes to the research community. Therefore, in the orchard approach, learning and development provides important information about the readiness of the community to build a culture of integrity. Evaluating training effectiveness to ensure training programs achieve their intended outcomes is crucial because it connects training investments to tangible results, ensuring that the effort put into developing and delivering training is worthwhile, and for pinpointing further development needs.

Effectiveness of research ethics and integrity (REI) training can be viewed through an established effectiveness framework, which identifies four outcome domains, namely:

1. reactions (participants’ self-assessment),

2. learning (knowledge, content),

3. behaviour (acting in the research community),

4. results (e.g. institutional outcomes).<ref>Kirkpatrick, D. L. (1959). Techniques for Evaluation Training Programs. Journal of the American Society of Training Directors, 13, 21-26.</ref><ref>Praslova, L. (2010). Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in higher education. ''Educational Assessment, Evaluation and Accountability, 22''(3), 215–225. [https://psycnet.apa.org/doi/10.1007/s11092-010-9098-7 https://doi.org/10.1007/s11092-010-9098-7]</ref>

Evaluating development of ethical competencies should be determined through done as a system to get a more holistic picture. To do this, one can combine different forms of measurement, such as self-assessment and facilitator feedback as well as attitudes and behaviour treats (in tasks that display REI competencies in the research community, like research proposals, ethics sections of theses, articles, etc.). Furthermore, measurement could take place at different time points to gain insight into the learning process, learning outcome, and long-term implications, namely:

• during the training (learning process),

• right after the training – students' and facilitator’s self-reports,

• later as part of another event or course where the display of REI competencies is expected (like RE section in theses and articles, research proposal, evaluation of RE situation in the department, etc.)

It is also important to consider what to do with the results, that is what kind of changes are necessary to improve teaching and/or the environment to build the culture of integrity.

Different tools can be used to collect various learning outputs and analysis instruments can be implemented to analyse the information that has been collected (Table 2) By analysis instruments we mean the taxonomies of learning and application of theoretical models, such as levels of reflection, ethical principles and so on (if data available are mainly in a qualitative format) or statistics and learning analytics (if the data are mainly in quantitative format).
{{{!}} class="wikitable"
{{!}}+Table 1: Tools and analytical instruments for collecting learning outputs in research ethics and integrity training
!Tool for collecting learning outputs
!Details
!Analysis instrument
{{!}}-
{{!}}'''''ProLearning'' app'''
{{!}}''ProLearning'': [http://www.prolearning.realto.ch/ www.prolearning.realto.ch]
{{!}}learning analytics
{{!}}-
{{!}}'''Engagement app'''
{{!}}App under development, [https://forms.office.com/Pages/ShareFormPage.aspx?id=WXWumNwQiEKOLkWT5i_j7twYn7PlpvpDlgGDpz2LgIdUMk5XRTVYQTVKRFRDWDlHOUdGU1FHTUlFVi4u&sharetoken=03epmvYBRpmfXvpRg9os form] (for copying and editing)
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Self-Reflection Form/Compass'''
{{!}}App under development, [https://forms.office.com/Pages/ShareFormPage.aspx?id=WXWumNwQiEKOLkWT5i_j7twYn7PlpvpDlgGDpz2LgIdUMk5XRTVYQTVKRFRDWDlHOUdGU1FHTUlFVi4u&sharetoken=03epmvYBRpmfXvpRg9os form] (for copying and editing)
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Pre-post texts'''
{{!}}Collect a short text (e.g. a response to a case or short essay) before the training and after the training
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Learning diaries'''
{{!}}Ask learners keep a diary over a certain period, for each submission provide some guiding questions or topics
{{!}}SOLO taxonomy, reflection levels, content criteria
{{!}}-
{{!}}'''Group reports'''
{{!}}Ask groups working together to provide a (short) group report (or provide a template with points to work on)
{{!}}SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Group discussions'''
{{!}}Monitor the group discussions to evaluate the level of understanding and content discussed (scaffold as appropriate)
{{!}}SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Group dynamics'''
{{!}}''CoTrack'' application: https://www.cotrack.website/en/
{{!}}learning analytics
{{!}}-
{{!}}'''Online learning platform'''
{{!}}Make use of accumulated authentic learning outputs in the learning platform.
{{!}}statistics, SOLO taxonomy, reflection scale, content criteria
{{!}}-
{{!}}'''Domain-specific/ domain-transcending measure'''
{{!}}Use either of the two forms (WP4.2) measuring recognition and exemplifying of ethical issues.
{{!}}statistics, SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Retention check'''
{{!}}After a certain time (few weeks/months) ask learners to provide a short text (analysis of a case, short essay on an ethics topic/question). Compare the levels of understanding to another piece collected during or right after the training.
{{!}}SOLO taxonomy, content criteria
{{!}}-
{{!}}'''Vignettes'''
{{!}}This can be used for measuring ethical sensitivity in (non-)training context
{{!}}statistics, EASM (based on the SOLO taxonomy), content criteria
{{!}}-
{{!}}'''National surveys'''
{{!}}Can be used for analysing training-related content in reports and monitoring the display of REI leadership.
{{!}}statistics, REI leadership framework
{{!}}}
Evaluation tools can give further insight into the effectiveness of the training and materials proposed. This will help trainers to adjust training content and delivery methods to improve trainees’ learning experience and outcomes. We propose mixing various tools for collecting learning outputs and adjusting them to the intended target groups (thoughout the trainig guide suggestions are provided on which tools would be most suitable for various target groups).
<references />
}}
{{Instruction Remarks Trainer
|Has Remarks=This project has received funding from the European Union's Horizon Europe research and innovation programme under GA No 101094714 (University of Oslo). UK participants in BEYOND are supported by UKRI grant number 10062742 (Trilateral Research) and by UKRI grant number 10067440 (Heriot-Watt University). Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union, Research Executive Agency, or UKRI. Neither the European Union nor the granting authority nor UKRI can be held responsible for them.)
}}
{{Custom TabContent Close Trainer}}
{{Related To
|Related To Instruction=Instruction:02a025d1-fb65-40b4-914f-5efed84ee112; Instruction:238dfd77-9a59-4dec-b498-3e2214ddc1fe; Instruction:55649946-f15f-4c94-89e1-b79dff231c0e; Instruction:30f33e85-09ed-4ea7-9ac2-7fb26d27ae55; Instruction:126b8806-0cc1-4625-b11b-f60f43ad9f6f; Instruction:Bcec25e1-c0d9-4e1d-b5c3-eb9c13ae3618; Instruction:204c2375-0867-4302-845f-cdc99e3d38bc; Instruction:Eeee007a-14ec-4ffa-b62e-514aafe4bd1a; Instruction:44eb9553-4c0a-40c9-9ed1-50b4f591e3fb; Instruction:4afd5ea7-0308-4e80-9dfc-6b66daf2ac81; Instruction:Ee61dfb7-737f-40ff-9568-aa669cd6865c
}}
{{Tags}}

Guide:D857e22a-342c-48b7-a1d5-a096a2fa1ebd

2025-01-31T10:09:26Z

0000-0001-7581-7849:

{{Guide
|Title=BEYOND Trainer Guide
|Has Related Initiative=Initiative:667449ef-1c0a-4e06-8374-830c8ad68cb8
|Guide Goal=The BEYOND Trainer Guide is a solid resource '''for educators''' who want to cultivate a culture of integrity and ethical behaviour in research by fostering responsible conduct of research. It provides a detailed framework that gives trainers the knowledge and practical tools they need to teach researchers the principles of responsible conduct of research. At its core, the guide provides a structured approach to delivering sessions that engage participants in discussions, activities and case studies.

These elements have been carefully crafted to stimulate deep engagement with the ethical dimensions inherent in every stage of the research process, from conception to dissemination of results, and at all career stages. Trainers are provided with tools that promote a culture of accountability and integrity and encourage researchers to uphold ethical principles not only in their research endeavours, but also in their interactions with colleagues, participants, and the broader academic community.

In essence, the guide serves as a '''comprehensive resource for educators''' seeking to equip researchers with the knowledge, skills, and attitudes necessary to conduct research ethically and with integrity. By fostering critical reflection, ethical decision-making, and responsible behaviour, educators play a critical role in training the next generation of researchers who are committed to the highest ethical standards in their work.

<div class="video-button" data-href="https://www.youtube.com/embed/LWjoBVl7raY">
The BEYOND Trainer Guide
2:46 min
</div>
}}
{{Custom TabContent Trainee Open}}
{{Guide List Wrapper Trainee Open}}
{{Guide List Wrapper Trainee Close}}
{{Custom TabContent Close Trainee}}
{{Custom TabContent Trainer Open}}
{{Guide List Wrapper Trainer Open}}
{{Guide Step Trainer
|Guide Step Type Trainer=Section
|Guide Step Section=Introduction
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Instruction
|Guide Step Element=Instruction:1c5bdda7-6444-497b-aaf2-53852a0fd8ee
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Section
|Guide Step Section=Define the learning objectives and teaching methods
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Instruction
|Guide Step Element=Instruction:4afd5ea7-0308-4e80-9dfc-6b66daf2ac81
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Instruction
|Guide Step Element=Instruction:Eeee007a-14ec-4ffa-b62e-514aafe4bd1a
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Section
|Guide Step Section=Learn About Existing Training Materials Used in this Guide
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Instruction
|Guide Step Element=Selecting Appropriate Material and Effectiveness Measurement Tools for your Target Audience
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Section
|Guide Step Section=Build your own training by learning about available resources
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Instruction
|Guide Step Element=Instruction:44eb9553-4c0a-40c9-9ed1-50b4f591e3fb
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Instruction
|Guide Step Element=Instruction:30f33e85-09ed-4ea7-9ac2-7fb26d27ae55
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Instruction
|Guide Step Element=Instruction:02a025d1-fb65-40b4-914f-5efed84ee112
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Instruction
|Guide Step Element=Instruction:Ee61dfb7-737f-40ff-9568-aa669cd6865c
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Instruction
|Guide Step Element=Instruction:126b8806-0cc1-4625-b11b-f60f43ad9f6f
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Instruction
|Guide Step Element=Instruction:238dfd77-9a59-4dec-b498-3e2214ddc1fe
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Instruction
|Guide Step Element=Instruction:55649946-f15f-4c94-89e1-b79dff231c0e
}}
{{Guide Step Trainer
|Guide Step Type Trainer=Instruction
|Guide Step Element=Instruction:Bcec25e1-c0d9-4e1d-b5c3-eb9c13ae3618
}}
{{Guide List Wrapper Trainer Close}}
{{Custom TabContent Close Trainer}}

Resource:31c71a72-eaef-4287-8df6-9788d07302fb

2021-10-13T11:24:01Z

0000-0001-7581-7849:

{{Resource
|Resource Type=Education
|Title=ENERI Decision Tree
|Is About=The ENERI Decision Tree is an online tool intended to help researchers, members of research ethics committees (RECs) and research integrity officers to anticipate, reflect and address ethical questions and challenges that might arise before, during or after a research project. Thus, the ENERI Decision Tree aims to facilitate responsible conduct of research throughout all phases of the research process. Moreover, it seeks to support the work of RECs and research integrity offices (RIOs) by providing guidance on how to respond to research ethics and research integrity challenges.
|Important Because=New and emerging technologies as well as the globalization of research and the rise of multi-center studies, to name just a few, have brought numerous challenges in terms of research ethics and research integrity. Based to a large extent on the [[Resource:F47b9bc7-c5a5-4b92-918b-438101bd9434|ENERI Research Ethics and Research Integrity Manual]], the Decision Tree guides researchers as well as members of RECs and RIOs through several of these challenges and provides them with tools to conduct research ethically and with integrity. More specifically, the Decision Tree includes summaries of and links to laws, guidelines, codes and other pertinent references. In this way, it covers the international, European and national levels, all of which researchers, RECs and RIOs usually need to consider.

The ENERI Decision Tree is based on three premises:

1) Good researchers should reflect on and respond to ethical issues and challenges before, during and after conducting their research

2) RECs should help researchers in doing good research

3) RIOs should assist researchers to monitor their research

The information in the Decision Tree is structured around the following topics:

'''Responsibility in research'''

*Research as a social practice
*The legal framework of research

'''Planning of the research'''

*Cross-national and international multi central research
*Responsibility in authorship
*Research with human participants: general provisions
*Research with animals
*Research in biotechnology
*Research in engineering, AI and robotics
*Research in biotechnology for agricultural and food purposes (outside of the biomedical sector)
*Research on human remains
*Study design and objectives, avoiding bias
*The role of funders
*Research with personal data

'''The actual research process'''

*Research with humans in biomedical research
*Research with human tissues/cells
*Research with embryonic stem cells, embryos, fetal tissues
*Research with samples and data taken from human biobanks
*Research with human participants in psychology
*Research with human participants - qualitative research
*Research with human beings in implementing technology/devices
*Research on the environment
*Minimal disturbance to the integrity of nature
*Monitoring animal welfare
*Making uncertainties and value assumptions explicit
*Dealing adequately with big data and complexity

'''Quality assurance and dissemination'''

*Sharing results in the scientific community, with the public and with stakeholders
*Mechanisms for quality assurance
*Were the methods and tools adequate for the claimed result? ''(under development)''
*Publication as public knowledge ''(under development)''
*Open science or restricted access ''(under development)''
*Stakeholder consultations ''(under development)''

'''Applications and monitoring'''

*Dual use and misuse
*Evaluation of success and failure ''(under development)''
*Consultation with beneficiaries and stakeholders ''(under development)''
*Assess necessity of retractions ''(under development)''
*Re-start the research afresh ''(under development)''

Each topic is a self-contained unit so that users can easily find tailored information to specific questions without having to read the whole Decision Tree. Like the ENERI Manual on Research Ethics and Research Integrity, the Decision Tree is a living document and will thus be updated periodically to account for new developments in research ethics and research integrity processes and policies.
|Important For=All stakeholders in research; Research Integrity Officers; Research integrity trainers; research integrity researchers; Ethics committee members; Research Ethics Committees; Researchers
|Has Best Practice=The ENERI Decision Tree summarizes and links to many important laws, regulations, codes and other documents that can help researchers to work ethically and with integrity and that can support RECs and RIOs in performing their roles adequately and fulfilling their responsibilities. More detailed information on all topics covered in the Decision Tree is available in the ENERI Manual on Research Ethics and Research Integrity. Besides, the [[Resource:C386dbba-2f69-4257-89c2-903898cf1f12|ENERI Classroom]] as well as the [[Guide:Bbe860a3-56a9-45f7-b787-031689729e52|VIRT2UE Training Guide]] provide access to educational materials on research ethics and research integrity that help fostering skills conducive to ethical reflection.

Furthermore, the [https://embassy.science/wiki-wiki/index.php/Special:BrowseData/Resource?_search_Resource_Type%5B0%5D=Cases cases] in the resources section of the Embassy as well as the educational scenarios developed by EnTIRE that are available in the [https://embassy.science/wiki-wiki/index.php/Special:BrowseData/Resource?_search_Resource_Type%5B0%5D=Education educational resources] section can be used for further reflections and deliberations on specific research ethics and research integrity problems.
}}
{{Link
|Has Link=https://eneri.eu/decision-tree/
}}
{{Related To
|Related To Resource=Resource:C386dbba-2f69-4257-89c2-903898cf1f12;Resource:777888d6-6152-44ce-bd15-646bb279e521;Resource:F6100097-fddb-4c77-9098-1bc767c34a6a;Resource:1d26fd13-1ced-44bc-8d19-e094b37f8f70;Resource:45a04c31-5a75-4816-8484-2dd9b71d1674;Resource:Aef6b98d-9cc5-4db0-bffd-4a3daa99a3f3;Resource:C99f17ec-3d1e-4f7a-bfc7-3e3607934ead;Resource:7f7810d8-74a2-42ac-906c-7f6a73fcd183;Resource:E99e20d0-8116-4d77-84ec-7df396703bf4;Resource:67caae86-68db-49ea-8305-2010fe701aa6
|Related To Theme=Theme:8c79e235-8481-45ea-bb57-c744dedbbb8a;Theme:B4f3369c-e0ac-4cf5-acd9-cb2a6c11181d;Theme:65e6f304-51e2-4e41-93d3-e48518248b39;Theme:D1477512-52a3-48a3-8ab6-72404cef4ab4;Theme:9cc6d88e-a142-4741-834f-5d6aa7d06e3f;Theme:9ac8c1db-f98b-41ee-858d-a8c93a647108;Theme:Cc85bbe7-b8ac-40ef-81a7-8e34b153233c
|Related To Instruction=Instruction:A440eed0-f9f4-4415-a2c4-2d6ff9f44b80;Instruction:6b129846-c455-4849-9eaf-0d25f3c5600e;Instruction:41bc2a1d-26f7-49f9-8bf7-9fc6b4ecf10c;Instruction:C0cf8cfb-6090-49e3-94f5-20f530f83ffd;Instruction:6cc77174-4f7b-48a6-95f3-eeb4dadcb0a3;Instruction:Ffff98bc-b81b-43ee-8fef-a264c1e25741;Instruction:379346fc-d254-467f-99be-41b1cea1c817
}}
{{Tags
|Involves=ENERI; EUREC; ENRIO; ALLEA; VIRT2UE; EnTIRE
}}

User:0000-0001-7581-7849

2021-10-13T10:00:58Z

0000-0001-7581-7849: create user page