Instruction:8e436b3a-28b3-4db2-8a9a-dc7632ceb4c0

Instructions

Quality of research outputs and data sets

Instructions for:TraineeTrainer

Related Initiative

ROSiE

Goal

After completing this module learners will be able to understand the importance of the quality of data sets and research outputs and their responsible use in open science.

Duration (hours)

2

For whom is this important?

All stakeholders in researchCitizen Scientists

Method

Blended learning Individual learning

Part of

ROSiE

Steps

1Introduction

2Learning about Key Issues

3Test your Knowledge

4Case Study

What is this about?

Citizen science, according to the European Citizen Science Association (ESCA), is "an ‘umbrella’ term that describes a variety of ways in which the public participates in science. The main characteristics are that: (1) citizens are actively involved in research, in partnership or collaboration with scientists or professionals; and (2) there is a genuine outcome, such as new scientific knowledge, conservation action or policy change."

Why is this important?

Currently, citizen science is becoming more and more important in different fields of science. For example, in natural sciences, it enables large-scale data collection by involving a vast number of individuals which would be challenging to achieve for traditional research methods within the same timeframe and resources. This training will guide you through the crucial elements of responsible citizen science, including protection of human research participants, plants, animals and ecosystems; rights of citizen scientists; conflicts of interest; quality of research outputs etc. By the end of this training, you will gain a deeper understanding of responsible open science and acquire the following skills and attitudes necessary for responsible practising of citizen science.

Practical Tips

This module is part of the ROSiE training on Responsible Open Science for Citizen Scientists. To complete the trainig please complete the other modules in the citizen science guide.

By the end of this training, you will gain a deeper understanding of responsible open science and acquire the following skills and attitudes necessary for responsible practising of citizen science:

Local and Global Citizenship:

• Awareness of the importance and social benefits of Open Science in local and global contexts.
• Participation in ethics and integrity self-regulation of Open Science and citizen science community.

Personal and Social Responsibility:

• Personal and professional responsibility for implementation of Open Science and production of results.
• Openness to share own research data, results, tools and publications and appreciation of efforts of others.

Epistemic Skills

• Ability to organize, present and use open data and knowledge with integrity.
• Ability to critically assess data, knowledge and scientific results produced by others.
• Ability to identify ethical and integrity issues in Open Science.

Collaborative Problem-Solving

• Ability to apply critical thinking skills in collaborative analysis of ethical and integrity problems in Open Science.
• Discussing, finding solutions and making decisions to handle ethics and integrity issues within the Open Science community.

1

Introduction

The collection of research data is arguably one of the most challenging aspects of open science practice because it is highly vulnerable to misconduct (Hofmann, 2022). Misconduct related to data collection can be particularly costly to science and society, especially when data is shared open access for reuse and re-analysis. Therefore, it is crucial to ensure that both researchers and citizen scientists share an understanding of data collection standards. These standards ensure that data collected at different institutions and by various researchers are compatible and interoperable, facilitating the integration of datasets. This allows for meaningful reuse, comparisons, re-analysis, and the reproduction of research findings by other scientists. Adherence to data collection standards also contributes to the long-term accessibility of research data. Proper documentation and standardized formats make it easier for future researchers to understand, use, and build upon the data, thereby preserving the scientific record.

References

1. Hofmann, B. (2022). Open science knowledge production: Addressing epistemological challenges and ethical implications. Publications, 10(3), 24. https://doi.org/10.3390/publications10030024

2

Learning about Key Issues

Citizen science projects collect and share diverse types of data. As pointed out by Balázs et al.: "Some projects are solely quantitative data projects, while others are solely qualitative. Mixed-method citizen science projects also exist which include both quantitative and qualitative data collection, generation, and manipulation." (Balázs et al., 2021) Due to this variety of data and other reasons, data quality in citizen science encounters various challenges that can impact the reliability and usability of the collected information. For example, analysis of the data collected by iNaturalist project revealed that the data suffers from various kinds of biases, for example, towards certain taxa (such as birds, plants, and mammals). Also, there is some evidence of spatial sampling bias. For example, about 58% of all threatened species observations in iNaturalist come from the U.S., Canada, Mexico, Russia and New Zealand (Soroye et al., 2022).

Balázs et al. point out the two main aspects of data quality in citizen science - reliability and validity. Reliability refers to the stability and consistency of data over time. In the context of citizen science, reliable data means that results can be replicated consistently. (Balázs et al., 2021) For example, in a project tracking water quality in a river, if different citizen scientists using the same measurement tools consistently report similar results for the same water samples, the data is deemed reliable. Validity in data refers to the extent to which the data accurately represents what it is supposed to measure or describe. For example, in a citizen science project on weather monitoring, if citizen scientists consistently report all relevant weather parameters (temperature, humidity, precipitation), the data is valid as it provides a comprehensive view of weather conditions.

Data contextualization refers to the practice of providing essential context and information surrounding a dataset, enabling a better understanding of how the data was generated, its purpose, and its quality. It includes metadata, attribution, and curation details to situate the data within its broader context. (Balázs et al., 2021) For example, in a climate monitoring citizen science project, metadata could include details about the creation of data set, contributors, methodology, instruments used, calibration procedures, and the temporal and spatial resolution of data. Metadata enhances the understanding and usability of the data.

Four aspects of data accuracy in citizen science. Balázs B. et al. https://doi.org/10.1007/978-3-030-58278-4_8, CC BY 4.0

References

1. Balázs, B., Mooney, P., Nováková, E., Bastin, L., Jokar Arsanjani, J. (2021). Data Quality in Citizen Science. In: The Science of Citizen Science. Springer https://doi.org/10.1007/978-3-030-58278-4_8
2. Soroye, P. et al. (2022). The risks and rewards of community science for threatened species monitoring. Conservation Science and Practice, 4(9), e12788. https://doi.org/10.1111/csp2.12788

3

Test your Knowledge

Use the flashcards below to test your knowledge!

ROSiE Test: Quality of data

4

Case Study

Read the slides below and complete the interactive exercises!

ROSiE Case Study: Quality of research outputs and data sets

Remarks

Additional Resources:

1. Balázs, B., Mooney, P., Nováková, E., Bastin, L., Jokar Arsanjani, J. (2021). Data Quality in Citizen Science. In: The Science of Citizen Science. Springer https://doi.org/10.1007/978-3-030-58278-4_8
2. Herodotou, C., Scanlon, E., & Sharples, M. (2021). Methods of promoting learning and data quality in citizen and Community Science. Frontiers in Climate, 53. https://doi.org/10.3389/fclim.2021.614567

Steps

Other information

Good Practices & Misconduct

• Open science