Instruction:4bc3edc0-d493-4f35-a742-e4cee5569562

Modules

New Tools and Services for Reproducibility

Instructions for:TraineeTrainer

Related Initiative

TIER2

Goal

TIER2 aims to better understand the causes, consequences and possible solutions of perceived poor levels of reproductivity of research across research contexts. With a focus on social, life, and computer sciences, as well as research publishers and funders, the project aims to increase awareness, build capacity, and propose innovative solutions sensitive to varied research cultures. Central to its approach are eight Pilot activities designed to develop, implement, and evaluate new reproducibility-related tools and practices.

Duration (hours)

1

For whom is this important?

All stakeholders in research

Steps

1Pilot 1 - Decision Aid

2Pilot 2 - Reproducibility Management Plan (RMP)

3Pilot 3 - Reproducible Workflows

4Pilot 4 - The Reproducibility Checklist

5Pilot 5 - The Reproducibility Promotion Plan for Funders (RPP)

6Pilot 6 - The Reproducibility Monitoring Dashboard

7Pilot 7 - Editorial Workflows to Increase Data Sharing

8Pilot 8- An Editorial Reference Handbook for Reproducibility and FAIRness

1

Pilot 1 - Decision Aid

TIER2's Decision Aid provides clarity on the meaning, relevance, and feasibility of ‘reproducibility’ for researchers to aid them in identifying what type of reproducibility is relevant for their research and indicate what they must consider regarding how feasible such ‘reproducibility’ would be for them.

2

Pilot 2 - Reproducibility Management Plan (RMP)

The Reproducibility Management Plan (RMP) Pilot aims to create a prototype of key thematic subjects and questions that will serve as the starting point to support reproducibility at the planning stage of research. Work involves defining what an RMP is, integrating it into the ARGOS service, and testing its effectiveness with feedback from the community. The pilot addresses researchers, beneficiaries and funders for its adoption.

3

Pilot 3 - Reproducible Workflows

Reproducible Workflows has adapted the SCHEMA open-source platform for reproducible workflows in life and computer sciences by leveraging software containerisation, workflow description languages (CWL, Snakemake), and experiment packaging specifications (RO-crate), particularly emphasising machine learning in computer science.

4

Pilot 4 - The Reproducibility Checklist

The Reproducibility Checklist for Computational Social Science Research provides a simple, structured framework to enhance the reproducibility of computational methods shared on Methods Hub. Designed for minimal effort and maximum usability, the checklist helps researchers document and share their data and code in a consistent, verifiable way that fosters trust and collaboration within the social science community. This pilot contributes to more efficient, transparent, and reproducible computational social science by integrating practical usability with clear documentation standards.

The checklist tempelate is publicly available through GitHub project repositoriy and is accompanied by user guidance for researchers who want to submit a method. These resources explain how to apply the checklist in practice, both independently and in connection with Methods Hub.

5

Pilot 5 - The Reproducibility Promotion Plan for Funders (RPP)

The Reproducibility Promotion Plan for Funders (RPP) has developed a policy template with recommendations for funders to foster reproducible practices both in the research they fund across three key areas of funding work: evaluation and monitoring, policy and definitions, and incentives. The RPP provides actionable recommendations and best practice examples that funders and funding institutions can adapt to meet their specific needs.

All drafts of the RPP can be found on OSF.

6

Pilot 6 - The Reproducibility Monitoring Dashboard

The Reproducibility Monitoring Dashboard hosts tools that enable funding agencies to track and monitor the reusability of research artifacts across various projects, programs, topics, and disciplines. This auto-generated dashboard assesses the impacts of policies related to data and code sharing.  

7

Pilot 7 - Editorial Workflows to Increase Data Sharing

This tool is aimed at increasing data sharing in published work. Data sharing is an important building block for increased reproducibility & transparency, but current rates of sharing are low.

8

Pilot 8- An Editorial Reference Handbook for Reproducibility and FAIRness

The Editorial Reference Handbook informs and assists journals to operationalise a set of checks necessary to make the data underlying published research findings more findable, accessible, interoperable and reusable (FAIR) to underpin reproducibility. The Handbook targets in-house staff managing the manuscripts primarily, but it will also benefit reviewers, authors and even those providing services to publishers by making the fundamental checks and requirements transparent and understandable.

The Handbook includes three interlinked components: a checklist of 12 elements, a flowchart that outlines the ideal internal manuscript submission workflow (who should perform the checks and when), and a guidance to help users to implement the checks effectively, with definitions and tips.  

The Handbook fills a gap, because no common guidance existed on the practical implementation of these checks across a complex publishing workflow and the variety of individuals and teams involved. This pilot saw the participation of more than 30 individuals, from more than 20 journals of major publishers: CUP, Cell Press, EMBO Press, Taylor & Francis, GigaScience Press, OUP, PLOS, Springer Nature.

Steps

Other information