The REPETO (pronounce to rhyme with Geppetto) project is an NSF-funded research coordination network (RCN) that promotes the concept of practical reproducibility, i.e., a practice where many – or even most – experiments are packaged in such a way that they can be repeated cost-effectively and thus become a means of exploring research on a par with what reading papers is today. We focus on experiments in the computer science community and seek to understand the cost/benefit equation of reproducibility for these types of experiments and the factors that make their reproducibility difficult or infeasible.
The project fosters community practices to make reproducibility a part of mainstream research and education activities in computer science. We will develop a range of activities and artifacts including teaching methodology for packaging experiments for cost-effective replication; using reproducibility in teaching; collaboration with reproducibility initiatives sponsored through conferences and institutions; community events emphasizing repeating or replicating experiments such as hackathons, competitions, or rankings; fostering repositories of replicable experiments and monitoring their usage/replication; as well as reporting on state of art and emergent requirements for the support of practical reproducibility.
The outcome of the project will be a collection of computer science experiments replicable on open platforms, an understanding of how much and to what extent they are used in mainstream research and education activities via relevant metrics, as well as a series of reports on current enablers and obstacles towards mainstream use of reproducibility in computer science research.