The REPETO Badge is assigned to artifacts on Chameleon Trovi to distinguish practically reproducible experiments from other artifacts on the hub. Adding the badge to your artifact heightens the visibility of your research. Other researchers can search for REPETO badges on Trovi to find reproducible experiments relevant to their work, replicate them, and modify them as the starting point for further research.

This document summarizes our guidelines for researchers who are developing REPETO artifacts for Trovi. We ask that you adhere to these guidelines if you plan to develop reproducible artifacts. You can then apply for a REPETO badge on Trovi by completing this form. See our resource page for tips and examples of sound experiment packaging here.

Overview

A REPETO badge simply means that the author of the experiment followed a few simple documentation rules that will make it easier for experiment reviewers to identify whether their experiment is relevant to them or not. It does not signify reproducibility status (i.e., that anybody verified that the experiment can be reproduced beyond the author’s attestation) or support status – though the rules encourage authors to document both.

A Trovi artifact is composed of a preamble (artifact long description) and artifact body (somewhere inside the artifact). The following documentation is required in the preamble:

Artifact description [required]:

One to two sentences describing roughly what the artifact does, e.g., “This artifact represents a power management experiment.”

Paper [required]:

What paper does this artifact reproduce? (citation + link)

Estimated time to reproduce [required]:

How much time, roughly, should the reviewer allocate to reproduce the experiment?

Reproducibility status [required]:

How has the reproducibility of this experiment been verified? Has it been awarded a badge? Has it been evaluated as part of a reproducibility initiative? Has it been verified by a colleague?

DOI [optional]:

A DOI if available (via e.g., Zenodo) can help the experiment reviewer cite the experiment in their papers.

Testbed [optional]:

Which testbed does this experiment run on?

Support [required]:

Status: <fully supported, best effort>, <email for support>

Experiment Pattern [optional]:

One to two sentences describing the experiment pattern that the artifact implements, e.g., “This experiment creates a multi-node MPI cluster.” Some experiment reviewers may be interested in the experimental containers as well as the experiments themselves and use the experiment for that reason.

Additionally, in the artifact body the author should clearly explain the “reproducibility condition”, i.e., provide recommendations on how to verify the reproducibility condition, e.g., observe a trend on a graph, data values in a certain interval, or of certain statistical properties.

Additional/optional style recommendations:

Where possible, experiments packaged for reproducibility should clearly identify experiment stages, i.e., container creation, experiment body, and data analysis, as different parts of an experiment may be of interest to different reviewers.

Submit your artifact for review here.