Welcome to the Reproducible Experiments (REPETO) resource page! Here, we aim to provide researchers and enthusiasts with a curated collection of known reproducible experiments from various sources. Whether you are participating in a hackathon, conducting research, or simply interested in exploring reproducible experiments, this page will be your go-to resource for discovering verified and reliable experiments.
Contents
Experiment Suggestions for Reproducibility Hackathon on Dec. 15!
Explore our comprehensive guide for the Virtual Reproducibility Hackathon, focusing on cutting-edge experiments and methodologies in various domains of computer science. This resource is curated to aid researchers, academicians, and students in replicating and understanding significant recent works in the field.
Machine Learning: Delve into experimental patterns designed to enhance reproducibility in Machine Learning. This includes connecting Google Colab to a server on Chameleon, enabling the use of robust computational resources while maintaining a familiar working environment. The section also showcases replicated experiments from pivotal ML research, offering insights into practical applications and methodologies.
Networking: Addressing the intricacies of network research, this section presents experimental patterns like network emulation on platforms like Chameleon and FABRIC. It includes replicated studies from key networking papers, emphasizing the adaptability and robustness of various network conditions.
Each section is enriched with detailed artifacts and descriptions, replicating results from notable papers and studies in the respective fields. The suggestions and examples serve as a valuable resource for fostering innovation and promoting reproducibility in computer science research.
ML Reproducibility Challenge
The ML community has been very active in fostering a reproducibility culture and there is a good repository of papers from each year here
European Conference on Information Retrieval (ECIR)
The ECIR is dedicated to promoting excellence in information retrieval research and fostering a culture of research reproducibility. We present a collection of four artifacts from the 2022 ECIR conference.
- Accelerating Parallel Operation for Compacting Selected Elements on GPUs
- MARTINI: The Little Match and Replace Tool for Automatic Code Rewriting
- Mallob: Scalable SAT Solving On Demand With Decentralized Job Scheduling
- SimSGamE : Scheduling simulator for modern game engines
Materials from the International Conference on Parallel Processing (2023)
The International Conference on Parallel Processing (ICPP) recognizes the importance of fostering and enabling reproducible research and it is also committed to raising standards and good practices for reproducibility. For that reason, ICPP is converging with other similar reproducibility initiatives such as the IEEE Transactions on Parallel and Distributed Computing Journal (TPDS) and The International Conference for High Performance (SC’23) reproducibility initiatives, while maintaining the inherent characteristics of the conference.
