Inaugural Summer of Reproducibility (SoR’23) Gets Underway

The REPETO team recently launched the first Summer of Reproducibility (SoR) — a mentorship-centered program aimed at making computational research efforts reproducible. In this inaugural year, we enabled the matching of 19 SoR Fellows (primarily undergraduate and graduate students) from around the world with 21 reproducibility focused mentors. The REPETO team created the SoR in order to promote the creation of reproducibility artifacts that are practical and accessible to a wider population of researchers and students.  The SoR provides students (and other newcomers to reproducibility) the opportunity to work on this effort and acquire relevant, valuable skills.

The Summer of Reproducibility is managed under the REPETO team members at the UC Santa Cruz under their Open Source Research Experience (OSRE) program, which follows the model set forth by programs like the Google Summer of Code and Outreachy. In a similar way that those long-standing programs bring more contributors into open source projects, the SoR pairs faculty and other researchers with interested students working on summer projects to increase the accessibility and capacity around reproducible artifacts. The opening of the application process in January 2023 saw our mentors create “project ideas” that students use as a basis for their project proposals. The proposal process, which spanned February and March, was iterative, giving mentors an opportunity to provide feedback and assess if the applicant is a good match for proposed work. The deadline for student applications was early April; we received 38 proposals from students all over the world, for which 19 were selected for funding.  Students are provided stipends for their summer projects, are encouraged to blog about their work throughout the summer, and provide and receive evaluations and feedback throughout the project period. At the end of the summer, successfully completed projects will be asked to report what was achieved over the summer through blog posts, conference posters, and lightning talks. 

Quote from recent student mid-term evaluation: “My experience so far has been great! My mentor always guides me through when I meet obstacles and gives me instant feedback as needed. I learned a lot not only about the research materials but also about how to become a more organized person….. I also learnt how to write more structural reports that can help the audience to understand the progress much better. These skills aren’t taught in class yet are really helpful. I also learnt a couple of quite useful technical skills along the way that could generate better research results.

2023 SoR Fellows

Fellows Name Project Title
Chandra Shekhar Pandey Interactive open educational resources for machine learning courses
Charis Christopher Hulu Predict Genomics Workflow Execution Time using Two-Stage Approach
Eunsoo Justin Shin Leveraging ML-augmented I/O in Linux
Faishal Zharfan Reproduce and benchmark self-adaptive edge applications under dynamic resource management
Goodness Ay ScaleBugs: Reproducible Scalability Bugs
Haoran Wu GPU Emulator for Easy Reproducibility of DNN Training
Jesse Lima Verify the reproducibility of an experiment in noWorkflow
Jiayuan Zhu DataVizFlow: A Record and Visualize Experimental Results Platform
Jonathan Edwin Using Reproducibility in Machine Learning Education: A Study on Cutout, U-Net, and Siamese Networks
Kangrui Wang Automatic Cluster Performance Shifts Detection Toolkit
Krishna Madhwani Public Artifact Data and Visualization (Experiment Log & Record and visualize experimental results)
Luiza Zucchi Hesketh Advancing Reproducible Science through Open Source Laboratory Protocols as Software
Maharani Ayu Putri Irawan FlashNet: Towards Reproducible Data Science for Storage System
Mohamed Saeed Using Reproducibility in Machine Learning Education: Levels of reproduction
Shayantan Banerjee Reproducible Analysis & Models for Predicting Genomics Workflow Execution Time
Srishti Jaiswal Teaching Computer Networks with Reproducible Research: Classroom competition for adaptive video
Xueyuan Ren Measuring Open-source Database Systems under TPC-C with Unreported Settings
Zahra Nabila ScaleBugs: Reproducible Scalability Bugs
Zhiyan (Alex) Wang Reproducible Evaluation of Multi-level Erasure Coding


The SoR organizers are looking forward to SoR 2024, and will be posting more here soon on how to become a mentor or fellow in next year’s program.

 

Leave a Reply

Your email address will not be published. Required fields are marked *