Matching reproducibility researchers with summer students and sponsors
The Summer of Reproducibility (SoR) is a program of UC Santa Cruz’s 2024 Open Source Research Experience. The SoR program focuses on research projects to produce and use reproducibility artifacts (see also ctuning.org and sysartifacts).
*We are currently accepting project ideas until Feb. 22, 2024 (extended from Feb. 6, 2024). See the important links below for instructions on how to submit a project idea.*
Contents
Important Links
- General Information for OSRE Mentors
- Instructions on Submitting Project Ideas via Git (tag your project “reproducibility”) OR via Google Docs (share with slieggi@ucsc.edu) (NOTE: the submission process is the same whether you are submitting a project for SoR or OSRE more generally)
- 2024 Timeline
- FAQs for Mentors (we encourage you to review these FAQs before submitting your project to ensure you are including all pertinent information in the project submission)
- Current Reproducibility Projects (2024) and Past Reproducibility Projects (2023) (after going to the link, you can view the reproducibility projects by selecting the “reproducibility” filter button above the project descriptions)
Timeline
Important Dates
Feb. 22, 2024 – Deadline for mentors to submit their project ideas to be listed on OSRE’s website.
Feb. 22 – Apr. 2, 2024 – Students begin preparing proposals for project ideas listed on OSRE’s website. Mentors help student candidates improve their applications.
Mar. 20, 2024 – Deadline for students to join Slack channel (a student proposal requirement – do so as early as possible).
Apr. 2, 2024 – Student proposal deadline.
Apr. 3 – 22, 2024 – Mentors evaluate and rank student proposals.
Apr. 24, 2024 – Student proposals rankings completed by Org Admins.
May 1, 2024 – Accepted student proposals announced.
May 2 – May 28, 2024 – Community Bonding / Onboarding.
June – August 2024 – SoR participants work on projects / includes one mid-term evaluation.
SoR Overview
Quite a few conferences are now offering awards for reproducibility artifacts. This encourages authors to produce reproducibility artifacts. There is great potential for using these artifacts not only for validating research results but also as teaching tools in classrooms and as baselines in research labs. Simplifying the process of producing and using reproducibility artifacts can significantly accelerate the rate of insights. The Summer of Reproducibility allows summer students to help out in this cutting-edge effort and acquire valuable skills related to reproducibility. The Summer of Reproducibility (SoR) is a collaboration between the Open Source Research Experience (OSRE) organized by the OSPO UC Santa Cruz and the REPETO project. SoR provides support for undergraduate and graduate students contributing to open-source research efforts. The goal of the program is to help make computational research efforts reproducible. REPETO is funded by the NSF FAIROS RCN program and is a collaboration of Kate Keahey, U Chicago (lead PI), Haryadi Gunawi, U Chicago (co-PI), Cormac Flanagan (PI) and Stephanie Lieggi (co-PI), UC Santa Cruz, and Fraida Fund, NYU Tandon (PI).
In 2023, the SoR supported 19 students from around the world working with mentors who were top experts in their field. (For more on last year’s projects check out the OSRE23 student/fellows page and look for the #reproducibility tag) The Summer of Reproducibility 2024 is looking to work with even more mentors and fellows this year. SoR 2024 will provide support for undergraduates, graduate students, and other newcomers to the field who want to contribute to open-source research efforts related to making computational research reproducible. The SoR allows selected students and other contributors (our “SoR Fellows”) to help in this cutting-edge effort and acquire valuable skills.
Submitting a Project Idea for SoR 2024
Mentors are encouraged to submit project ideas by Feb. 6, 2024. However, project idea submissions will be accepted through the end of Feb. 2024.
If you are interested in being a SoR mentor the first step is to create “project ideas” (see instructions for submitting project ideas) for students to base their proposals around. A project idea is a brief overview of the activity/task a student would work on over the summer.
Accepted project ideas are compiled and published on the OSRE 2024 webpage (select the #reproducibility tag to filter) for students to browse; interested students then reach out to mentors as they build their SoR project proposal. SoR mentors are typically researchers or faculty from any university or organization looking to support the development and use of reproducibility artifacts. If you have questions about being a mentor or about project requirements, check out the mentor information video, the Mentor FAQ, or contact the SoR administrators directly (ospo-info-group@ucsc.edu).
How to Structure an SoR Project
The SoR program supports projects that focus on developing and using reproducible artifacts for Computer Science research. Project ideas for the SoR program should steer students towards producing reproducibility artifacts for use by others and/or using artifacts to validate research results, evaluate/develop teaching tools for classrooms, and extend research insights. See the SoR program page to read more about these objectives.
We do not require that SoR projects use any particular technology or platform. However, we encourage mentors to design projects that leverage public infrastructure and open-source platforms (such as Chameleon Cloud and CloudLab) to exploit the availability of shared resources, which can lead to more robust reproducibility artifacts and usage. For example, Chameleon enhances experiment reproducibility by offering shared access to uniform hardware resources and enabling users to configure, deploy, and share experimental environments (with the Trovi service) through cloud-based tools. Additionally, Chameleon’s Jupyter Notebook integration supports the implementation, analysis, and visualization of experiments within a cohesive environment, significantly facilitating the packaging of experiments for repeatability and further research collaboration.
Latest News on SoR
Summer of Reproducibility 2024 Is Coming!
