reproducibility.tex

%! TEX root = main.tex

In our work, we strive for achieving reproducibility of the results, and all the code we developed for this research is available publicly on GitHub under an open-source license, while all the data is available in open archival repositories.
PetIBM is an open-source CFD library based on the immersed boundary method, and is available at \url{https://github.com/barbagroup/PetIBM} under the permissive BSD-3 license. 
The software was peer reviewed and published in the Journal of Open Source Software \cite{chuang_petibm_2018}. 
Our PINN solvers based on the NVIDIA \emph{Modulus} toolkit can be found following the links in the GitHub repository for this paper, located at \url{https://github.com/barbagroup/jcs_paper_pinn/}. 
There, the folder prefixed by \texttt{repro-pack} corresponds to a git submodule pointing to the relevant commit on a branch of the repository for the full reproducibility package of the first author's PhD dissertation \cite{chuang_thesis_2023}.
The branch named \texttt{jcs-paper} contains the modified plotting scripts to produce the publication-quality figures in this paper.    
A snapshot of the repro-pack is archived on Zenodo, and the DOI is 10.5281/zenodo.7988067.
As described in the README of the repro-pack, readers can use pre-generated data for plotting the figures in this paper, or they can re-run the solutions using the code and data available in the repro-pack.
The latter option is of course limited by the computational resources available to the reader.
For the first option, the reader can find the raw data in a Zenodo archive, with DOI: 10.5281/zenodo.7988106.
To facilitate reproducibility of the computational environment, we execute all cases using Singularity/Apptainer images for both the PetIBM and PINN cases. 
All the container recipes are included in the repro-pack under the \texttt{resources} folder. 
The \emph{Modulus} toolkit was open-sourced by NVIDIA in March 2023,\footnote{\url{https://developer.nvidia.com/blog/physics-ml-platform-modulus-is-now-open-source/}} under the Apache License 2.0.
This is a permissive license that requires preservation of copyright and license notices and provides an express grant of patent rights. 
When we started this research, \emph{Modulus} was not yet open-source, but it was publicly available through the conditions of an End User Agreement. 
Documentation of those conditions can be found via the May 21, 2022, snapshot of the \emph{Modulus} developer website on the Internet Archive Wayback Machine.\footnote{\url{https://web.archive.org/web/20220521223413/https://catalog.ngc.nvidia.com/orgs/nvidia/teams/modulus/containers/modulus}}
We are confident that following the best practices of open science described in this statement provides good conditions for reproducibility of our results. 
Readers can inspect the code if any detail is unclear in the paper narrative, and they can re-analyze our data or re-run the computational experiments.
We spared no effort to document, organize, and preserve all the digital artifacts for this work.