Upload tutorials output to a csv in 'artifacts' branch

[esantorella]

Motivation

Writes the runtime and memory output we already produce to the 'artifacts' branch. The upload happens when the tutorials run on push and in the nightly cron. An example will show here after the nightly cron finishes running: https://github.com/pytorch/botorch/tree/artifacts/tutorial_performance_data Currently there are a couple test files in there, which I plan to clean up.

Test Plan

[x] Check that the upload works "with smoke test" by setting it to run on a push to this branch: https://github.com/pytorch/botorch/actions/runs/4226339782
[x] Check that it runs in the nightly cron and that output looks as expected: https://github.com/pytorch/botorch/blob/artifacts/tutorial_performance_data/standard_590b6edd_2023-02-20%2019%3A09%3A44.557008.csv

[codecov]

Codecov Report

Merging #1695 (347ef9a) into main (ecd83ca) will not change coverage.
The diff coverage is n/a.

❗ Current head 347ef9a differs from pull request most recent head bacd842. Consider uploading reports for the commit bacd842 to get more accurate results

@@            Coverage Diff            @@
##              main     #1695   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          169       169           
  Lines        14537     14537           
=========================================
  Hits         14537     14537           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[facebook-github-bot]

@esantorella has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@esantorella has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[saitcakmak]

Thanks for putting this together. This lgtm as is, though I feel like it would be much easier to sift through the outputs if we were to append them to the same file rather than creating a new file each time. Do you think we could read the last file with pandas, append the new df and save that instead?

[esantorella]

Thanks for putting this together. This lgtm as is, though I feel like it would be much easier to sift through the outputs if we were to append them to the same file rather than creating a new file each time. Do you think we could read the last file with pandas, append the new df and save that instead?

Yeah I think that's a good idea -- let me see if I can get it working.

By the way, I'm working on a notebook to make it easier to visualize the data and provide examples for how to work with it: https://github.com/esantorella/botorch/blob/tutorials_analytics/notebooks/tutorials_performance_tracking.ipynb

The problem with the notebook is that with this setup, there's no branch that both contains the csv data and has up-to-date Python functionality. The notebook would be a lot easier to run if there were just one CSV and that could be grabbed from GitHub, so I like your idea a lot.

[esantorella]

On second thought, I don't love the job of having a simple "concatenate csvs" line since that makes it hard to change the format of data in the future. I'm going to land this without the concatenation step, then in a subsequent PR have Pandas deal with merging data.

[facebook-github-bot]

@esantorella merged this pull request in 6854751.