Skip to content

Conversation

@mhucka
Copy link
Member

@mhucka mhucka commented Dec 10, 2025

The nightly jobs are failing due to running out of disk space. (It's the same problem we had recently with the CI checks workflow.) This PR updates the jobs to use the larger self-hosted runners from Google ML Velocity.

In addition, this PR makes some simplifications to the workflows to get rid of some of the overelaborate things I did before, and make minor other changes to keep with best practices such as putting timeouts on jobs.

Make steps simpler by using tricks such as setting SHELLOPTS, and use
new larger runner (from Google ML Velocity) for the build job because
the job is running out of disk space on the default runner.
These are safe to use.
Make steps simpler by using tricks such as setting SHELLOPTS, and use
new larger runner (from Google ML Velocity) for the build job because
the job is running out of disk space on the default runner.
It's impossible to test with the new ML runners in your fork, because
the fork is not part of the TensorFlow org and thus doesn't have access
to the runners. Need to provide the ability to specify the runner when
invoking the workflow manually.
In some contexts in GitHub Actions workflows, you have to use single
quotes. Remembering when and where in YAML is too hard. Just use single
quotes everywhere.s
@mhucka mhucka force-pushed the mh-runners-for-nightly-jobs branch from 7945915 to e34f32a Compare December 10, 2025 23:24
@mhucka mhucka added the area/devops Involves build systems, Make files, Bazel files, continuous integration, and/or other DevOps topics label Dec 10, 2025
@mhucka mhucka marked this pull request as ready for review December 11, 2025 00:02
Despite that this was valid YAML, the GitHub expression parser seems to
have a hard time with string literals that contain dashes. And
apparently, it's unnecessary to provide a fallback in this case anyway.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/devops Involves build systems, Make files, Bazel files, continuous integration, and/or other DevOps topics

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant