Options for tools provisioning

[bachorp]

It would be great to have more flexibility with regards to how the tools (i.e. the runner binary) are provisioned. It should be possible to pin the runner version and/or provide a custom download URL and/or binary. This is especially important in air-gapped environments.
The tools could also be cached and served by garm instead of being downloaded directly from within the runner.

[gabriel-samfira]

Hi @bachorp!

There are a couple of options that you can use with existing features:

• Cache the tools inside the image you're using. The userdata will skip downloading the runner if it already finds it on the machine.
• The tools that GARM manages are passed to the userdata script as the DownloadURL context value. You can theoretically set that to a static URL or ignore the {{ .DownloadURL }} and just parse your own URL/cache and use that.

This PR: #525 adds an easy way to edit templates an set them on pools.

That being said, I was thinking of making it possible to supply to garm some mechanism by which it can find tools. We need it anyway for a potential "agent" implementation. Something like simlpestreams might work as a metadata source for how to find tools, but I am not sure yet.

[bachorp]

Hi @gabriel-samfira, these are very viable options. What do you think about the ability to disable fetching of the URL in garm?

[gabriel-samfira]

Hi @gabriel-samfira, these are very viable options. What do you think about the ability to disable fetching of the URL in garm?

ohh. Right. The pool won't start unless tools are available. And you mentioned air-gaped environments. Let me try and figure something out.

[gabriel-samfira]

So, for gitea (and github....sort of), the runner download assets are parsed from the releases page of the runner software. GitHub actually offers an API endpoint that is essentially a wrapper/gate for what gets published on the releases page of https://github.com/actions/runner/ (releases page: https://api.github.com/repos/actions/runner/releases) . GHES servers I think cache those runners internally, but in the case of GHES, we also need an auth token to fetch the actual runner archive. This is exposed automatically by the github API.

For gitea the releases page is here: https://gitea.com/api/v1/repos/gitea/act_runner/releases (basically the repo here: https://gitea.com/gitea/act_runner).

For me, it would be easy to implement the ability to point a particular gitea endpoint to a different releases page. But you would essentially have to host the https://gitea.com/gitea/act_runner repo in your own instance, and potentially maintain releases similar to upstream. You don't have to host the code. We only really care about the binary releases that are published. So to have your own local cache, you would have to download the releases from upstream and publish them in your own repo. GARM would parse those via the API and select the newest ones. The names must be identical to upstream. We rely on the name to determine architecture, OS, version, etc.

Going with a mirror that we can point an endpoint to, does not break architectural boundaries (it wouldn't be a hack), and allows the current code to function as it does now (we just change the source of truth for the releases page).

Would this be an acceptable implementation? Before I dig deeper, I need to make sure I don't make your life harder by implementing this.

[bachorp]

I think you would still want the ability to opt out of fetching the tools URL entirely. If the actions runner is pre-provisioned (which I think is advisable anyway), I don't want to rely on gitea.com to be accessible nor do I want to self-host the respective release page for a binary that remains unused.

[gabriel-samfira]

That is completely understandable.

I am trying to figure out a way to make this work, without adding special cases to GARM itself. I can try and add a toggle by which I can just default the tools to a static "nightly" list of tools. This should allow the code to run without a lot of changes, and you can define runner install scripts that just ignores the tools. Even the default scripts will use the cached runner if it finds it.

My reticence to this is that such an option adds less flexibility than specifying a local mirror on your gitea instance, and it pretty much forces you to keep a set of gitea runner images up to date with the latest runners (or at least make sure it's with a runner version compatible with your deployment of Gitea).

The option I mentioned above can easily be achieved by using the gitea instance you want to use GARM with. You just need to create a repo in any user or org called act_runner (or anything else) and mirror the upstream act_runner release to it using something like:

https://gist.github.com/gabriel-samfira/006ecd5802dbb25e70348601a776c369 (see the first comment for usage)

You don't have to mirror all releases. The latest should do (ie: use --from-release=v0.2.10 flag). The above script can be run from a scheduled workflow in your gitea instance. It should be idempotent. After the initial sync, it will only sync newer releases. The added benefit of this is that if you want to make changes to the runner, you can easily cut your own releases which include your own changes. It adds more flexibility, it's local and it's hosted on the same gitea instance you manage anyway.

In any case, I will have a look through the code to see how hard each case would be to implement.

[gabriel-samfira]

yeah. I went ahead and added both. Will propose a PR soon.

[gabriel-samfira]

If you have the cycles, please take this for a spin: #537

[gabriel-samfira]

the nightly image now includes the change. After updating, go to Endpoints and update your gitea endpoint to use internal metadata URLs. Your pool manager should come up without issues, even if you can't call out to the upstream act_runner repo.

[bachorp]

@gabriel-samfira Thank you! I will test this next week.

[gabriel-samfira]

@gabriel-samfira Thank you! I will test this next week.

Anytime! Let me know how it goes!

[bachorp]

Works fine on my end :) Two points I might add:

• I believe there should be no fallback to the default URL in case the connection to the custom URL is disrupted. If a custom URL has been supplied, the user explicitly overrides the default and should not be surprised by a binary pulled in from elsewhere.
• The user interface might be improved. Tools Metadata URL should be immutable/grayed out if Use Internal Tools Metadata is checked. And maybe something like action runner release page would be more descriptive than tools metadata URL? I've made a sketch with some suggestions.

And thank you for your work on garm! I really like it.