Better caching on self-hosted runner

[juchiast]

I'm using self-hosted runner and network speed become a bottleneck when using the github cache provider. Is it possible to support other solutions such as https://github.com/runs-on/cache ?

[NobodyXu]

I think it's possible, rust-cache already supports build-jet caching using option cache-provider

[sehz]

We are running into same issue

[abonander]

@Swatinem you've said before in #126 (comment) that you don't want to overcomplicate the code by adding a bunch of extra backends. Is that still the case?

The problem I'm seeing is that we can only integrate caching backends that publish an NPM package with the same shape as https://www.npmjs.com/package/@actions/cache

We can't just invoke an arbitrary cache action (at least not without a lot more code), so integrating https://github.com/runs-on/cache and https://github.com/whywaita/actions-cache-s3 are out.

However, what would you say to a "dry run" mode that does the cache prep and then just returns the paths to cache and the cache key in the output? Then the user could invoke any cache action they want in a subsequent step. That would be so much more composable, and wouldn't require modifying rust-cache to support new providers.

This is a real issue we're facing because our self-hosted runners don't have a ton of upload bandwidth, so we end up waiting 5-10 minutes for the cache to upload at the conclusion of a run, or we hit timeouts. It doesn't make sense to post the cache to Github's servers anyway if we can just host it in a local S3 bucket in Minio or something.

[Swatinem]

That actually makes a lot of sense. Though the challenge there I guess would be that the meat of the action, namely pre-cleaning the cache happens in the post-run phase.
So you would have to find a way to pass these things along in the post-run phase somehow.

Contributions welcome :-)

[abonander]

Would it make sense to separate the action into its pre-run and post-run phases for this?

I think it's okay if it gets a little complicated, this is obviously for advanced usage anyway.

[Swatinem]

That might work, though I’m not sure how to properly do that :-D