GitHub rate limit issues running GitHub Actions pipelines

[flelli]

The build process runs thousands of tests and some of them (integration and functional tests) run actual operations against GitHub using the test user. Integration tests only send a few requests, while functional ones hit hard on the rate limit.

Because of the rate limits in GitHub, some of these operations fail. However, retrying the failed jobs (after a short while) succeeds.
Parallelization has been prevented in the pipeline configuration for functional tests in order to limit the number of failed jobs to re-run in case the limit is reached. Nonetheless, this is not enough and jobs still fail.

The current limit is 5.000 requests per user/hour and there is no way to overcome it. The limit seems to be reset after 30 minutes. This tool can be used to interpret the x-ratelimit-reset.

Limits can be inspected by running:

$ curl -i https://api.github.com/users/octocat
> HTTP/2 200
> x-ratelimit-limit: 60
> x-ratelimit-remaining: 56
> x-ratelimit-used: 4
> x-ratelimit-reset: 1372700873

and when monitoring these numbers it becomes very clear thar when the x-ratelimit-remaining goes down to 0 the tests start failing.

Be careful as querying the above API also counts against the rate limit so do not uset it continuously or the limit will never reset.

[flelli]

The workarounds to this issue may be:

• parsing the x-ratelimit-reset and put jobs on hold for that specific time; this wouldn't actually solve the issue because one job may start clean and reach the limit (and fail) all at once if it performs too many requests
• narrow down the number of functional tests; this would affect the reliability of the project
• using multiple users to split the tests among, so none of them reaches the threshold