Frequent "429 TOO MANY REQUESTS" errors from nur-update

[rycee]

My GitLab CI job frequently fails due to

curl -sfSL -XPOST https://nur-update.herokuapp.com/update?repo=rycee

erroring out with the message

curl: (22) The requested URL returned error: 429 TOO MANY REQUESTS

See https://gitlab.com/rycee/nur-expressions/pipelines for the historic runs (the jobs with two stages are the ones pinging nur-update).

Even though the nur-update service gives the 429 error, the update still seems to happen in the NUR repo. For example 8044a5a corresponds to the failing job https://gitlab.com/rycee/nur-expressions/-/jobs/438621162.

I think my repo does hit nur-update more frequently than most NURs, though. Should I try fixing my CI job to avoid curling more than once a day or so?

[Mic92]

This error comes from heroku. I am using a free instance right now. Maybe it is hitting some limits? Has anybody else seen this happen?

[Mic92]

The number of requests don't seems excessive. I have other apps that certainly get more requests.

[Mic92]

I also see now other people having this problem:

2020-02-17T09:54:56.100075+00:00 heroku[router]: at=info method=POST path="/update?repo=sikmir" host=nur-update.herokuapp.com request_id=5369c090-02ef-46a0-99e3-d7b711a7b5d0 fwd="35.196.158.85" dyno=web.1 connect=1ms service=485ms status=200 bytes=734 protocol=https

2020-02-17T09:54:56.099966+00:00 app[web.1]: 10.71.246.139 - - [17/Feb/2020:09:54:56 +0000] "POST /update?repo=sikmir HTTP/1.1" 200 573 "-" "curl/7.47.0"

2020-02-17T10:25:40.018678+00:00 app[web.1]: 10.63.109.106 - - [17/Feb/2020:10:25:40 +0000] "POST /update?repo=rycee HTTP/1.1" 200 573 "-" "curl/7.64.1"

2020-02-17T10:25:40.024014+00:00 heroku[router]: at=info method=POST path="/update?repo=rycee" host=nur-update.herokuapp.com request_id=90da5bb7-d365-44bc-8f7c-327171bac1de fwd="213.39.27.190" dyno=web.1 connect=0ms service=386ms status=200 bytes=734 protocol=https

2020-02-17T10:26:16.212875+00:00 heroku[router]: at=info method=POST path="/update?repo=moredhel" host=nur-update.herokuapp.com request_id=52ea3e15-f21d-4fed-b38f-e9a2629572b6 fwd="213.39.27.190" dyno=web.1 connect=1ms service=360ms status=429 bytes=228 protocol=https

2020-02-17T10:26:16.207429+00:00 app[web.1]: 10.91.151.49 - - [17/Feb/2020:10:26:16 +0000] "POST /update?repo=moredhel HTTP/1.1" 429 53 "-" "curl/7.64.1"

2020-02-17T10:26:22.589073+00:00 app[web.1]: 10.95.191.28 - - [17/Feb/2020:10:26:22 +0000] "POST /update?repo=moredhel HTTP/1.1" 429 53 "-" "curl/7.64.1"

2020-02-17T10:26:22.589106+00:00 heroku[router]: at=info method=POST path="/update?repo=moredhel" host=nur-update.herokuapp.com request_id=756837ae-aa68-4ed0-bdfa-853fb4234cad fwd="213.39.27.190" dyno=web.1 connect=1ms service=247ms status=429 bytes=228 protocol=https

2020-02-17T10:29:30.244508+00:00 heroku[router]: at=info method=POST path="/update?repo=moredhel" host=nur-update.herokuapp.com request_id=eac31909-4a43-45d6-88c3-842ba5a30db7 fwd="213.39.27.190" dyno=web.1 connect=0ms service=512ms status=429 bytes=228 protocol=https

2020-02-17T10:29:30.244445+00:00 app[web.1]: 10.30.205.3 - - [17/Feb/2020:10:29:30 +0000] "POST /update?repo=moredhel HTTP/1.1" 429 53 "-" "curl/7.64.1"

2020-02-17T11:00:21.979639+00:00 heroku[router]: at=info method=GET path="/?update=repo=moredhel" host=nur-update.herokuapp.com request_id=eea82db1-71a2-48c5-ad15-2b977fe5358e fwd="129.215.90.157" dyno=web.1 connect=0ms service=1ms status=200 bytes=707 protocol=https

2020-02-17T11:00:21.979569+00:00 app[web.1]: 10.11.198.97 - - [17/Feb/2020:11:00:21 +0000] "GET /?update=repo=moredhel HTTP/1.1" 200 546 "-" "curl/7.68.0"

2020-02-17T11:00:27.804301+00:00 app[web.1]: 10.71.137.172 - - [17/Feb/2020:11:00:27 +0000] "POST /?update=repo=moredhel HTTP/1.1" 405 178 "-" "curl/7.68.0"

2020-02-17T11:00:27.804495+00:00 heroku[router]: at=info method=POST path="/?update=repo=moredhel" host=nur-update.herokuapp.com request_id=3401e213-d128-4a9e-b44a-95d745698a5f fwd="129.215.90.157" dyno=web.1 connect=1ms service=34ms status=405 bytes=367 protocol=https

2020-02-17T11:00:45.962843+00:00 heroku[router]: at=info method=POST path="/update?=repo=moredhel" host=nur-update.herokuapp.com request_id=d73b37b2-f68e-44d8-8ea5-9c29f0a62c1f fwd="129.215.90.157" dyno=web.1 connect=0ms service=328ms status=400 bytes=194 protocol=https

2020-02-17T11:00:45.962312+00:00 app[web.1]: 10.111.188.18 - - [17/Feb/2020:11:00:45 +0000] "POST /update?=repo=moredhel HTTP/1.1" 400 25 "-" "curl/7.68.0"

2020-02-17T11:00:51.553581+00:00 app[web.1]: 10.37.240.182 - - [17/Feb/2020:11:00:51 +0000] "POST /update?repo=moredhel HTTP/1.1" 200 573 "-" "curl/7.68.0"

2020-02-17T11:00:51.553276+00:00 heroku[router]: at=info method=POST path="/update?repo=moredhel" host=nur-update.herokuapp.com request_id=a76336c0-6a13-4b66-b859-a94849267890 fwd="129.215.90.157" dyno=web.1 connect=1ms service=386ms status=200 bytes=734 protocol=https

[Mic92]

Could be also API limits from travis ci...