Notes on PRs are sometimes higher than Gitlab limit (note=>["is too long (maximum is 1000000 characters)"])

[xens]

What Renovate type, platform and version are you using?

• Self-hosted Gitlab instance (v13.4.4)
• Self-hosted Renovate bot (renovateVersion": "23.77.0")

Describe the bug

From time to time Renovate stack-traces on some projects with the following error:

400 (Bad request) \"Note {:note=>[\"is too long (maximum is 1000000 characters)\"]}\" not given

Relevant debug logs

ERROR: Error ensuring PR: Response code 400 (Bad Request) (repository=xxxxxxxx/renovate-helper, branch=renovate/renovate-23.x)
       "err": {
         "name": "HTTPError",
         "timings": {
           "start": 1605859340945,
           "socket": 1605859340949,
           "lookup": 1605859340952,
           "connect": 1605859340955,
           "secureConnect": 1605859340963,
           "upload": 1605859341022,
           "response": 1605859341702,
           "end": 1605859341703,
           "phases": {
             "wait": 4,
             "dns": 3,
             "tcp": 3,
             "tls": 8,
             "request": 59,
             "firstByte": 680,
             "download": 1,
             "total": 758
           }
         },
         "message": "Response code 400 (Bad Request)",
         "stack": "HTTPError: Response code 400 (Bad Request)\n    at Request.<anonymous> (/usr/local/lib/node_modules/renovate/node_modules/got/dist/source/as-promise/index.js:117:42)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (node:internal/process/task_queues:93:5)",
         "options": {
           "headers": {
             "user-agent": "https://github.com/renovatebot/renovate",
             "accept-encoding": "gzip, deflate",
             "private-token": "**redacted**",
             "accept": "application/json",
             "content-type": "application/json",
             "content-length": "1271466"
           },
           "url": "https://gitlab-selfhosted/renovate-helper/merge_requests/34/notes",
           "username": "",
           "password": "",
           "method": "POST",
           "http2": false
         },
         "response": {
           "statusCode": 400,
           "statusMessage": "Bad Request",
           "body": {
             "message": "400 (Bad request) \"Note {:note=>[\"is too long (maximum is 1000000 characters)\"]}\" not given"
           },
           "headers": {
             "date": "Fri, 20 Nov 2020 08:02:21 GMT",
             "content-type": "application/json",
             "content-length": "114",
             "connection": "close",
             "set-cookie": [
               "__cfduid=xxxxxxxxxxx; expires=Sun, 20-Dec-20 08:02:20 GMT; path=/; domain=.xxxxxxxx; HttpOnly; SameSite=Lax; Secure"
             ],
             "cache-control": "no-cache",
             "vary": "Origin",
             "x-content-type-options": "nosniff",
             "x-frame-options": "SAMEORIGIN",
             "x-request-id": "xxxxxxxx",
             "x-runtime": "0.428610",
             "cf-cache-status": "DYNAMIC",
             "server": "cloudflare",
             "cf-ray": "5f50a5f10f0c34bf-DUB"
           },
           "httpVersion": "1.1"
         }
       }

What is "interesting" is that the note for this particular PR that generates the stack-trace is actually 2511 chars long, so we're far from the 1000000 limit, so it looks like that there's some kind of loop happening on certain conditions. More interestingly if I run the Bot in DEBUG mode the version bump work just fine, the bot is able to bump the version on the same PR that failed without the DEBUG mode

    "renovate-config-validator": "renovate-config-validator" 
  },
  "dependencies": {
-     "renovate": "23.84.5"
+     "renovate": "23.84.8"
  }
}

To Reproduce

I'll try to add more info I don't have the exact steps for now.

Additional context

It happens usually on repositories where changes are frequent and where the Renovate-bot need to bump versions on existing PRs. For example we have a repository that contains the code for our Renovate bot. The Renovate bot scans this repo everyday but we only merge the PRs once a week, thus the bot needs to edit ~7 time the PR as new releases for Renovate are issued every day.

{
  "name": "renovate-bot",
  "version": "0.0.0",
  "logLevel": "trace",
  "private": true,
  "description": "Configuration for the Renovate bot",
  "repository": "git@git.xxxxx.git",
  "author": "XXXXXXX",
  "scripts": {
    "renovate": "renovate",
    "renovate-config-validator": "renovate-config-validator" 
  },
  "dependencies": {
    "renovate": "23.84.5"
  }
}

[edomaur]

Cool, thanks @xens !

[xens]

When it happens I've identified two workarounds for now:

1. Delete the PR and let the bot recreate one from scratch
2. Put the bot in debug-mode

[xens]

I've progressed a bit on this issue, I was able to capture the whole stderr when the problem occurred, just giving an extract here, as the error payload is 1.3MB large with >1B loc.

So basically we have an issue with our Artifactory config on certain private NPM repo, and Renovate tries to POST the whole stderr payload (not honoring Gitlab limits on notes) which is too large and thus rejected by our Gitlab instance.

 INFO: Repository started (repository=xxxxxx/renovate-helper)
       "renovateVersion": "23.84.8"
(node:7) DeprecationWarning: Instance method `debug.destroy()` is deprecated and no longer does anything. It will be removed in the next major version of `debug`.
(Use `node --trace-deprecation ...` to show where the warning was created)
 INFO: Dependency extraction complete (repository=xxxxxx/renovate-helper)
       "baseBranch": "master",
       "stats": {
         "managers": {
           "dockerfile": {"fileCount": 1, "depCount": 1},
           "npm": {"fileCount": 1, "depCount": 1}
         },
         "total": {"fileCount": 2, "depCount": 2}
       }
 WARN: No github.com token has been configured. Skipping release notes retrieval (repository=KS/container-factory/renovate-helper)
       "manager": "npm",
       "depName": "renovate",
       "sourceUrl": "https://github.com/renovatebot/renovate"
 INFO: Branch updated (repository=xxxxxx/renovate-helper, branch=renovate/renovate-23.x)
       "commitSha": "6633150"
 INFO: PR updated (repository=xxxxxx/renovate-helper, branch=renovate/renovate-23.x)
       "pr": 37,
       "prTitle": "fix(deps): update dependency renovate to v23.86.2"
 WARN: artifactErrors (repository=xxxxxx/renovate-helper, branch=renovate/renovate-23.x)
       "artifactErrors": [
         {
           "lockFile": "package-lock.json",
           "stderr": 
npm WARN old lockfile The package-lock.json file was created with an old version of npm,\n
npm WARN old lockfile so supplemental metadata must be fetched from the registry.\n
npm WARN old lockfile \n
npm WARN old lockfile This is a one-time fix-up, please be patient...\n
npm WARN old lockfile \n
npm WARN old lockfile Error: Unable to authenticate, need: Basic realm=\"Artifactory Realm\"\n
npm WARN old lockfile     at /usr/local/lib/node_modules/npm/node_modules/npm-registry-fetch/check-response.js:113:17\n
npm WARN old lockfile     at processTicksAndRejections (node:internal/process/task_queues:93:5)\n
npm WARN old lockfile     at async Array.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:592:9)\n
npm WARN old lockfile  Could not fetch metadata for @arcanis/slice-ansi@1.0.2 HttpErrorAuthUnknown: Unable to authenticate, need: Basic realm=\"Artifactory Realm\"\n
npm WARN old lockfile     at /usr/local/lib/node_modules/npm/node_modules/npm-registry-fetch/check-response.js:113:17\n
npm WARN old lockfile     at processTicksAndRejections (node:internal/process/task_queues:93:5)\n
npm WARN old lockfile     at async Array.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/build-ideal-tree.js:592:9) {\n
npm WARN old lockfile   headers: [Object: null prototype] {\n
npm WARN old lockfile     date: [ 'Wed, 25 Nov 2020 19:01:07 GMT' ],\n
npm WARN old lockfile     'content-type': [ 'application/json;charset=ISO-8859-1' ],\n
npm WARN old lockfile     'content-length': [ '91' ],\n
npm WARN old lockfile     connection: [ 'keep-alive' ],\n
npm WARN old lockfile     'x-jfrog-version': [ 'Artifactory/7.10.6 71006900' ],\n
npm WARN old lockfile     'x-artifactory-id': [ '5ce25b6b1510e8d5e7e60ef7ece74710af7116ef' ],\n
npm WARN old lockfile     'x-artifactory-node-id': [ 'xxxxxxxxxxx-artifactory-member-1' ],\n
npm WARN old lockfile     'www-authenticate': [ 'Basic realm=\"Artifactory Realm\"' ],\n
npm WARN old lockfile     'x-fetch-attempts': [ '1' ]\n
npm WARN old lockfile   },\n
npm WARN old lockfile   statusCode: 401,\nnpm WARN old lockfile   code: 'E401',\n
npm WARN old lockfile   method: 'GET',\nnpm WARN old lockfile   uri: 'https://xxxxxxxxx.jfrog.io/xxxxxxxx/api/npm/npm-registry-virtual/@arcanis%2fslice-ansi',\n
npm WARN old lockfile   body: { errors: [ [Object] ] },\n
npm WARN old lockfile   pkgid: '@arcanis/slice-ansi@1.0.2'\n}

[rarkins]

So it's the MR comment for artifact errors that's too long, not the MR body?

[xens]

Absolutely

[rarkins]

OK, that's easy to fix then. I'll push a fix that ensures we apply our truncation when adding a PR/MR comment that has unknown length.

[renovate-release]

🎉 This issue has been resolved in version 23.87.1 🎉

The release is available on:

• GitHub release
• 23.87.1

Your semantic-release bot 📦🚀