Grouped bundler update fails with "undefined method `force_encoding' for nil:NilClass"

[jdelStrother]

Is there an existing issue for this?

• I have searched the existing issues

Package ecosystem

Bundler

Package manager version

2.4.12

Language version

Ruby 3.2.2

Manifest location and content before the Dependabot update

/Gemfile

dependabot.yml content

version: 2

registries:
  contribsys: # Define access for Sidekiq/contribsys registry
    type: rubygems-server
    url: https://enterprise.contribsys.com/
    username: ${{secrets.SIDEKIQ_USERNAME}}
    password: ${{secrets.SIDEKIQ_PASSWORD}}

  ruby-github:
    type: rubygems-server
    url: https://rubygems.pkg.github.com/audioboom/
    token: ${{ secrets.GH_PACKAGE_REGISTRY_TOKEN }}

updates:
  - package-ecosystem: bundler
    directory: "/"
    schedule:
      interval: daily
    open-pull-requests-limit: 99
    # This seems bad, but is the default for rubygems.org, as far as I can tell.
    # So it's only necessary because we added contribsys for Sidekiq,
    # which I think we have to assume is probably fine.
    insecure-external-code-execution: allow
    registries:
      - contribsys
      - ruby-github
    vendor: true

    groups:
      rubocop-dependencies:
        patterns:
          - "rubocop*"
      rspec-dependencies:
        patterns:
          - "rspec*"
      cloud-service-dependencies:
        # Both AWS & Google Cloud have lots of auto-generated releases with not-very-helpful release notes.
        # Just group them into one big update
        patterns:
          - "aws-*"
          - "google-*"
      flipper-dependencies:
        patterns:
          - "flipper*"
      sidekiq-dependencies:
        patterns:
          - "sidekiq*"
      sentry-dependencies:
        patterns:
          - "sentry*"

Updated dependency

No response

What you expected to see, versus what you actually saw

Since enabling grouped updates, our dependabot updates fail with -

...
  proxy | 2023/05/14 07:59:44 [216] GET https://index.rubygems.org:443/versions
  proxy | 2023/05/14 07:59:44 [216] 304 https://index.rubygems.org:443/versions
  proxy | 2023/05/14 08:00:14 [218] GET https://rubygems.org:443/gems/rubocop-1.51.0.gem
  proxy | 2023/05/14 08:00:14 [218] 200 https://rubygems.org:443/gems/rubocop-1.51.0.gem
updater | 2023/05/14 08:00:20 ERROR <job_662289885> undefined method `force_encoding' for nil:NilClass
updater | 
updater |       string = string.dup.force_encoding(encoding)
updater |                          ^^^^^^^^^^^^^^^
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/parser-3.2.2.0/lib/parser/base.rb:98:in `setup_source_buffer'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/parser-3.2.2.0/lib/parser/base.rb:32:in `parse'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/bundler/lib/dependabot/bundler/file_parser/gemfile_declaration_finder.rb:45:in `parsed_gemfile'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/bundler/lib/dependabot/bundler/file_parser/gemfile_declaration_finder.rb:50:in `declaration_node'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/bundler/lib/dependabot/bundler/file_parser/gemfile_declaration_finder.rb:17:in `gemfile_includes_dependency?'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/bundler/lib/dependabot/bundler/file_parser.rb:55:in `block (2 levels) in gemfile_dependencies'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/bundler/lib/dependabot/bundler/file_parser.rb:54:in `each'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/bundler/lib/dependabot/bundler/file_parser.rb:54:in `block in gemfile_dependencies'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/bundler/lib/dependabot/bundler/file_parser.rb:51:in `each'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/bundler/lib/dependabot/bundler/file_parser.rb:51:in `gemfile_dependencies'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/bundler/lib/dependabot/bundler/file_parser.rb:23:in `parse'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/dependabot-updater/lib/dependabot/updater/group_update_creation.rb:31:in `block in compile_all_dependency_changes_for'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/dependabot-updater/lib/dependabot/updater/group_update_creation.rb:29:in `each'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/dependabot-updater/lib/dependabot/updater/group_update_creation.rb:29:in `compile_all_dependency_changes_for'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/group_update_all_versions.rb:86:in `block in run_grouped_dependency_updates'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/group_update_all_versions.rb:75:in `each'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/group_update_all_versions.rb:75:in `run_grouped_dependency_updates'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/dependabot-updater/lib/dependabot/updater/operations/group_update_all_versions.rb:43:in `perform'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/dependabot-updater/lib/dependabot/updater.rb:72:in `run'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/dependabot-updater/lib/dependabot/update_files_command.rb:38:in `perform_job'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> /home/dependabot/dependabot-updater/lib/dependabot/base_command.rb:52:in `run'
updater | 2023/05/14 08:00:20 ERROR <job_662289885> bin/update_files.rb:23:in `<main>'
updater | 2023/05/14 08:00:20 INFO <job_662289885> Sending event ad23e13cd224427599efae059be3ccab to Sentry
  proxy | 2023/05/14 08:00:21 [220] POST https://sentry.io:443/api/1451818/store/
  proxy | 2023/05/14 08:00:21 [220] 200 https://sentry.io:443/api/1451818/store/
updater | 2023/05/14 08:00:21 INFO Results:
updater | Dependabot encountered '1' error(s) during execution, please check the logs for more details.
updater | +---------------+
updater | |    Errors     |
updater | +---------------+
updater | | unknown_error |
updater | +---------------+
updater | time="2023-05-14T08:00:21Z" level=info msg="task complete" container_id=job-662289885-updater exit_code=0 job_id=662289885 step=updater

I can share the rest of the log & our Gemfile if there's somewhere private I can send them?

Native package manager behavior

No response

Images of the diff or a link to the PR, issue, or logs

No response

Smallest manifest that reproduces the issue

No response

[brrygrdn]

👋 @jdelStrother I've had a look into this to see if I could figure out a way to reproduce it but I'm not having much success.

I'd like to request permission to access your repository if you don't mind - your repository owner will receive an email from GitHub about the access request and need to give explicit permission which will give me a timeboxed period to access the repository to try debugging the issue first hand.

Is that something you would be ok with?

Alternatively, if you'd prefer to just provide your Gemfile/Gemfile.lock, etc privately, can I ask you file a support request? If you link to this issue it'll get referred to me internally and I can take it from there.

[jdelStrother]

Thanks - sure, send over the access request

[brrygrdn]

Cheers, I've been able to reproduce this and shouldn't require further access.

I've confirmed two things:

• The dependency the update is being clobbered on can be updated individually
• If I bypass the first dependency group and run any other group, the same problem occurs

The error happens after we make the first change to the files which results in the following modifications:

2023/05/25 17:40:34 DEBUG Dependency files updated:
2023/05/25 17:40:34 DEBUG   - /Gemfile 
2023/05/25 17:40:34 DEBUG   - /Gemfile.lock ( Changed 1 times )
2023/05/25 17:40:34 DEBUG   - /vendor/cache/[redacted]-old-version ( Changed 1 times )
2023/05/25 17:40:34 DEBUG   - /vendor/cache/[redacted]-new-version ( Changed 1 times )

The initial change seems ~typical, however it is resulting in the Gemfile becoming nil on a subsequent run as it hasn't changed which is unexpected.

My suspicion is that we aren't falling back to the original files properly under certain circumstances - or we aren't rewinding a file buffer properly after reading it once - I'll try to narrow down the repro conditions.

[jdelStrother]

@brrygrdn is there any workaround I can use in the meantime, or shall I just go back to non-grouped updates for now?

[brrygrdn]

@jdelStrother Unfortunately I think the only workaround for now would be to disable vendoring and have Dependabot just update your Gemfiles or revert to non-grouped updates.

I think I have a fix narrowed down but in terms of ETA it'll be some time after Wednesday 7th before it'll ship.

[jdelStrother]

Thanks - I'll disable grouping for now

[brrygrdn]

Just as a head's up on this, as you can see we have a couple of drafts open on this as the fix has potential knock-on implications for other ecosystems we've been trying to assess.

We have a way forward on this now which I'm actively working on but it'll take us a little longer than anticipated to remediate it and we're still clearing our backlog after pausing releases for a while. I'm hopeful we'll have meaningful progress towards a fix by the middle of next week.