Skip to content

Commit

Permalink
Improve Bundler configuration
Browse files Browse the repository at this point in the history
When developing Bundler native helpers, I sometimes edit one helper and
then run `helpers/v2/build` in the development container and then rerun
specs affected by the change.

This currently doesn't work and results in something like:

```
[dependabot-core-dev] ~/dependabot-core/bundler $ helpers/v2/build
Successfully installed bundler-2.3.25
1 gem installed
Setting `path` to nil, since path and path.system are mutually exclusive
Fetching gem metadata from https://rubygems.org/......
Using rake 13.0.6
Fetching concurrent-ruby 1.1.10
Fetching minitest 5.16.3
Fetching ansi 1.5.0
Fetching public_suffix 5.0.0
Installing ansi 1.5.0
Installing minitest 5.16.3
Installing public_suffix 5.0.0
(...)

[dependabot-core-dev] ~/dependabot-core/bundler $ bundle exec rspec spec/dependabot/bundler/file_parser_spec.rb:627
Could not find debug-1.6.3, gpgme-2.0.21, parallel_tests-4.0.0, (...), unf_ext-0.0.8.2 in locally installed gems
```

The problem a bit complex due to how Bundler works which is a bit
unintuitive sometimes. Consider that:

1. When `BUNDLE_BIN` is set and `BUNDLE_PATH` are set, like in
  `Dockerfile.development` `bundle install` generates a
  `.bundle/bin/bundle` [binstub]. In this case,
  `/home/dependabot/dependabot-core/omnibus/.bundle/bin/bundle`.

2. Since `Dockerfile.development` also changes `PATH` to include
  `BUNDLE_BIN`, the above binstub is what gets run when you run `bundle`
  inside the development container.

3. The mentioned binstub sets `BUNDLE_GEMFILE` when it gets run.

4. When `BUNDLE_GEMFILE` is set, the root of configuration is taken
  relative to the `BUNDLE_GEMFILE`. This means, `bundle config` uses
  `/home/dependabot/dependabot-core/.bundle/config`.

5. When installing native helpers, the proper version of bundler is
  installed to a specific folder under the native helpers location, for
  example, `/opt/bundler/v2/.bundle`. Then `GEM_HOME` is set to that
  location, and `BUNDLE_PATH` is configured to "system" (meaning,
  default to `GEM_HOME`). This so that shelling out to native helpers
  can use the proper version of Bundler just by setting `GEM_HOME` to either
  `/opt/bundler/v1/.bundle` or `/opt/bundler/v2/.bundle`, according to
  the version of Bundler present in the `Gemfile.lock` file we're
  updating.

6. Due to bullet point 4 above, the `bundle config path.system true`
   command that gets run when installing native helpers saves the
   setting to `/home/dependabot/dependabot-core/omninus/.bundle/config`,
   instead of `/opt/bundler/v2/.bundle/config`. That means the setting
   accidentally applies to the subsequent `bundle exec rspec` command
   that ends up looking for gems in the global `GEM_HOME`, not in
   `/home/dependabot/dependabot-core/omnibus/.bundle`. Thus the error
   presented at the beginning.

The solution, I believe, is to remove the `BUNDLE_BIN` setting, which
causes Bundler to automatically generate binstubs for all gems every
time Bundler is run. In particular, that avoids the
`/home/dependabot/dependabot-omnibus/.bundle/bin/bundle` script from
being generated and avoids this issue altogether.

However, we still want to keep the convenience of not needing to run
`bundle exec` in the development container. So we add an explicit
`bundle binstubs` command to generate all application binstubs, except
for the one for Bundler, which we opt-out of.

The behavior of automatically generating binstubs when `BUNDLE_BIN` is
set will get removed in Bundler 3 anyways, so this is also a good move
in preparation for that.

[binstub]: https://github.com/rubygems/rubygems/blob/ea695143d951719a6a2e3baebb524b68cee70ce2/bundler/lib/bundler/templates/Executable.bundler.
  • Loading branch information
deivid-rodriguez committed Nov 23, 2022
1 parent 0ac4e37 commit 616f5b1
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 5 deletions.
4 changes: 1 addition & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,7 @@ ARG BUNDLER_V1_VERSION=1.17.3
ARG BUNDLER_V2_VERSION=2.3.25
ENV BUNDLE_SILENCE_ROOT_WARNING=1
# Allow gem installs as the dependabot user
ENV BUNDLE_PATH=".bundle" \
BUNDLE_BIN=".bundle/bin"
ENV PATH="$BUNDLE_BIN:$PATH:$BUNDLE_PATH/bin"
ENV BUNDLE_PATH=".bundle"

# Install Ruby, update RubyGems, and install Bundler
RUN mkdir -p /tmp/ruby-install \
Expand Down
6 changes: 4 additions & 2 deletions Dockerfile.development
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,11 @@ COPY --chown=dependabot:dependabot omnibus/Gemfile omnibus/dependabot-omnibus.ge
WORKDIR ${CODE_DIR}

RUN cd omnibus \
&& bundle install
&& bundle install \
&& BUNDLE_BIN=.bundle/bin bundle binstubs --all \
&& rm .bundle/bin/bundle # let RubyGems manage Bundler

ENV PATH="${CODE_DIR}/omnibus/$BUNDLE_BIN:$PATH"
ENV PATH="${CODE_DIR}/omnibus/.bundle/bin:$PATH"

RUN GREEN='\033[0;32m'; NC='\033[0m'; \
for d in `find ${CODE_DIR} -type f -mindepth 2 -maxdepth 2 \
Expand Down

0 comments on commit 616f5b1

Please sign in to comment.