Skip to content

Commit

Permalink
CI: Only bundle install for ecosystem under test
Browse files Browse the repository at this point in the history
Currently in the CI container we bundle install for _every_ package,
this ends up taking 4 to 5 minutes on every build for every ecosystem.

Since the tests are split up per ecosystem, we only need to install the
gems for the package that is being tested in this matrix step.

This also introduces a `script/ci-test` per ecosystem, it currently runs
both rubocop and rspec for all ecosystems, but the expectation is that
these scripts might diverge per ecosystem.

Running both rspec and rubocop in the same script is done to prevent us
from having to `bundle install` for each of those steps.

We could take this approach a step further and split out a CI container
per ecosystem, and that would let us run rubocop and rspec as separate
steps, but I don't think this is currently worth it.

This should shave about 5 minutes of every matrix build.
  • Loading branch information
jurre committed Apr 1, 2021
1 parent fb8e3d4 commit ac95c72
Show file tree
Hide file tree
Showing 20 changed files with 91 additions and 23 deletions.
5 changes: 1 addition & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -102,9 +102,6 @@ jobs:
if: matrix.suite.name == 'python'
run: |
docker run --rm "$CORE_CI_IMAGE" bash -c "pyenv exec flake8 python/helpers/. --count --exclude=./.*,./python/spec/fixtures --show-source --statistics"
- name: Run Ruby Rubocop linting
run: |
docker run --rm "$CORE_CI_IMAGE" bash -c "cd /home/dependabot/dependabot-core/${{ matrix.suite.path }} && bundle exec rubocop ."
- name: Run js linting and tests
if: matrix.suite.name == 'npm_and_yarn'
run: |
Expand All @@ -123,4 +120,4 @@ jobs:
- name: Run ${{ matrix.suite.name }} tests with rspec
run: |
docker run --env "CI=true" --env "DEPENDABOT_TEST_ACCESS_TOKEN=$DEPENDABOT_TEST_ACCESS_TOKEN" --env "SUITE_NAME=${{ matrix.suite.name }}" --rm "$CORE_CI_IMAGE" bash -c \
"cd /home/dependabot/dependabot-core/${{ matrix.suite.path }} && bundle exec rspec spec"
"cd /home/dependabot/dependabot-core/${{ matrix.suite.path }} && ./script/ci-test"
19 changes: 0 additions & 19 deletions Dockerfile.ci
Original file line number Diff line number Diff line change
Expand Up @@ -48,25 +48,6 @@ COPY python/Gemfile python/dependabot-python.gemspec ${CODE_DIR}/python/
COPY terraform/Gemfile terraform/dependabot-terraform.gemspec ${CODE_DIR}/terraform/
COPY omnibus/Gemfile omnibus/dependabot-omnibus.gemspec ${CODE_DIR}/omnibus/

RUN cd common && bundle install
RUN cd bundler && bundle install
RUN cd cargo && bundle install
RUN cd composer && bundle install
RUN cd dep && bundle install
RUN cd docker && bundle install
RUN cd elm && bundle install
RUN cd git_submodules && bundle install
RUN cd github_actions && bundle install
RUN cd go_modules && bundle install
RUN cd gradle && bundle install
RUN cd hex && bundle install
RUN cd maven && bundle install
RUN cd npm_and_yarn && bundle install
RUN cd nuget && bundle install
RUN cd python && bundle install
RUN cd terraform && bundle install
RUN cd omnibus && bundle install

COPY common/ ${CODE_DIR}/common/
COPY bundler/ ${CODE_DIR}/bundler/
COPY cargo/ ${CODE_DIR}/cargo/
Expand Down
5 changes: 5 additions & 0 deletions bundler/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions cargo/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions common/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions composer/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions dep/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions docker/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions elm/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions git_submodules/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions github_actions/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions go_modules/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions gradle/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions hex/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions maven/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions npm_and_yarn/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions nuget/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions omnibus/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions python/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec
5 changes: 5 additions & 0 deletions terraform/script/ci-test
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/sh

bundle install
bundle exec rubocop .
bundle exec rspec spec

0 comments on commit ac95c72

Please sign in to comment.