Skip to content

Commit

Permalink
ops: Build cleanups
Browse files Browse the repository at this point in the history
Cleans up a bunch of configuration in `ops/`, in preparation for switching most of our build configuration to CircleCI. Changes include:

1. Removing the "builder image" concept in favor of a single multi-target Dockerfile. Once this goes live on CircleCI, we'll be able to leverage Docker Layer caching more effectively.
2. Fixes the nightly build, which is currently failing on the legacy batch submitter.
3. Removes the legacy rpc-proxy since it isn't used anywhere anymore.
4. Re-enables CODEOWNERS.
5. Removes the `docker-compose-nobuild.yml` file, since the main `docker-compose.yml` file now supports images and builds.

Meta:

- Fixes ENG-1838
  • Loading branch information
mslipper committed Mar 8, 2022
1 parent 67146ed commit 0fb3050
Show file tree
Hide file tree
Showing 27 changed files with 99 additions and 965 deletions.
14 changes: 0 additions & 14 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,6 @@ jobs:
image-name: data-transport-layer
target: data-transport-layer
dockerfile: ./ops/docker/Dockerfile.packages
build-batch-submitter:
docker:
- image: cimg/base:2021.04
steps:
- build-dockerfile:
image-name: batch-submitter
target: batch-submitter
dockerfile: ./ops/docker/Dockerfile.packages
build-go-batch-submitter:
docker:
- image: cimg/base:2021.04
Expand Down Expand Up @@ -264,11 +256,6 @@ workflows:
- optimism
- slack
<<: *slack-nightly-build-fail-post-step
- build-batch-submitter:
context:
- optimism
- slack
<<: *slack-nightly-build-fail-post-step
- build-deployer:
context:
- optimism
Expand Down Expand Up @@ -306,7 +293,6 @@ workflows:
<<: *slack-nightly-build-fail-post-step
requires:
- build-dtl
- build-batch-submitter
- build-go-batch-submitter
- build-deployer
- build-l2geth
Expand Down
36 changes: 20 additions & 16 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -1,17 +1,21 @@
# CODEOWNERS can be disruptive because it automatically requests review from individuals across the
# board. We still like to use this file to track who's working on what, but all lines are commented
# out so that GitHub won't trigger review requests.
go/bss-core @cfromknecht @tynes
go/batch-submitter @cfromknecht @tynes
go/gas-oracle @tynes
go/l2geth-exporter @optimisticben @mslipper
go/op-exporter @optimisticben @mslipper
go/proxyd @mslipper @inphi
go/teleportr @mslipper @cfromknecht

# l2geth/ @smartcontracts @tynes @karlfloersch
# packages/specs/l2geth/ @smartcontracts @tynes @karlfloersch
# packages/contracts/ @smartcontracts @ben-chain @maurelian @elenadimitrova
# packages/specs/protocol/ @smartcontracts @ben-chain @maurelian
# ops/ @tynes @karlfloersch
# packages/core-utils/ @smartcontracts @annieke @ben-chain
# packages/common-ts/ @annieke
# packages/core-utils/src/watcher.ts @K-Ho
# packages/message-relayer/ @K-Ho
# packages/batch-submitter/ @annieke @karlfloersch
# packages/data-transport-layer/ @annieke
# packages/replica-healthcheck/ @annieke
# integration-tests/ @tynes
integration-tests/ @tynes @mslipper

packages/core-utils @smartcontracts @tynes
packages/common-ts/ @smartcontracts
packages/message-relayer/ @smartcontracts
packages/data-transport-layer/ @tynes @smartcontracts
packages/replica-healthcheck @optimisticben @tynes
packages/sdk @smartcontracts @mslipper
packages/contracts @elenadimitrova @maurelian @smartcontracts

l2geth @tynes @cfromknecht @smartcontracts

ops @tynes @optimisticben @mslipper
2 changes: 1 addition & 1 deletion .github/workflows/ext-test-snx.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ jobs:
if: failure()
uses: jwalton/gh-docker-logs@v1
with:
images: 'ethereumoptimism/builder,ethereumoptimism/hardhat,ethereumoptimism/deployer,ethereumoptimism/data-transport-layer,ethereumoptimism/l2geth,ethereumoptimism/message-relayer,ethereumoptimism/batch-submitter,ethereumoptimism/l2geth,ethereumoptimism/integration-tests'
images: 'ethereumoptimism/hardhat,ethereumoptimism/deployer,ethereumoptimism/data-transport-layer,ethereumoptimism/l2geth,ethereumoptimism/message-relayer,ethereumoptimism/batch-submitter,ethereumoptimism/l2geth,ethereumoptimism/integration-tests'
dest: '~/logs'

- name: Tar logs
Expand Down
126 changes: 30 additions & 96 deletions .github/workflows/publish-canary.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ jobs:
runs-on: ubuntu-latest
# map the step outputs to job outputs
outputs:
builder: ${{ steps.packages.outputs.builder }}
l2geth: ${{ steps.packages.outputs.l2geth }}
message-relayer: ${{ steps.packages.outputs.message-relayer }}
data-transport-layer: ${{ steps.packages.outputs.data-transport-layer }}
Expand All @@ -25,7 +24,6 @@ jobs:
hardhat-node: ${{ steps.packages.outputs.hardhat-node }}
canary-docker-tag: ${{ steps.docker-image-name.outputs.canary-docker-tag }}
proxyd: ${{ steps.packages.outputs.proxyd }}
rpc-proxy : ${{ steps.packages.outputs.rpc-proxy }}
op-exporter : ${{ steps.packages.outputs.op-exporter }}
l2geth-exporter : ${{ steps.packages.outputs.l2geth-exporter }}
batch-submitter-service : ${{ steps.packages.outputs.batch-submitter-service }}
Expand Down Expand Up @@ -91,10 +89,6 @@ jobs:
env:
CUSTOM_IMAGE_NAME: ${{ github.event.inputs.customImageName }}

# The below code is duplicated, would be ideal if we could use a matrix with a
# key/value being dynamically generated from the `publishedPackages` output
# while also allowing for parallelization (i.e. `l2geth` not depending on `builder`)
# and all jobs executing in parallel once `builder` is built
l2geth:
name: Publish L2Geth Version ${{ needs.canary-publish.outputs.canary-docker-tag }}
needs: canary-publish
Expand Down Expand Up @@ -173,44 +167,10 @@ jobs:
push: true
tags: ethereumoptimism/hardhat-node:${{ needs.canary-publish.outputs.canary-docker-tag }}

builder:
name: Prepare the base builder image for the services
needs: canary-publish
runs-on: ubuntu-latest
# we re-output the variables so that the child jobs can access them
outputs:
message-relayer: ${{ needs.canary-publish.outputs.message-relayer }}
data-transport-layer: ${{ needs.canary-publish.outputs.data-transport-layer }}
contracts: ${{ needs.canary-publish.outputs.contracts }}
integration-tests: ${{ needs.canary-publish.outputs.integration-tests }}
replica-healthcheck: ${{ needs.canary-publish.outputs.replica-healthcheck }}
canary-docker-tag: ${{ needs.canary-publish.outputs.canary-docker-tag }}

steps:
- name: Checkout
uses: actions/checkout@v2

- name: Login to Docker Hub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_ACCESS_TOKEN_USERNAME }}
password: ${{ secrets.DOCKERHUB_ACCESS_TOKEN_SECRET }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1

- name: Build and push
uses: docker/build-push-action@v2
with:
context: .
file: ./ops/docker/Dockerfile.monorepo
push: true
tags: ethereumoptimism/builder:${{ needs.canary-publish.outputs.canary-docker-tag }}

message-relayer:
name: Publish Message Relayer Version ${{ needs.builder.outputs.canary-docker-tag }}
needs: builder
if: needs.builder.outputs.message-relayer != ''
name: Publish Message Relayer Version ${{ needs.canary-publish.outputs.canary-docker-tag }}
needs: canary-publish
if: needs.canary-publish.outputs.message-relayer != ''
runs-on: ubuntu-latest

steps:
Expand All @@ -229,15 +189,15 @@ jobs:
uses: docker/build-push-action@v2
with:
context: .
file: ./ops/docker/Dockerfile.message-relayer
file: ./ops/docker/Dockerfile.packages
target: relayer
push: true
tags: ethereumoptimism/message-relayer:${{ needs.builder.outputs.canary-docker-tag }}
build-args: BUILDER_TAG=${{ needs.builder.outputs.canary-docker-tag }}
tags: ethereumoptimism/message-relayer:${{ needs.canary-publish.outputs.canary-docker-tag }}

data-transport-layer:
name: Publish Data Transport Layer Version ${{ needs.builder.outputs.canary-docker-tag }}
needs: builder
if: needs.builder.outputs.data-transport-layer != ''
name: Publish Data Transport Layer Version ${{ needs.canary-publish.outputs.canary-docker-tag }}
needs: canary-publish
if: needs.canary-publish.outputs.data-transport-layer != ''
runs-on: ubuntu-latest

steps:
Expand All @@ -256,15 +216,15 @@ jobs:
uses: docker/build-push-action@v2
with:
context: .
file: ./ops/docker/Dockerfile.data-transport-layer
file: ./ops/docker/Dockerfile.packages
target: data-transport-layer
push: true
tags: ethereumoptimism/data-transport-layer:${{ needs.builder.outputs.canary-docker-tag }}
build-args: BUILDER_TAG=${{ needs.builder.outputs.canary-docker-tag }}
tags: ethereumoptimism/data-transport-layer:${{ needs.canary-publish.outputs.canary-docker-tag }}

contracts:
name: Publish Deployer Version ${{ needs.builder.outputs.canary-docker-tag }}
needs: builder
if: needs.builder.outputs.contracts != ''
name: Publish Deployer Version ${{ needs.canary-publish.outputs.canary-docker-tag }}
needs: canary-publish
if: needs.canary-publish.outputs.contracts != ''
runs-on: ubuntu-latest

steps:
Expand All @@ -283,15 +243,15 @@ jobs:
uses: docker/build-push-action@v2
with:
context: .
file: ./ops/docker/Dockerfile.deployer
file: ./ops/docker/Dockerfile.packages
target: deployer
push: true
tags: ethereumoptimism/deployer:${{ needs.builder.outputs.canary-docker-tag }}
build-args: BUILDER_TAG=${{ needs.builder.outputs.canary-docker-tag }}
tags: ethereumoptimism/deployer:${{ needs.canary-publish.outputs.canary-docker-tag }}

integration_tests:
name: Publish Integration tests ${{ needs.builder.outputs.integration-tests }}
needs: builder
if: needs.builder.outputs.integration-tests != ''
name: Publish Integration tests ${{ needs.canary-publish.outputs.integration-tests }}
needs: canary-publish
if: needs.canary-publish.outputs.integration-tests != ''
runs-on: ubuntu-latest

steps:
Expand All @@ -310,15 +270,15 @@ jobs:
uses: docker/build-push-action@v2
with:
context: .
file: ./ops/docker/Dockerfile.integration-tests
file: ./ops/docker/Dockerfile.packages
target: integration-tests
push: true
tags: ethereumoptimism/integration-tests:${{ needs.builder.outputs.canary-docker-tag }}
build-args: BUILDER_TAG=${{ needs.builder.outputs.canary-docker-tag }}
tags: ethereumoptimism/integration-tests:${{ needs.canary-publish.outputs.canary-docker-tag }}

replica-healthcheck:
name: Publish Replica Healthcheck Version ${{ needs.builder.outputs.canary-docker-tag }}
needs: builder
if: needs.builder.outputs.replica-healthcheck != ''
name: Publish Replica Healthcheck Version ${{ needs.canary-publish.outputs.canary-docker-tag }}
needs: canary-publish
if: needs.canary-publish.outputs.replica-healthcheck != ''
runs-on: ubuntu-latest

steps:
Expand All @@ -337,10 +297,10 @@ jobs:
uses: docker/build-push-action@v2
with:
context: .
file: ./ops/docker/Dockerfile.replica-healthcheck
file: ./ops/docker/Dockerfile.packages
target: replica-healthcheck
push: true
tags: ethereumoptimism/replica-healthcheck:${{ needs.builder.outputs.canary-docker-tag }}
build-args: BUILDER_TAG=${{ needs.builder.outputs.canary-docker-tag }}
tags: ethereumoptimism/replica-healthcheck:${{ needs.canary-publish.outputs.canary-docker-tag }}

proxyd:
name: Publish proxyd Version ${{ needs.canary-publish.outputs.canary-docker-tag }}
Expand Down Expand Up @@ -453,32 +413,6 @@ jobs:
GITCOMMIT=${{ steps.build_args.outputs.GITCOMMIT }}
GITVERSION=${{ steps.build_args.outputs.GITVERSION }}
rpc-proxy:
name: Publish rpc-proxy Version ${{ needs.canary-publish.outputs.canary-docker-tag }}
needs: canary-publish
if: needs.canary-publish.outputs.rpc-proxy != ''
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1

- name: Login to Docker Hub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_ACCESS_TOKEN_USERNAME }}
password: ${{ secrets.DOCKERHUB_ACCESS_TOKEN_SECRET }}

- name: Build and push
uses: docker/build-push-action@v2
with:
context: .
file: ./ops/docker/Dockerfile.rpc-proxy
push: true
tags: ethereumoptimism/rpc-proxy:${{ needs.canary-publish.outputs.rpc-proxy }}

batch-submitter-service:
name: Publish batch-submitter-service Version ${{ needs.canary-publish.outputs.canary-docker-tag }}
needs: canary-publish
Expand Down
Loading

0 comments on commit 0fb3050

Please sign in to comment.