Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix docker cross-platform builds #10827

Merged
merged 32 commits into from
Jul 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
bd4f5c5
Disable CGO for op-node. Test cross-platform
bitwiseguy Jun 14, 2024
bfbe665
Install latest gcp cli version before publishing docker image
bitwiseguy Jun 14, 2024
0126328
Add oplabs-gcr context to docker-build job so it can publish
bitwiseguy Jun 18, 2024
cf78ce7
Do not save op-node docker image tag for devnet
bitwiseguy Jun 18, 2024
4e79e90
Use TARGETARCH to conditionally set CC, CXX
bitwiseguy Jul 7, 2024
8aab662
Remove BUILDPLATFORM
bitwiseguy Jul 7, 2024
98ac765
Fix dangling && in dockerfile
bitwiseguy Jul 7, 2024
0572b99
Use apk instead of apt-get
bitwiseguy Jul 7, 2024
bf01862
Install correct apk cross build packages
bitwiseguy Jul 7, 2024
0533d17
Remove non-existent cross-build apk package
bitwiseguy Jul 7, 2024
fa668ca
Use bullseye instead of alpine
bitwiseguy Jul 7, 2024
afb11c4
Remove hardcoded CGO_ENABLED=0
bitwiseguy Jul 7, 2024
1077b98
Set CC and CXX in makefile
bitwiseguy Jul 8, 2024
a0e4a86
Add echo to makefile
bitwiseguy Jul 8, 2024
7362544
Remove echo from makefile
bitwiseguy Jul 8, 2024
c3366de
Remove check-cross-platform test
bitwiseguy Jul 8, 2024
6718f0c
Remove default values for CC and CXX from makefile
bitwiseguy Jul 8, 2024
c094713
Add TARGETOS and TARGETARCH to top of dockerfile
bitwiseguy Jul 8, 2024
b4dbcc9
Remove echo from dockerfile
bitwiseguy Jul 8, 2024
5d6bdf5
Use debian TARGET_BASE_IMAGE
bitwiseguy Jul 8, 2024
7efe9aa
Add cross platform args to all relevant Makefiles
bitwiseguy Jul 9, 2024
1f9ea9c
Add temp circle ci job to publish and check-cross-platform
bitwiseguy Jul 9, 2024
d1d43b9
Set CGO_ENABLED=0. Use alpine for TARGET_BASE_IMAGE
bitwiseguy Jul 9, 2024
95939b5
Use CMD instead of ENTRYPOINT for op-supervisor-target image
bitwiseguy Jul 9, 2024
71078e4
Add CC, CXX, CGO_ENABLED args to op-plasma
bitwiseguy Jul 9, 2024
8879cc7
Remove CC and CXX from Makefiles
bitwiseguy Jul 10, 2024
d1df3f7
Use alpine image for builder
bitwiseguy Jul 10, 2024
ce97c51
scheduled-docker-publish trigger changed to hourly or manual
bitwiseguy Jul 10, 2024
0f5b281
Fix circle ci config
bitwiseguy Jul 10, 2024
02d50ef
Fix warnings in dockerfile
bitwiseguy Jul 10, 2024
997bff2
Attempt cross compile without setting CGO_ENABLED=0
bitwiseguy Jul 10, 2024
a231f98
Add CGO_ENABLED=0 back for make op-node
bitwiseguy Jul 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 56 additions & 2 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ parameters:
sdk_dispatch:
type: boolean
default: false
docker_publish_dispatch:
type: boolean
default: false

orbs:
go: circleci/go@1.8.0
Expand Down Expand Up @@ -316,7 +319,10 @@ jobs:
- run:
command: mkdir -p /tmp/docker_images
- when:
condition: "<<parameters.release>>"
condition:
or:
- "<<parameters.publish>>"
- "<<parameters.release>>"
steps:
- gcp-cli/install
- when:
Expand Down Expand Up @@ -2093,7 +2099,10 @@ workflows:

scheduled-docker-publish:
when:
equal: [ build_hourly, <<pipeline.schedule.name>> ]
or:
- equal: [ build_hourly, <<pipeline.schedule.name>> ]
# Trigger on manual triggers if explicitly requested
- equal: [ true, << pipeline.parameters.docker_publish_dispatch >> ]
jobs:
- docker-build:
name: op-node-docker-publish
Expand All @@ -2104,6 +2113,11 @@ workflows:
context:
- oplabs-gcr
- slack
- check-cross-platform:
name: op-node-cross-platform
op_component: op-node
requires:
- op-node-docker-publish
- docker-build:
name: op-batcher-docker-publish
docker_name: op-batcher
Expand All @@ -2113,6 +2127,11 @@ workflows:
context:
- oplabs-gcr
- slack
bitwiseguy marked this conversation as resolved.
Show resolved Hide resolved
- check-cross-platform:
name: op-batcher-cross-platform
op_component: op-batcher
requires:
- op-batcher-docker-publish
- docker-build:
name: op-program-docker-publish
docker_name: op-program
Expand All @@ -2122,6 +2141,11 @@ workflows:
context:
- oplabs-gcr
- slack
- check-cross-platform:
name: op-program-cross-platform
op_component: op-program
requires:
- op-program-docker-publish
- docker-build:
name: op-proposer-docker-publish
docker_name: op-proposer
Expand All @@ -2131,6 +2155,11 @@ workflows:
context:
- oplabs-gcr
- slack
- check-cross-platform:
name: op-proposer-cross-platform
op_component: op-proposer
requires:
- op-proposer-docker-publish
- docker-build:
name: op-challenger-docker-publish
docker_name: op-challenger
Expand All @@ -2140,6 +2169,11 @@ workflows:
context:
- oplabs-gcr
- slack
- check-cross-platform:
name: op-challenger-cross-platform
op_component: op-challenger
requires:
- op-challenger-docker-publish
- docker-build:
name: op-dispute-mon-docker-publish
docker_name: op-dispute-mon
Expand All @@ -2149,6 +2183,11 @@ workflows:
context:
- oplabs-gcr
- slack
- check-cross-platform:
name: op-dispute-mon-cross-platform
op_component: op-dispute-mon
requires:
- op-dispute-mon-docker-publish
- docker-build:
name: op-conductor-docker-publish
docker_name: op-conductor
Expand All @@ -2158,6 +2197,11 @@ workflows:
context:
- oplabs-gcr
- slack
- check-cross-platform:
name: op-conductor-cross-platform
op_component: op-conductor
requires:
- op-conductor-docker-publish
- docker-build:
name: op-heartbeat-docker-publish
docker_name: op-heartbeat
Expand All @@ -2167,6 +2211,11 @@ workflows:
context:
- oplabs-gcr
- slack
- check-cross-platform:
name: op-heartbeat-cross-platform
op_component: op-heartbeat
requires:
- op-heartbeat-docker-publish
- docker-build:
name: op-supervisor-docker-publish
docker_name: op-supervisor
Expand All @@ -2176,6 +2225,11 @@ workflows:
context:
- oplabs-gcr
- slack
- check-cross-platform:
name: op-supervisor-cross-platform
op_component: op-supervisor
requires:
- op-supervisor-docker-publish
- docker-build:
name: chain-mon-docker-publish
docker_name: chain-mon
Expand Down
2 changes: 1 addition & 1 deletion docker-bake.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ variable "IMAGE_TAGS" {

variable "PLATFORMS" {
// You can override this as "linux/amd64,linux/arm64".
// Only a specify a single platform when `--load` ing into docker.
// Only specify a single platform when `--load` ing into docker.
// Multi-platform is supported when outputting to disk or pushing to a registry.
// Multi-platform builds can be tested locally with: --set="*.output=type=image,push=false"
default = ""
Expand Down
2 changes: 1 addition & 1 deletion op-batcher/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ ifeq ($(shell uname),Darwin)
endif

op-batcher:
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) go build -v $(LDFLAGS) -o ./bin/op-batcher ./cmd
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) CGO_ENABLED=0 go build -v $(LDFLAGS) -o ./bin/op-batcher ./cmd

clean:
rm bin/op-batcher
Expand Down
2 changes: 1 addition & 1 deletion op-challenger/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ ifeq ($(shell uname),Darwin)
endif

op-challenger:
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) go build -v $(LDFLAGS) -o ./bin/op-challenger ./cmd
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) CGO_ENABLED=0 go build -v $(LDFLAGS) -o ./bin/op-challenger ./cmd

fuzz:
go test $(FUZZLDFLAGS) -run NOTAREALTEST -v -fuzztime 10s -fuzz FuzzKeccak ./game/keccak/matrix
Expand Down
2 changes: 1 addition & 1 deletion op-conductor/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ LDFLAGSSTRING +=-X main.Version=$(VERSION)
LDFLAGS := -ldflags "$(LDFLAGSSTRING)"

op-conductor:
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) go build -v $(LDFLAGS) -o ./bin/op-conductor ./cmd
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) CGO_ENABLED=0 go build -v $(LDFLAGS) -o ./bin/op-conductor ./cmd

clean:
rm bin/op-conductor
Expand Down
2 changes: 1 addition & 1 deletion op-dispute-mon/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ LDFLAGSSTRING +=-X github.com/ethereum-optimism/optimism/op-dispute-mon/version.
LDFLAGS := -ldflags "$(LDFLAGSSTRING)"

op-dispute-mon:
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) go build -v $(LDFLAGS) -o ./bin/op-dispute-mon ./cmd
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) CGO_ENABLED=0 go build -v $(LDFLAGS) -o ./bin/op-dispute-mon ./cmd
.PHONY: op-dispute-mon

clean:
Expand Down
2 changes: 1 addition & 1 deletion op-heartbeat/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ LDFLAGSSTRING +=-X main.Version=$(VERSION)
LDFLAGS := -ldflags "$(LDFLAGSSTRING)"

op-heartbeat:
env GO111MODULE=on go build -v $(LDFLAGS) -o ./bin/op-heartbeat ./cmd
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) CGO_ENABLED=0 go build -v $(LDFLAGS) -o ./bin/op-heartbeat ./cmd

clean:
rm bin/op-heartbeat
Expand Down
2 changes: 1 addition & 1 deletion op-node/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ ifeq ($(shell uname),Darwin)
endif

op-node:
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) go build -v $(LDFLAGS) -o ./bin/op-node ./cmd/main.go
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) CGO_ENABLED=0 go build -v $(LDFLAGS) -o ./bin/op-node ./cmd/main.go

clean:
rm bin/op-node
Expand Down
2 changes: 1 addition & 1 deletion op-plasma/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ LDFLAGSSTRING +=-X main.Version=$(VERSION)
LDFLAGS := -ldflags "$(LDFLAGSSTRING)"

da-server:
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) go build -v $(LDFLAGS) -o ./bin/da-server ./cmd/daserver
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) CGO_ENABLED=0 go build -v $(LDFLAGS) -o ./bin/da-server ./cmd/daserver

clean:
rm bin/da-server
Expand Down
2 changes: 1 addition & 1 deletion op-program/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ op-program: \
op-program-client-mips

op-program-host:
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) go build -v -ldflags "$(LDFLAGSSTRING)" -o ./bin/op-program ./host/cmd/main.go
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) CGO_ENABLED=0 go build -v -ldflags "$(LDFLAGSSTRING)" -o ./bin/op-program ./host/cmd/main.go

op-program-client:
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) go build -v -ldflags "$(PC_LDFLAGSSTRING)" -o ./bin/op-program-client ./client/cmd/main.go
Expand Down
2 changes: 1 addition & 1 deletion op-proposer/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ LDFLAGSSTRING +=-X main.Version=$(VERSION)
LDFLAGS := -ldflags "$(LDFLAGSSTRING)"

op-proposer:
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) go build -v $(LDFLAGS) -o ./bin/op-proposer ./cmd
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) CGO_ENABLED=0 go build -v $(LDFLAGS) -o ./bin/op-proposer ./cmd

clean:
rm bin/op-proposer
Expand Down
3 changes: 1 addition & 2 deletions op-supervisor/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,8 @@ LDFLAGSSTRING +=-X main.Version=$(VERSION)
LDFLAGSSTRING +=-X main.Meta=$(VERSION_META)
LDFLAGS := -ldflags "$(LDFLAGSSTRING)"


op-supervisor:
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) go build -v $(LDFLAGS) -o ./bin/op-supervisor ./cmd
env GO111MODULE=on GOOS=$(TARGETOS) GOARCH=$(TARGETARCH) CGO_ENABLED=0 go build -v $(LDFLAGS) -o ./bin/op-supervisor ./cmd

clean:
rm bin/op-supervisor
Expand Down
Loading