Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ci): Add Makefile for easy local build #2985

Merged
merged 27 commits into from
Oct 2, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 112 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
DOCKER_BUILD_CMD=docker build
PROTOCOL_VERSION=2.0
DOCKERFILES=docker
CONTEXT=.
# Additional packages versions
DOCKER_VERSION_MIN=27.1.2
NODE_VERSION_MIN=20.17.0
YARN_VERSION_MIN=1.22.19
RUST_VERSION=nightly-2024-08-01
SQLX_CLI_VERSION=0.8.1
FORGE_MIN_VERSION=0.2.0

# Versions and packages checks
check-nodejs:
@command -v node >/dev/null 2>&1 || { echo >&2 "Node.js is not installed. Please install Node.js v$(NODE_VERSION_MIN) or higher."; exit 1; }
@NODE_VERSION=$$(node --version | sed 's/v//'); \
if [ "$$(echo $$NODE_VERSION $(NODE_VERSION_MIN) | tr " " "\n" | sort -V | head -n1)" != "$(NODE_VERSION_MIN)" ]; then \
echo "Node.js version $$NODE_VERSION is too low. Please update to v$(NODE_VERSION_MIN)."; exit 1; \
fi

check-yarn:
@command -v yarn >/dev/null 2>&1 || { echo >&2 "Yarn is not installed. Please install Yarn v$(YARN_VERSION_MIN) or higher."; exit 1; }
@YARN_VERSION=$$(yarn --version); \
if [ "$$(echo $$YARN_VERSION $(YARN_VERSION_MIN) | tr " " "\n" | sort -V | head -n1)" != "$(YARN_VERSION_MIN)" ]; then \
echo "Yarn version $$YARN_VERSION is too low. Please update to v$(YARN_VERSION_MIN)."; exit 1; \
fi

check-rust:
@command -v rustc >/dev/null 2>&1 || { echo >&2 "Rust is not installed. Please install Rust v$(RUST_VERSION)."; exit 1; }
@command -v rustup install $$RUST_VERSION >/dev/null 2>&1
artmakh marked this conversation as resolved.
Show resolved Hide resolved

check-sqlx-cli:
@command -v sqlx >/dev/null 2>&1 || { echo >&2 "sqlx-cli is not installed. Please install sqlx-cli v$(SQLX_CLI_VERSION)"; exit 1; }
@SQLX_CLI_VERSION_LOCAL=$$(sqlx --version | cut -d' ' -f2); \
if [ "$$(echo $$SQLX_CLI_VERSION_LOCAL $(SQLX_CLI_VERSION) | tr " " "\n" | sort -V | head -n1)" != "$(SQLX_CLI_VERSION)" ]; then \
echo "sqlx-cli version $$SQLX_CLI_VERSION is wrong. Please update to v$(SQLX_CLI_VERSION)"; exit 1; \
fi

check-docker:
@command -v docker >/dev/null 2>&1 || { echo >&2 "Docker is not installed. Please install Docker v$(DOCKER_VERSION_MIN) or higher."; exit 1; }
@DOCKER_VERSION=$$(docker --version | grep -oE '[0-9]+\.[0-9]+\.[0-9]+'); \
if [ "$$(echo $$DOCKER_VERSION $(DOCKER_VERSION_MIN) | tr " " "\n" | sort -V | head -n1)" != "$(DOCKER_VERSION_MIN)" ]; then \
echo "Docker version $$DOCKER_VERSION is too low. Please update to v$(DOCKER_VERSION_MIN) or higher."; exit 1; \
fi

check-foundry:
@command -v forge --version >/dev/null 2>&1 || { echo >&2 "Foundry is not installed. Please install Foundry"; exit 1; }
@FORGE_VERSION=$$(forge --version | grep -oE '[0-9]+\.[0-9]+\.[0-9]+'); \
if [ "$$(echo $$FORGE_VERSION $(FORGE_MIN_VERSION) | tr " " "\n" | sort -V | head -n1)" != "$(FORGE_MIN_VERSION)" ]; then \
echo "Forge version $$FORGE_VERSION is too low. Please update to v$(FORGE_MIN_VERSION) or higher."; exit 1; \
fi

# Check for all required tools
check-tools: check-nodejs check-yarn check-rust check-sqlx-cli check-docker check-foundry
@echo "All required tools are installed."

# Check that contracts are checkout properly
check-contracts:
@if [ ! -d contracts/l1-contracts/lib/openzeppelin-contracts-upgradeable/lib/erc4626-tests ] || [ -z "$$(ls -A contracts/l1-contracts/lib/openzeppelin-contracts-upgradeable/lib/erc4626-tests)" ]; then \
echo "l1-contracts submodule is missing. Please download repo with `git clone --recurse-submodules https://github.com/matter-labs/zksync-era.git`"; \
exit 1; \
fi

# Build and download needed contracts
prepare-contracts: check-tools check-contracts
@export ZKSYNC_HOME=$$(pwd) && \
export PATH=$$PATH:$${ZKSYNC_HOME}/bin && \
mkdir -p ./volumes/postgres && \
docker compose up -d postgres && \
zkt || true && \
cp etc/tokens/test.json . && \
zk_supervisor contracts

# Download setup-key
prepare-keys:
@curl -LO https://storage.googleapis.com/matterlabs-setup-keys-us/setup-keys/setup_2\^26.key

# Targets for building each container
build-contract-verifier: check-tools prepare-contracts prepare-keys
$(DOCKER_BUILD_CMD) --file $(DOCKERFILES)/contract-verifier/Dockerfile --load \
--tag contract-verifier:$(PROTOCOL_VERSION) $(CONTEXT)

build-server-v2: check-tools prepare-contracts prepare-keys
$(DOCKER_BUILD_CMD) --file $(DOCKERFILES)/server-v2/Dockerfile --load \
--tag server-v2:$(PROTOCOL_VERSION) $(CONTEXT)

build-circuit-prover-gpu: check-tools prepare-keys
$(DOCKER_BUILD_CMD) --file $(DOCKERFILES)/circuit-prover-gpu/Dockerfile --load \
--platform linux/amd64 \
--tag prover:$(PROTOCOL_VERSION) $(CONTEXT)

build-witness-generator: check-tools prepare-keys
$(DOCKER_BUILD_CMD) --file $(DOCKERFILES)/witness-generator/Dockerfile --load \
--tag witness-generator:$(PROTOCOL_VERSION) $(CONTEXT)


# Build all containers
build-all: build-contract-verifier build-server-v2 build-witness-generator build-circuit-prover-gpu cleanup

# Clean generated images
clean: cleanup
@ git submodule update --recursive
docker rmi contract-verifier:$$(PROTOCOL_VERSION)
docker rmi server-v2:$$(PROTOCOL_VERSION)-$$(PROTOCOL_VERSION)
docker rmi prover:$$(PROTOCOL_VERSION)
docker rmi witness-generator:$$(PROTOCOL_VERSION)

# Clean after build-all
cleanup:
@echo "Running cleanup..." && \
docker compose down -v >/dev/null 2>&1 && \
rm -rf ./volumes/postgres
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ The following questions will be answered by the following resources:
| What do I need to develop the project locally? | [development.md](docs/guides/development.md) |
| How can I set up my dev environment? | [setup-dev.md](docs/guides/setup-dev.md) |
| How can I run the project? | [launch.md](docs/guides/launch.md) |
| How can I build Docker images? | [build-docker.md](docs/guides/build-docker.md) |
| What is the logical project structure and architecture? | [architecture.md](docs/guides/architecture.md) |
| Where can I find protocol specs? | [specs.md](docs/specs/README.md) |
| Where can I find developer docs? | [docs](https://docs.zksync.io) |
Expand Down
44 changes: 44 additions & 0 deletions docs/guides/build-docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Build docker images

This document explains how to build docker images.
artmakh marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites

Install prerequisites: see

[Installing dependencies](./setup-dev.md)

## Build docker files

You may build all images with

```shell
make build-all
```

You will get those images:

```shell
contract-verifier:2.0
contract-server-v2:2.0
artmakh marked this conversation as resolved.
Show resolved Hide resolved
prover:2.0
witness-generator:2.0
```

Alternativly, you may build only needed components - available targets are
artmakh marked this conversation as resolved.
Show resolved Hide resolved

```shell
make build-contract-verifier
make build-server-v2
make build-circuit-prover-gpu
make build-witness-generator
```

## Building updated images

Simply run

```shell
make clean
make build-all
```
Loading