Skip to content

Conversation

@nkuba
Copy link
Member

@nkuba nkuba commented Jul 11, 2022

We already download artifacts of deployed contracts from the NPM registry.
Here we add support for extracting addresses of contracts from the
artifacts. The address is extracted and stored in _address/<CONTRACT>
file and then embedded into the go code.

This solution is intended to be used for clients building for a specific environment. For example, if we set CONTRACTS_NPM_PACKAGE_TAG environment variable to goerli. It will download a package of contracts deployed on the Goerli network and extract addresses from it. The client will be built specifically for the Goerli network, without the need of configuring the addresses in a config file.

There will be a possibility to overwrite the addresses with a configuration file or environment variables.

Addresses extraction will happen on go generate ./... command execution along with the go bindings generation.

To test the solution for Goerli please run:
CONTRACTS_NPM_PACKAGE_TAG=goerli go generate./...

Please note that for development which is a default tag the addresses will be replaced with zeros, as the contracts artifacts that are published with development tags are intended only for ABIs generation, as the contracts are deployed locally in CI, and won't be accessible anywhere.

In a follow-up PR we will resolve contract addresses to initialize the chain handle.

@nkuba nkuba requested review from lukasz-zimnoch and pdyraga July 11, 2022 15:14
@nkuba nkuba self-assigned this Jul 11, 2022
@nkuba nkuba marked this pull request as draft July 12, 2022 12:11
nkuba added 6 commits July 12, 2022 19:50
We want to resolve the package to download with `npm view` as it checks
the registry for the current package that is tagged. When `npm pack` was
used it used cached version of the tagged package and wasn't downloading
the latest one.
We already download artifacts of deployed contracts from NPM registry.
Here we add support of extracting addresses of contracts from the
artifacts. The address is extracted and stored in `_address/<CONTRACT>`
file and then embeded into the go code.

This solution is intended to be used for clients building for specific
environment. For example if we set `CONTRACTS_NPM_PACKAGE_TAG`
environment variable to `goerli`. It will download a package of
contracts deployed on Goerli network and extract addresses from it. The
client will be build specific for Goerli network, without a need of
configuring the addresses in a config file.

There will be a possibility to overwrite the addresses with a
configuration file or environment variables.

Addresses extraction will happen on `go generate ./...` command
execution along with the go bindings generation.

To test the solution for Goerli please run:
`CONTRACTS_NPM_PACKAGE_TAG=goerli go generate./...`

Please note that for `development` which is a default tag the addresses
will be replaced with zeros, as the contracts artifacts that are published with
`development` tags are intended only for ABIs generation, as the
contracts are deployed locally in CI, and won't be accessible anywhere.
We cleanup always before running bindings generation as current
configuration of make doesn't work well with source artifacts that are
extracted from tar archive.
Added a json artifact reference as a prerequisite for binding
generation.
`jq` is required by go generator command.
The files are required to exist by gosec.
@nkuba nkuba force-pushed the resolve-addresses branch from 006aea5 to 01a746d Compare July 12, 2022 17:51
@nkuba nkuba marked this pull request as ready for review July 12, 2022 17:52
@pdyraga
Copy link
Member

pdyraga commented Jul 13, 2022

I am having problems testing it locally. Anything extra I should do?

❯ CONTRACTS_NPM_PACKAGE_TAG=goerli go generate ./...
rm -rf _contracts && mkdir _contracts
npm pack --silent \
                --pack-destination=_contracts \
                @keep-network/random-beacon@2.0.0-goerli.0 \
                | xargs -I{} tar -zxf _contracts/{} -C _contracts
rm -rf _address/*
rm -rf abi/*
rm -rf contract/*
mkdir tmp && mv cmd/cmd*.go tmp
rm -rf cmd/*
mv tmp/* cmd && rm -rf tmp
jq -jr .address /Users/piotr/git/keep-network/keep-core/pkg/chain/ethereum/beacon/gen/_contracts/package/artifacts/RandomBeacon.json > _address/RandomBeacon
jq -jr .address /Users/piotr/git/keep-network/keep-core/pkg/chain/ethereum/beacon/gen/_contracts/package/artifacts/BeaconSortitionPool.json > _address/BeaconSortitionPool
jq .abi /Users/piotr/git/keep-network/keep-core/pkg/chain/ethereum/beacon/gen/_contracts/package/artifacts/RandomBeacon.json > abi/RandomBeacon.abi
go run github.com/ethereum/go-ethereum/cmd/abigen --abi abi/RandomBeacon.abi --pkg abi --type RandomBeacon --out abi/RandomBeacon.go
go run github.com/keep-network/keep-common/tools/generators/ethlike abi/RandomBeacon.abi contract/RandomBeacon.go cmd/RandomBeacon.go
jq .abi /Users/piotr/git/keep-network/keep-core/pkg/chain/ethereum/beacon/gen/_contracts/package/artifacts/BeaconSortitionPool.json > abi/BeaconSortitionPool.abi
go run github.com/ethereum/go-ethereum/cmd/abigen --abi abi/BeaconSortitionPool.abi --pkg abi --type BeaconSortitionPool --out abi/BeaconSortitionPool.go
go run github.com/keep-network/keep-common/tools/generators/ethlike abi/BeaconSortitionPool.abi contract/BeaconSortitionPool.go cmd/BeaconSortitionPool.go
rm -rf _contracts && mkdir _contracts
npm pack --silent \
                --pack-destination=_contracts \
                 \
                | xargs -I{} tar -zxf _contracts/{} -C _contracts
rm -rf _address/*
rm -rf abi/*
rm -rf contract/*
mkdir tmp && mv cmd/cmd*.go tmp
rm -rf cmd/*
mv tmp/* cmd && rm -rf tmp
make: *** No rule to make target `_address/WalletRegistry', needed by `all'.  Stop.
pkg/chain/ethereum/ecdsa/gen/gen.go:8: running "make": exit status 2
rm -rf _contracts && mkdir _contracts
npm pack --silent \
                --pack-destination=_contracts \
                 \
                | xargs -I{} tar -zxf _contracts/{} -C _contracts
rm -rf _address/*
rm -rf abi/*
rm -rf contract/*
mkdir tmp && mv cmd/cmd*.go tmp
rm -rf cmd/*
mv tmp/* cmd && rm -rf tmp
make: *** No rule to make target `_address/Bridge', needed by `all'.  Stop.
pkg/chain/ethereum/tbtc/gen/gen.go:8: running "make": exit status 2
generate: open /Users/piotr/git/keep-network/keep-core/pkg/chain/ethereum/tbtc/gen/abi/Bridge.go: no such file or directory

@nkuba
Copy link
Member Author

nkuba commented Jul 13, 2022

I am having problems testing it locally. Anything extra I should do?

❯ CONTRACTS_NPM_PACKAGE_TAG=goerli go generate ./...
rm -rf _contracts && mkdir _contracts
npm pack --silent \
                --pack-destination=_contracts \
                @keep-network/random-beacon@2.0.0-goerli.0 \
                | xargs -I{} tar -zxf _contracts/{} -C _contracts
rm -rf _address/*
rm -rf abi/*
rm -rf contract/*
mkdir tmp && mv cmd/cmd*.go tmp
rm -rf cmd/*
mv tmp/* cmd && rm -rf tmp
jq -jr .address /Users/piotr/git/keep-network/keep-core/pkg/chain/ethereum/beacon/gen/_contracts/package/artifacts/RandomBeacon.json > _address/RandomBeacon
jq -jr .address /Users/piotr/git/keep-network/keep-core/pkg/chain/ethereum/beacon/gen/_contracts/package/artifacts/BeaconSortitionPool.json > _address/BeaconSortitionPool
jq .abi /Users/piotr/git/keep-network/keep-core/pkg/chain/ethereum/beacon/gen/_contracts/package/artifacts/RandomBeacon.json > abi/RandomBeacon.abi
go run github.com/ethereum/go-ethereum/cmd/abigen --abi abi/RandomBeacon.abi --pkg abi --type RandomBeacon --out abi/RandomBeacon.go
go run github.com/keep-network/keep-common/tools/generators/ethlike abi/RandomBeacon.abi contract/RandomBeacon.go cmd/RandomBeacon.go
jq .abi /Users/piotr/git/keep-network/keep-core/pkg/chain/ethereum/beacon/gen/_contracts/package/artifacts/BeaconSortitionPool.json > abi/BeaconSortitionPool.abi
go run github.com/ethereum/go-ethereum/cmd/abigen --abi abi/BeaconSortitionPool.abi --pkg abi --type BeaconSortitionPool --out abi/BeaconSortitionPool.go
go run github.com/keep-network/keep-common/tools/generators/ethlike abi/BeaconSortitionPool.abi contract/BeaconSortitionPool.go cmd/BeaconSortitionPool.go
rm -rf _contracts && mkdir _contracts
npm pack --silent \
                --pack-destination=_contracts \
                 \
                | xargs -I{} tar -zxf _contracts/{} -C _contracts
rm -rf _address/*
rm -rf abi/*
rm -rf contract/*
mkdir tmp && mv cmd/cmd*.go tmp
rm -rf cmd/*
mv tmp/* cmd && rm -rf tmp
make: *** No rule to make target `_address/WalletRegistry', needed by `all'.  Stop.
pkg/chain/ethereum/ecdsa/gen/gen.go:8: running "make": exit status 2
rm -rf _contracts && mkdir _contracts
npm pack --silent \
                --pack-destination=_contracts \
                 \
                | xargs -I{} tar -zxf _contracts/{} -C _contracts
rm -rf _address/*
rm -rf abi/*
rm -rf contract/*
mkdir tmp && mv cmd/cmd*.go tmp
rm -rf cmd/*
mv tmp/* cmd && rm -rf tmp
make: *** No rule to make target `_address/Bridge', needed by `all'.  Stop.
pkg/chain/ethereum/tbtc/gen/gen.go:8: running "make": exit status 2
generate: open /Users/piotr/git/keep-network/keep-core/pkg/chain/ethereum/tbtc/gen/abi/Bridge.go: no such file or directory

For goerli only threshold network and random beacon generation will work. We need to publish contracts first. I suggest in this PR we test just the development network.

@pdyraga
Copy link
Member

pdyraga commented Jul 13, 2022

For goerli only threshold network and random beacon generation will work. We need to publish contracts first. I suggest in this PR we test just the development network.

OK, this is fine. I tested with just go generate ./... and can confirm address extraction for development package is skipped. Also, when running for Goerli, the extraction was able to extract staking contract and random beacon contract addresses. Let's fix the problem with .gitignore and I am good to merge.

@nkuba nkuba requested a review from pdyraga July 13, 2022 16:18
The directory no longer exist.
@nkuba nkuba requested a review from pdyraga July 13, 2022 16:45
The directories have missing `_address/*` files so gosec fails. We could
put the files into repo or generate them, but also just exclude the
directories.
@pdyraga pdyraga added this to the v2.0.0-m1 milestone Jul 13, 2022
@pdyraga pdyraga merged commit fbdb910 into main Jul 13, 2022
@pdyraga pdyraga deleted the resolve-addresses branch July 13, 2022 20:15
@pdyraga pdyraga removed the request for review from lukasz-zimnoch July 15, 2022 12:11
@pdyraga pdyraga removed this from the v2.0.0-m0 milestone Sep 30, 2022
@pdyraga pdyraga added this to the v2.0.0-m4 milestone Sep 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants