Skip to content

Commit 7a572d8

Browse files
yzang2019Yiming Zang
and
Yiming Zang
authored
[CI] Add integration test framework for sei chain (#863)
* Add integration test framework for sei chain * Add template link * Add a simple example * Fix workflow file * Fix bash * Test * Fix if condition * Turn off TTY * Use python eval * Add python3 * Add debug log * Remove -ti * Fix one integration test * Fix one integration test * Fix one integration test * Fix one integration test * Fix a typo --------- Co-authored-by: Yiming Zang <yzang@twitter.com>
1 parent f9e682b commit 7a572d8

20 files changed

+422
-407
lines changed

.github/workflows/integration-test.yml

+18-9
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,13 @@ jobs:
2222
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
2323
steps:
2424
- uses: actions/checkout@v3
25+
- uses: actions/setup-python@v4
26+
with:
27+
python-version: '3.10'
28+
29+
- name: Pyyaml
30+
run: |
31+
pip3 install pyyaml
2532
2633
- name: Set up Go
2734
uses: actions/setup-go@v3
@@ -37,19 +44,21 @@ jobs:
3744
do
3845
sleep 10
3946
done
40-
sleep 5
47+
sleep 10
4148
4249
- name: Verify Sei Chain is able to start up
43-
run: docker exec -i sei-node-0 integration_test/startup/startup_test.sh
50+
run: python3 integration_test/scripts/runner.py integration_test/startup/startup_test.yaml
4451

4552
- name: Testing Dex Module
46-
run: docker exec -i sei-node-0 integration_test/dex_module/place_order_test.sh
53+
run: |
54+
docker exec sei-node-0 integration_test/contracts/deploy_dex_contract.sh mars
55+
python3 integration_test/scripts/runner.py integration_test/dex_module/place_order_test.yaml
4756
4857
- name: Testing Wasm Module
4958
run: |
50-
docker exec -i sei-node-0 integration_test/contracts/deploy_timelocked_token_contract.sh
51-
docker exec -i sei-node-0 integration_test/wasm_module/timelocked_token_test_1.sh
52-
docker exec -i sei-node-0 integration_test/contracts/deploy_timelocked_token_contract.sh
53-
docker exec -i sei-node-0 integration_test/wasm_module/timelocked_token_test_2.sh
54-
docker exec -i sei-node-0 integration_test/contracts/deploy_timelocked_token_contract.sh migration
55-
docker exec -i sei-node-0 integration_test/wasm_module/timelocked_token_test_3.sh
59+
docker exec sei-node-0 integration_test/contracts/deploy_timelocked_token_contract.sh
60+
python3 integration_test/scripts/runner.py integration_test/wasm_module/timelocked_token_delegation_test.yaml
61+
python3 integration_test/scripts/runner.py integration_test/wasm_module/timelocked_token_admin_test.yaml
62+
python3 integration_test/scripts/runner.py integration_test/wasm_module/timelocked_token_withdraw_test.yaml
63+
docker exec sei-node-0 integration_test/contracts/deploy_timelocked_token_contract.sh
64+
python3 integration_test/scripts/runner.py integration_test/wasm_module/timelocked_token_emergency_withdraw_test.yaml

.gitignore

-1
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,6 @@ parallelization/**/Cargo.lock
4141

4242
# integration test resources
4343
integration_test/**/*.txt
44-
integration_test/**/*.json
4544

4645
# Coverage Files
4746
coverage.html

docker/localnode/Dockerfile

+4-1
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@ FROM ubuntu:latest
22
RUN apt-get update && \
33
apt-get install -y make git golang jq python3 curl vim
44
RUN rm -rf build/generated
5+
SHELL ["/bin/bash", "-c"]
6+
57

68
VOLUME [ "/sei-protocol" ]
79
VOLUME [ "/root/go/pkg/mod" ]
@@ -21,4 +23,5 @@ COPY scripts/step2_genesis.sh /usr/bin/genesis.sh
2123
COPY scripts/step3_add_validator_to_genesis.sh /usr/bin/add_validator_to_gensis.sh
2224
COPY scripts/step4_persistent_peers.sh /usr/bin/persistent_peers.sh
2325
COPY scripts/step5_start_sei.sh /usr/bin/start_sei.sh
24-
COPY scripts/step6_start_price_feeder.sh /usr/bin/start_price_feeder.sh
26+
COPY scripts/step6_start_price_feeder.sh /usr/bin/start_price_feeder.sh
27+
ENV PATH "$PATH:$HOME/go/bin"

integration_test/README.md

+76
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
# Integration Test Framework
2+
This page provides an overview of how to use the testing framework
3+
to quickly add new integration test cases.
4+
5+
## Getting Started
6+
These instructions will help you set up the integration test framework
7+
on your local machine for development and testing purposes.
8+
9+
### Prerequisites
10+
- An up-to-date Python 3.x
11+
- Pyyaml install (pip3 install pyyaml)
12+
- Docker and docker compose installed and running
13+
14+
### Usage
15+
1. Ensure docker containers are up and running: `make docker-cluster-start`
16+
2. Execute the tests with this command: `python3 integration_test/scripts/runner.py test.yaml`
17+
18+
## Writing Tests
19+
Each integration test is defined in a YAML file under its specific module folder under the integration_test directory
20+
21+
There's a template yaml file which you can copy from to start with: [template](https://github.com/sei-protocol/sei-chain/tree/master/integration_test/template/template_test.yaml)
22+
23+
A typical yaml test case would look like this:
24+
```yaml
25+
- name: <Replace with test description>
26+
inputs:
27+
# Add comments for what this command is doing
28+
- cmd: <Replace with bash command>
29+
env: <Add if you want to store the output as an env variable>
30+
node: <Optional, default is sei-node-0>
31+
# Add comments for what this command is doing
32+
- cmd: <Replace with bash command>
33+
env: RESULT
34+
verifiers:
35+
# Add comments for what should the expected result
36+
- type: eval
37+
expr: <Replace with a valid python eval>
38+
- type: regex
39+
result: RESULT
40+
expr: <Replace with regular expression>
41+
```
42+
43+
One simple example for verify chain is started and running fine:
44+
```yaml
45+
- name: Test number of validators should be equal to 4
46+
inputs:
47+
# Query num of validators
48+
- cmd: seid q tendermint-validator-set |grep address |wc -l
49+
env: RESULT
50+
verifiers:
51+
- type: eval
52+
expr: RESULT == 4
53+
```
54+
55+
### Explanation
56+
57+
| field_name | required | description |
58+
|------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------|
59+
| name | Yes | Defines the purpose of the test case . |
60+
| inputs | Yes | Contains a list of command inputs to run one by one. |
61+
| cmd | Yes | Exact seid or bash command to run. |
62+
| env | No | If given, the command output will be persisted to this env variable, which can be referenced by all below commands |
63+
| node | No | If given, the command will be executed on a specific container, default to sei-node-0 |
64+
| verifiers | Yes | Contains a list of verify functions to check correctness |
65+
| type | Yes | Currently support either `eval` or `regex`. |
66+
| result | Yes | Pick any env variables you want to pass in for regex match |
67+
| expr | Yes | If type is eval, then the format is `[env] >\|==\|!=\|>=\|>\|<=\|< [number]` <br/> If type is regex, then provide a valid regular expression. | |
68+
69+
### Notes & Tips
70+
There are some tricks and tips you should know when adding a new test case:
71+
1. Try to avoid using sing quote `'` in your command as much as possible, use `"` to replace whenever possible
72+
2. Sometimes you need to escape `"` and make it `\"`
73+
3. Use jq expressions to simplify the output and make your verification logic easier
74+
4. Commands will be executed one by one and will be wrapped within `docker exec -ti`
75+
5. Chain is keep running and is stateful, so some tests might not be idempotent which is fine
76+
6. You can define more than one verifier and each one check a different env

integration_test/contracts/deploy_dex_contract.sh

+2-1
Original file line numberDiff line numberDiff line change
@@ -31,8 +31,9 @@ printf "12345678\n" | $seidbin tx dex register-contract "$contract_addr" "$contr
3131

3232
echo '{"batch_contract_pair":[{"contract_addr":"'$contract_addr'","pairs":[{"price_denom":"SEI","asset_denom":"ATOM","price_tick_size":"0.0000001", "quantity_tick_size":"0.0000001"}]}]}' > integration_test/contracts/"$contract_name"-pair.json
3333
contract_pair=$(printf "12345678\n" | $seidbin tx dex register-pairs integration_test/contracts/"$contract_name"-pair.json -y --from=$keyname --chain-id=$chainid --fees=10000000usei --gas=500000 --broadcast-mode=block --output=json)
34+
rm -rf integration_test/contracts/"$contract_name"-pair.json
3435

35-
sleep 15s
36+
sleep 10s
3637

3738
echo "Deployed contracts:"
3839
echo "$contract_addr"

integration_test/contracts/deploy_timelocked_token_contract.sh

+7-7
Original file line numberDiff line numberDiff line change
@@ -22,13 +22,13 @@ key_admin4=$(printf "12345678\n" |$seidbin keys show admin4 -a)
2222
key_op=$(printf "12345678\n" |$seidbin keys show op -a)
2323
key_staking=$(printf "12345678\n" |$seidbin keys show staking_reward_dest -a)
2424
key_unlock=$(printf "12345678\n" |$seidbin keys show unlocked_dest -a)
25-
printf "12345678\n" | $seidbin tx bank send admin "$key_admin1" 10000000usei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
26-
printf "12345678\n" | $seidbin tx bank send admin "$key_admin2" 10000000usei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
27-
printf "12345678\n" | $seidbin tx bank send admin "$key_admin3" 10000000usei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
28-
printf "12345678\n" | $seidbin tx bank send admin "$key_admin4" 10000000usei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
29-
printf "12345678\n" | $seidbin tx bank send admin "$key_op" 10000000usei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
30-
printf "12345678\n" | $seidbin tx bank send admin "$key_staking" 10000000usei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
31-
printf "12345678\n" | $seidbin tx bank send admin "$key_unlock" 10000000usei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
25+
printf "12345678\n" | $seidbin tx bank send admin "$key_admin1" 10000000sei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
26+
printf "12345678\n" | $seidbin tx bank send admin "$key_admin2" 10000000sei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
27+
printf "12345678\n" | $seidbin tx bank send admin "$key_admin3" 10000000sei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
28+
printf "12345678\n" | $seidbin tx bank send admin "$key_admin4" 10000000sei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
29+
printf "12345678\n" | $seidbin tx bank send admin "$key_op" 10000000sei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
30+
printf "12345678\n" | $seidbin tx bank send admin "$key_staking" 10000000sei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
31+
printf "12345678\n" | $seidbin tx bank send admin "$key_unlock" 10000000sei -y --chain-id=$chainid --gas=5000000 --fees=1000000usei --broadcast-mode=block
3232

3333

3434
# Deploy goblin contract
Binary file not shown.

integration_test/dex_module/place_order_test.sh

-25
This file was deleted.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
- name: Test placing a new order and query the placed order
2+
inputs:
3+
# Get contract address, this requires contrat already deployed
4+
- cmd: tail -1 integration_test/contracts/mars-addr.txt
5+
env: CONTRACT_ADDR
6+
# Prepare parameter
7+
- cmd: echo "LONG?1.01?5?SEI?ATOM?LIMIT?{\"leverage\":\"1\",\"position_effect\":\"Open\"}"
8+
env: PARAMS
9+
# Place an order and set ORDER_ID
10+
- cmd: printf "12345678\n" | seid tx dex place-orders $CONTRACT_ADDR $PARAMS --amount=1000000000usei -y --from=admin --chain-id=sei --fees=1000000usei --gas=50000000 --broadcast-mode=block --output json|jq -M -r ".logs[].events[].attributes[] | select(.key == \"order_id\").value"
11+
env: ORDER_ID
12+
# Query the order by id
13+
- cmd: seid q dex get-orders-by-id $CONTRACT_ADDR SEI ATOM $ORDER_ID --output json |jq .order.status
14+
env: RESULT
15+
verifiers:
16+
# Order id should be greater or equal to 0
17+
- type: eval
18+
expr: ORDER_ID >= 0
19+
# Order status should be something like PLACED
20+
- type: regex
21+
result: RESULT
22+
expr: '^.*PLACED.*'

integration_test/scripts/runner.py

+96
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
import argparse
2+
import re
3+
import subprocess
4+
import yaml
5+
6+
7+
class TestRunner:
8+
9+
def __init__(self):
10+
pass
11+
12+
def load_yaml_file(self, filepath):
13+
with open(filepath, 'r') as f:
14+
data = yaml.safe_load(f)
15+
return data
16+
17+
# Function to process YAML
18+
def process_data(self, data):
19+
for test in data:
20+
self.run_test(test)
21+
22+
# Function to execute a single test case
23+
def run_test(self, test):
24+
test_name = test["name"]
25+
print("\n========== " + test_name + " =========")
26+
inputs = test["inputs"]
27+
env_map = {}
28+
for input in inputs:
29+
cmd = input['cmd']
30+
container = input.get("node", "sei-node-0")
31+
output = self.run_bash_command(cmd, True, container, env_map, False)
32+
if input.get('env'):
33+
env_map[input['env']] = output
34+
result = output
35+
print(f'Input : {cmd}')
36+
print(f'Output: {result}')
37+
for verifier in test["verifiers"]:
38+
if not self.verify_result(env_map, verifier):
39+
print("Test failed for {}".format(verifier))
40+
exit(1)
41+
print("Test Passed")
42+
43+
# Function to verify the result of a single test case
44+
def verify_result(self, env_map, verifier):
45+
type = verifier["type"]
46+
if type == "eval":
47+
elems = verifier["expr"].strip().split()
48+
if len(elems) == 3:
49+
if env_map.get(elems[0]):
50+
elems[0] = env_map[elems[0]]
51+
if env_map.get(elems[2]):
52+
elems[2] = env_map[elems[2]]
53+
expr = f'{elems[0]} {elems[1]} {elems[2]}'
54+
return eval(expr)
55+
return False
56+
elif type == "regex":
57+
env_key = verifier["result"]
58+
expression = str(verifier["expr"])
59+
result = env_map[env_key].strip()
60+
return re.match(expression, result)
61+
else:
62+
return False
63+
64+
# Helper function to execute a single command within the docker container
65+
def run_bash_command(self, command, docker=False, container="", env_map=None, verbose=False):
66+
if docker:
67+
envs = ""
68+
if env_map:
69+
for key in env_map:
70+
envs += f'-e {key}=\'{env_map[key]}\' '
71+
full_cmd = f'docker exec {envs} {container} /bin/bash -c \'export PATH=$PATH:/root/go/bin && {command}\''
72+
else:
73+
full_cmd = command
74+
if verbose:
75+
print(f'Command: {full_cmd}')
76+
process = subprocess.Popen(full_cmd, stdout=subprocess.PIPE, shell=True)
77+
output, error = process.communicate()
78+
return output.decode().strip()
79+
80+
81+
def main():
82+
parser = argparse.ArgumentParser(
83+
description='Integration test runner',
84+
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
85+
)
86+
parser.add_argument('filepath', type=str, help='The path to the JSON file')
87+
args = parser.parse_args()
88+
runner = TestRunner()
89+
data = runner.load_yaml_file(args.filepath)
90+
runner.process_data(data)
91+
92+
93+
if __name__ == '__main__':
94+
main()
95+
96+

integration_test/startup/startup_test.sh

-23
This file was deleted.
+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
- name: Test number of validators should be equal to 4
2+
inputs:
3+
# Query num of validators
4+
- cmd: seid q tendermint-validator-set |grep address |wc -l
5+
env: RESULT
6+
verifiers:
7+
- type: eval
8+
expr: RESULT == 4
9+
10+
- name: Test block height should be greater than 0
11+
inputs:
12+
# Get the current block height
13+
- cmd: seid status |jq -M -r .SyncInfo.latest_block_height
14+
env: RESULT
15+
verifiers:
16+
- type: eval
17+
expr: RESULT > 0
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
- name: <Replace with test description>
2+
inputs:
3+
# Add comments for what this command is doing
4+
- cmd: <Replace with bash command>
5+
env: <Add if you want to store the output as an env variable>
6+
node: <Optional, default is sei-node-0>
7+
# Add comments for what this command is doing
8+
- cmd: <Replace with bash command>
9+
env: RESULT
10+
verifiers:
11+
# Add comments for what should the expected result
12+
- type: eval
13+
expr: <Replace with a valid python eval>
14+
- type: regex
15+
result: RESULT
16+
expr: <Replace with regular expression>

0 commit comments

Comments
 (0)