Skip to content

CircleCI: parameterizing no_output_timeout and -short #2618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
algojohnlee merged 7 commits into from
Jul 28, 2021
Merged

CircleCI: parameterizing no_output_timeout and -short #2618

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ceb2e29
test parameterizing no_output_timeout and -short
algobarb Jul 23, 2021
2f03503
add params to general integration
algobarb Jul 23, 2021
af3a01e
revert back to original relnightly filter
algobarb Jul 23, 2021
569c011
edit description
algobarb Jul 23, 2021
a98b5e1
revert back to med for integration
algobarb Jul 27, 2021
fe189aa
make sure arm64_integration is running short
algobarb Jul 27, 2021
57a0e74
rename go_test_flags to short_test_flag
algobarb Jul 27, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
148 changes: 46 additions & 102 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,60 +216,20 @@ commands:
keys:
- 'libsodium-fork-{{ .Environment.CIRCLE_STAGE }}-{{ checksum "tmp/libsodium.md5" }}'

generic_shorttest:
description: Run short tests from build workspace, for re-use by diferent architectures
generic_buildtest:
description: Run build tests from build workspace, for re-use by diferent architectures
parameters:
circleci_home:
type: string
default: "/home/circleci"
result_subdir:
type: string
steps:
- attach_workspace:
at: << parameters.circleci_home >>
- run: mkdir -p /tmp/results/<< parameters.result_subdir >>
- restore_cache:
keys:
- 'go-cache-{{ .Environment.CIRCLE_STAGE }}-'
- run:
name: Run short tests
no_output_timeout: 30m
command: |
set -e
set -x
export PATH=$(echo "$PATH" | sed -e 's|:<< parameters.circleci_home >>/\.go_workspace/bin||g' | sed -e 's|:/usr/local/go/bin||g')
export KMD_NOUSB=True
export GOPATH="<< parameters.circleci_home >>/go"
export PATH="${PATH}:${GOPATH}/bin"
export ALGORAND_DEADLOCK=enable
GOLANG_VERSION=$(./scripts/get_golang_version.sh)
eval "$(~/gimme "${GOLANG_VERSION}")"
scripts/configure_dev.sh
scripts/buildtools/install_buildtools.sh -o "gotest.tools/gotestsum"
PACKAGES="$(go list ./... | grep -v /go-algorand/test/)"
export PACKAGE_NAMES=$(echo $PACKAGES | tr -d '\n')
export PARTITION_TOTAL=$CIRCLE_NODE_TOTAL
export PARTITION_ID=$CIRCLE_NODE_INDEX
export PARALLEL_FLAG="-p 1"
gotestsum --format pkgname --junitfile /tmp/results/<< parameters.result_subdir >>/results.xml --jsonfile /tmp/results/<< parameters.result_subdir >>/testresults.json -- --tags "sqlite_unlock_notify sqlite_omit_load_extension" -short -race -timeout 1h -coverprofile=coverage.txt -covermode=atomic -p 1 $PACKAGE_NAMES
- store_artifacts:
path: /tmp/results
destination: test-results
- store_test_results:
path: /tmp/results
- save_cache:
key: 'go-cache-{{ .Environment.CIRCLE_STAGE }}-{{ .Environment.CIRCLE_BUILD_NUM }}'
paths:
- tmp/go-cache

generic_nightlytest:
description: Run nightly tests from build workspace, for re-use by diferent architectures
parameters:
circleci_home:
no_output_timeout:
type: string
default: "/home/circleci"
result_subdir:
default: 30m
short_test_flag:
Copy link
Contributor

@egieseke egieseke Jul 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this could be made more generic to support any test flag?

Copy link
Contributor Author

@algobarb algobarb Jul 27, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I want to do that anymore:

  1. We're calling a bunch of other parameters inside the test itself right now, so it could be confusing to add an any_test flag because you can't add whatever parameter you want. Only parameters that haven't been specified yet
  2. -short is the main flag we're using and it helps to repeat the same variable name used in generic_integration for consistency
  3. Will mentioned that he wants us to stop calling gotestsum directly in the file eventually, so I expect further modifications on this command. (1426 in our CircleCI epic)

type: string
default: ""
steps:
- attach_workspace:
at: << parameters.circleci_home >>
Expand All @@ -278,8 +238,8 @@ commands:
keys:
- 'go-cache-{{ .Environment.CIRCLE_STAGE }}-'
- run:
name: Run nightly tests
no_output_timeout: 45m
name: Run build tests
no_output_timeout: << parameters.no_output_timeout >>
command: |
set -e
set -x
Expand All @@ -297,7 +257,7 @@ commands:
export PARTITION_TOTAL=$CIRCLE_NODE_TOTAL
export PARTITION_ID=$CIRCLE_NODE_INDEX
export PARALLEL_FLAG="-p 1"
gotestsum --format pkgname --junitfile /tmp/results/<< parameters.result_subdir >>/results.xml --jsonfile /tmp/results/<< parameters.result_subdir >>/testresults.json -- --tags "sqlite_unlock_notify sqlite_omit_load_extension" -race -timeout 1h -coverprofile=coverage.txt -covermode=atomic -p 1 $PACKAGE_NAMES
gotestsum --format pkgname --junitfile /tmp/results/<< parameters.result_subdir >>/results.xml --jsonfile /tmp/results/<< parameters.result_subdir >>/testresults.json -- --tags "sqlite_unlock_notify sqlite_omit_load_extension" << parameters.short_test_flag >> -race -timeout 1h -coverprofile=coverage.txt -covermode=atomic -p 1 $PACKAGE_NAMES
- store_artifacts:
path: /tmp/results
destination: test-results
Expand All @@ -308,7 +268,6 @@ commands:
paths:
- tmp/go-cache


upload_coverage:
description: Collect coverage reports and upload them
steps:
Expand All @@ -326,52 +285,19 @@ commands:
default: "/home/circleci"
result_subdir:
type: string
steps:
- attach_workspace:
at: << parameters.circleci_home >>
- run: mkdir -p /tmp/results/<< parameters.result_subdir >>
- run:
name: Run integration tests
no_output_timeout: 30m
command: |
set -x
export PATH=$(echo "$PATH" | sed -e 's|:<< parameters.circleci_home >>/\.go_workspace/bin||g' | sed -e 's|:/usr/local/go/bin||g')
export KMD_NOUSB=True
export GOPATH="<< parameters.circleci_home >>/go"
export PATH="${PATH}:${GOPATH}/bin"
export ALGORAND_DEADLOCK=enable
export BUILD_TYPE=integration
GOLANG_VERSION=$(./scripts/get_golang_version.sh)
eval "$(~/gimme "${GOLANG_VERSION}")"
scripts/configure_dev.sh
scripts/buildtools/install_buildtools.sh -o "gotest.tools/gotestsum"
export ALGOTEST=1
export SHORTTEST=-short
export TEST_RESULTS=/tmp/results/<< parameters.result_subdir >>
export PARTITION_TOTAL=$CIRCLE_NODE_TOTAL
export PARTITION_ID=$CIRCLE_NODE_INDEX
test/scripts/run_integration_tests.sh
- store_artifacts:
path: /tmp/results
destination: test-results
- store_test_results:
path: /tmp/results

generic_nightlyintegration:
description: Run nightly integration tests from build workspace, for re-use by diferent architectures
parameters:
circleci_home:
no_output_timeout:
type: string
default: "/home/circleci"
result_subdir:
default: 30m
short_test_flag:
type: string
default: ""
steps:
- attach_workspace:
at: << parameters.circleci_home >>
- run: mkdir -p /tmp/results/<< parameters.result_subdir >>
- run:
name: Run integration tests
no_output_timeout: 45m
no_output_timeout: << parameters.no_output_timeout >>
command: |
set -x
export PATH=$(echo "$PATH" | sed -e 's|:<< parameters.circleci_home >>/\.go_workspace/bin||g' | sed -e 's|:/usr/local/go/bin||g')
Expand All @@ -385,7 +311,7 @@ commands:
scripts/configure_dev.sh
scripts/buildtools/install_buildtools.sh -o "gotest.tools/gotestsum"
export ALGOTEST=1
export SHORTTEST=-short
export SHORTTEST=<< parameters.short_test_flag >>
export TEST_RESULTS=/tmp/results/<< parameters.result_subdir >>
export PARTITION_TOTAL=$CIRCLE_NODE_TOTAL
export PARTITION_ID=$CIRCLE_NODE_INDEX
Expand Down Expand Up @@ -425,8 +351,9 @@ jobs:
parallelism: 4
steps:
- prepare_go
- generic_shorttest:
- generic_buildtest:
result_subdir: amd64-short
short_test_flag: "-short"
- upload_coverage

amd64_test_nightly:
Expand All @@ -436,8 +363,9 @@ jobs:
parallelism: 4
steps:
- prepare_go
- generic_nightlytest:
- generic_buildtest:
result_subdir: amd64-nightly
no_output_timeout: 45m
- upload_coverage

amd64_integration:
Expand All @@ -451,6 +379,7 @@ jobs:
- prepare_go
- generic_integration:
result_subdir: amd64-integration
short_test_flag: "-short"

amd64_integration_nightly:
machine:
Expand All @@ -461,8 +390,9 @@ jobs:
SKIP_E2E_SUBS: "true"
steps:
- prepare_go
- generic_nightlyintegration:
- generic_integration:
result_subdir: amd64-integrationnightly
no_output_timeout: 45m

amd64_e2e_subs:
machine:
Expand All @@ -474,6 +404,7 @@ jobs:
- prepare_go
- generic_integration:
result_subdir: amd64-e2e_subs
short_test_flag: "-short"

amd64_e2e_subs_nightly:
machine:
Expand All @@ -483,8 +414,9 @@ jobs:
E2E_SUBS_ONLY: "true"
steps:
- prepare_go
- generic_nightlyintegration:
- generic_integration:
result_subdir: amd64-e2e_subs_nightly
no_output_timeout: 45m

arm64_build:
machine:
Expand All @@ -503,8 +435,9 @@ jobs:
steps:
- checkout
- prepare_go
- generic_shorttest:
- generic_buildtest:
result_subdir: arm64-short
short_test_flag: "-short"
- upload_coverage

arm64_test_nightly:
Expand All @@ -515,8 +448,9 @@ jobs:
steps:
- checkout
- prepare_go
- generic_nightlytest:
- generic_buildtest:
result_subdir: arm64-nightly
no_output_timeout: 45m
- upload_coverage

arm64_integration:
Expand All @@ -531,6 +465,7 @@ jobs:
- prepare_go
- generic_integration:
result_subdir: arm64-integration
short_test_flag: "-short"

arm64_integration_nightly:
machine:
Expand All @@ -542,8 +477,9 @@ jobs:
steps:
- checkout
- prepare_go
- generic_nightlyintegration:
- generic_integration:
result_subdir: arm64-integration-nightly
no_output_timeout: 45m

arm64_e2e_subs:
machine:
Expand All @@ -556,6 +492,7 @@ jobs:
- prepare_go
- generic_integration:
result_subdir: arm64-e2e_subs
short_test_flag: "-short"

arm64_e2e_subs_nightly:
machine:
Expand All @@ -566,8 +503,9 @@ jobs:
steps:
- checkout
- prepare_go
- generic_nightlyintegration:
- generic_integration:
result_subdir: arm64-e2e_subs-nightly
no_output_timeout: 45m

mac_amd64_build:
macos:
Expand All @@ -592,9 +530,10 @@ jobs:
steps:
#- run: git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core fetch --unshallow
- prepare_go
- generic_shorttest:
- generic_buildtest:
result_subdir: mac-amd64-short
circleci_home: /Users/distiller
short_test_flag: "-short"
- upload_coverage

mac_amd64_test_nightly:
Expand All @@ -607,9 +546,10 @@ jobs:
steps:
#- run: git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core fetch --unshallow
- prepare_go
- generic_nightlytest:
- generic_buildtest:
result_subdir: mac-amd64-short
circleci_home: /Users/distiller
no_output_timeout: 45m
- upload_coverage

mac_amd64_integration:
Expand All @@ -626,6 +566,7 @@ jobs:
- generic_integration:
result_subdir: mac-amd64-integration
circleci_home: /Users/distiller
short_test_flag: "-short"

mac_amd64_integration_nightly:
macos:
Expand All @@ -638,9 +579,10 @@ jobs:
steps:
#- run: git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core fetch --unshallow
- prepare_go
- generic_nightlyintegration:
- generic_integration:
result_subdir: mac-amd64-integration-nightly
circleci_home: /Users/distiller
no_output_timeout: 45m

mac_amd64_e2e_subs:
macos:
Expand All @@ -652,9 +594,10 @@ jobs:
steps:
#- run: git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core fetch --unshallow
- prepare_go
- generic_nightlyintegration:
- generic_integration:
result_subdir: mac-amd64-e2e_subs
circleci_home: /Users/distiller
short_test_flag: "-short"

mac_amd64_e2e_subs_nightly:
macos:
Expand All @@ -666,9 +609,10 @@ jobs:
steps:
#- run: git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core fetch --unshallow
- prepare_go
- generic_nightlyintegration:
- generic_integration:
result_subdir: mac-amd64-e2e_subs-nightly
circleci_home: /Users/distiller
no_output_timeout: 45m

windows_x64_build:
executor:
Expand Down