Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TT-765] Moves Client Compatability Tests to Nightly Run #11610

Merged
merged 14 commits into from
Dec 20, 2023
Prev Previous commit
Next Next commit
Link and docs
  • Loading branch information
kalverra committed Dec 20, 2023
commit 49ef7d791d203cb398dfce62e77f98c3137bbd3e
11 changes: 10 additions & 1 deletion .github/actions/notify-slack-jobs-result/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
# Notify Slack Jobs Result

Sends a Slack message to a specified channel detailing the results of one to many GHA job results using a regex.
Sends a Slack message to a specified channel detailing the results of one to many GHA job results using a regex. The job results will be grouped by the `github_job_name_regex` and displayed underneath the `message_title`, with the regex matching group displayed as an individual result. This is primarily designed for when you have test groups running in a matrix, and would like condensed reporting on their status by group. It's often accompanied by posting a Slack message before to start a thread, then attaching all the results to that thread like we do in the reporting section of the [live-testnet-test.yml workflow](../../workflows/live-testnet-tests.yml). Check out the example below, where we post an initial summary message, then use this action to thread together specific results:

```yaml
message_title: Optimism Goerli
github_job_name_regex: ^Optimism Goerli (?<cap>.*?) Tests$
```

![example](image.png)

## Inputs

```yaml
inputs:
Expand Down
6 changes: 3 additions & 3 deletions .github/actions/notify-slack-jobs-result/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,14 @@ runs:
# I feel like there's some clever, fully jq way to do this, but I ain't got the motivation to figure it out
echo "Querying test results at https://api.github.com/repos/${{inputs.github_repository}}/actions/runs/${{ inputs.workflow_run_id }}/jobs"

echo "Looking for jobs matching regex: `${{ inputs.github_job_name_regex }}`"
PARSED_RESULTS=$(curl \
-H "Authorization: Bearer ${{ inputs.github_token }}" \
'https://api.github.com/repos/${{inputs.github_repository}}/actions/runs/${{ inputs.workflow_run_id }}/jobs' \
| jq -r --arg pattern "${{ inputs.github_job_name_regex }}" '.jobs[]
| select(.name | test($pattern)) as $job
| $job.steps[]
| select(.name == "Run Tests")
| { conclusion: (if .conclusion == "success" then ":white_check_mark:" else ":x:" end), cap: ("*" + ($job.name | capture($pattern).cap) + "*") }')
| { conclusion: (if .conclusion == "success" then ":white_check_mark:" else ":x:" end), cap: ("*" + ($job.name | capture($pattern).cap) + "*"), html_url: $job.html_url }')

echo "Parsed Results:"
echo $PARSED_RESULTS
Expand All @@ -64,9 +63,10 @@ runs:
| {
conclusion: .conclusion,
cap: .cap
html_url: .html_url
}
]
| map("{\"type\": \"section\", \"text\": {\"type\": \"mrkdwn\", \"text\": \"\(.cap): \(.conclusion)\"}}")
| map("{\"type\": \"section\", \"text\": {\"type\": \"mrkdwn\", \"text\": \"<\(.html_url)|\(.cap)>: \(.conclusion)\"}}")
| join(",")')

echo "Formatted Results:"
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
78 changes: 0 additions & 78 deletions .github/workflows/client-compatibility-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -115,59 +115,6 @@ jobs:
else
echo "run_command=./smoke/${{ matrix.product.name }}_test.go" >> "$GITHUB_OUTPUT"
fi
- name: Check for "enable tracing" label
id: check-label
run: |
label=$(jq -r '.pull_request.labels[]?.name // empty' "$GITHUB_EVENT_PATH")

if [[ -n "$label" ]]; then
if [[ "$label" == "enable tracing" ]]; then
echo "Enable tracing label found."
echo "trace=true" >> $GITHUB_OUTPUT
else
echo "Enable tracing label not found."
echo "trace=false" >> $GITHUB_OUTPUT
fi
else
echo "No labels present or labels are null."
echo "trace=false" >> $GITHUB_OUTPUT
fi

- name: Setup Grafana and OpenTelemetry
id: docker-setup
if: steps.check-label.outputs.trace == 'true' && matrix.product.name == 'ocr2' && matrix.product.tag_suffix == '-plugins'
run: |
# Create network
docker network create --driver bridge tracing

# Make trace directory
cd integration-tests/smoke/
mkdir ./traces
chmod -R 777 ./traces

# Switch directory
cd ../../.github/tracing

# Create a Docker volume for traces
# docker volume create otel-traces

# Start OpenTelemetry Collector
# Note the user must be set to the same user as the runner for the trace data to be accessible
docker run -d --network=tracing --name=otel-collector \
-v $PWD/otel-collector-ci.yaml:/etc/otel-collector.yaml \
-v $PWD/../../integration-tests/smoke/traces:/tracing \
--user "$(id -u):$(id -g)" \
-p 4317:4317 otel/opentelemetry-collector:0.88.0 --config=/etc/otel-collector.yaml
- name: Locate Docker Volume
id: locate-volume
if: false
run: |
echo "VOLUME_PATH=$(docker volume inspect --format '{{ .Mountpoint }}' otel-traces)" >> $GITHUB_OUTPUT
- name: Show Otel-Collector Logs
if: steps.check-label.outputs.trace == 'true' && matrix.product.name == 'ocr2' && matrix.product.tag_suffix == '-plugins'
run: |
docker logs otel-collector
## Run this step when changes that require tests to be run are made
- name: Run Tests
uses: smartcontractkit/chainlink-github-actions/chainlink-testing-framework/run-tests@e865e376b8c2d594028c8d645dd6c47169b72974 # v2.2.16
env:
Expand Down Expand Up @@ -198,21 +145,6 @@ jobs:
QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }}
QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }}
QA_KUBECONFIG: ""
## Run this step when changes that do not need the test to run are made
- name: Run Setup
uses: smartcontractkit/chainlink-github-actions/chainlink-testing-framework/setup-run-tests-environment@e865e376b8c2d594028c8d645dd6c47169b72974 # v2.2.16
with:
test_download_vendor_packages_command: cd ./integration-tests && go mod download
go_mod_path: ./integration-tests/go.mod
cache_key_id: core-e2e-${{ env.MOD_CACHE_VERSION }}
cache_restore_only: "true"
QA_AWS_REGION: ${{ secrets.QA_AWS_REGION }}
QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }}
QA_KUBECONFIG: ${{ secrets.QA_KUBECONFIG }}
- name: Show Otel-Collector Logs
if: steps.check-label.outputs.trace == 'true' && matrix.product.name == 'ocr2' && matrix.product.tag_suffix == '-plugins'
run: |
docker logs otel-collector
- name: Collect Metrics
if: always()
id: collect-gha-metrics
Expand All @@ -223,16 +155,6 @@ jobs:
this-job-name: ETH Smoke Tests ${{ matrix.product.name }}${{ matrix.product.tag_suffix }}
test-results-file: '{"testType":"go","filePath":"/tmp/gotest.log"}'
continue-on-error: true
- name: Permissions on traces
if: steps.check-label.outputs.trace == 'true' && matrix.product.name == 'ocr2' && matrix.product.tag_suffix == '-plugins'
run: |
ls -l ./integration-tests/smoke/traces
- name: Upload Trace Data
if: steps.check-label.outputs.trace == 'true' && matrix.product.name == 'ocr2' && matrix.product.tag_suffix == '-plugins'
uses: actions/upload-artifact@v3
with:
name: trace-data
path: ./integration-tests/smoke/traces/trace-data.json
- name: Print failed test summary
if: always()
run: |
Expand Down
Loading