Skip to content

Commit

Permalink
Merge branch 'v2.x/staging' into feature/v3/jcl
Browse files Browse the repository at this point in the history
  • Loading branch information
1000TurquoisePogs authored Jan 30, 2024
2 parents 94a9151 + 453dd69 commit 426e2b0
Show file tree
Hide file tree
Showing 64 changed files with 810 additions and 437 deletions.
2 changes: 1 addition & 1 deletion .dependency/zwe_doc_generation/md-content.js
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ function generateDocumentationForNode(curNode, assembledParentNode) {
docContent += createDocContent(assembledDocNode[docType.fileName].content, docType);
const parentDocContent = createDocContent(assembledDocNode[docType.fileName].parentContent, docType);
if (parentDocContent) {
docContent += SUB_SECTION_HEADER_PREFIX + 'Inherited from parent command' + SEPARATOR + parentDocContent;
docContent += SEPARATOR + SUB_SECTION_HEADER_PREFIX + 'Inherited from parent command' + SEPARATOR + parentDocContent;
}
}

Expand Down
17 changes: 5 additions & 12 deletions .github/scripts/cicd_test/make_matrix.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,17 +32,9 @@ case $install_test_choice in
test_file="$KEYRING_TESTFILE"
;;

"z/OS node v14")
test_file="$ZOS_NODE_V14_TESTFILE"
;;

"z/OS node v16")
test_file="$ZOS_NODE_V16_TESTFILE"
;;

"z/OS node v18")
test_file="$ZOS_NODE_V18_TESTFILE"
test_force_system="zzow04"
test_force_system="zzow08"
;;

"Non-strict Verify External Certificate")
Expand Down Expand Up @@ -91,17 +83,18 @@ if [[ ! -z "$test_force_system" ]]; then
else
if [[ -z "$dont_parse_test_server" ]]; then
if [[ "$test_server" == "Any zzow servers" ]]; then
test_server="zzow0"$(echo $(($RANDOM % 3 + 2)))
test_server="zzow0"$(echo $(($RANDOM % 3 + 6)))
fi
TEST_FILE_SERVER="$test_file($test_server)"
else
any_occurrence=$(echo $test_file | grep -o "(any)" | wc -l)
interim_test_file_server=$test_file
for i in $(seq $any_occurrence); do
interim_test_file_server=$(echo $interim_test_file_server | sed "s#(any)#(zzow0$(echo $(($RANDOM % 3 + 2))))#")
# Generates zzow06, zzow07, zzow08
interim_test_file_server=$(echo $interim_test_file_server | sed "s#(any)#(zzow0$(echo $(($RANDOM % 3 + 6))))#")
done

TEST_FILE_SERVER=$(echo $interim_test_file_server | sed "s#(all)#(zzow02,zzow03,zzow04)#g")
TEST_FILE_SERVER=$(echo $interim_test_file_server | sed "s#(all)#(zzow06,zzow07,zzow08)#g")
fi
fi

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,24 +14,24 @@ TEST_SERVER=$(echo "$MATRIX_SERVER" | cut -d "-" -f2)

case $TEST_SERVER in

"zzow02")
TEST_SERVER_NICKNAME=marist-2
;;
"zzow06")
TEST_SERVER_NICKNAME=marist-6
;;

"zzow03")
TEST_SERVER_NICKNAME=marist-3
;;
"zzow07")
TEST_SERVER_NICKNAME=marist-7
;;

"zzow04")
TEST_SERVER_NICKNAME=marist-4
;;
"zzow08")
TEST_SERVER_NICKNAME=marist-8
;;

*)
printf "${RED}[Check 2 ERROR] Something went wrong when parsing test server nickname\n"
exit 1
;;
printf "${RED}[Check 2 ERROR] Something went wrong when parsing test server nickname\n"
exit 1
;;
esac

assert_env_var "TEST_SERVER"
assert_env_var "TEST_SERVER_NICKNAME"
printf "${GREEN}[Check 2/$TOTAL_CHECK] Test server name processing complete!${NC}\n"
printf "${GREEN}[Check 2/$TOTAL_CHECK] Test server name processing complete!${NC}\n"
5 changes: 3 additions & 2 deletions .github/workflows/build-packaging.yml
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ jobs:
with:
manifest-file-path: ${{ github.workspace }}/manifest.json
default-target-path: .pax/binaryDependencies/
expected-count: 29
expected-count: 30

# this step is not doing a publish, we are just utilizing this actions to get the PUBLISH_TARGET_PATH,
# and it will be used in the next step: [Download 3] Download SMPE build log
Expand Down Expand Up @@ -236,7 +236,7 @@ jobs:
with:
lock-repository: ${{ github.repository }}
github-token: ${{ secrets.GITHUB_TOKEN }}
lock-resource-name: zowe-psi-build-zzow03-lock
lock-resource-name: zowe-psi-build-zzow07-lock
lock-avg-retry-interval: 60

- name: '[SMPE Pax 4] Build PSWI'
Expand All @@ -249,6 +249,7 @@ jobs:
env:
ZOSMF_USER: ${{ secrets.ZOWE_PSWI_BUILD_USR }}
ZOSMF_PASS: ${{ secrets.ZOWE_PSWI_BUILD_PASSWD }}
ZZOW_SSH_PORT: ${{ secrets.SSH_MARIST_ALLSYS_PORT }}
VERSION: ${{ env.P_VERSION }}

- name: '[K8S] Build Kubernetes'
Expand Down
14 changes: 5 additions & 9 deletions .github/workflows/cicd-test-readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ This guide will describe how you should input into Github Actions workflow input

Currently we support three testing z/OS servers:

- zzow02 (ACF2)
- zzow03 (Top Secret/TSS)
- zzow04 (RACF)
- zzow06 (ACF2)
- zzow07 (Top Secret/TSS)
- zzow08 (RACF)

Testing pipeline is running tests in parallel. The workflow will try to acquire the resource lock if available. If the resource lock is occupied, the workflow will wait until the lock is succesfully acquired.

Expand All @@ -17,7 +17,7 @@ Workflow trigger is at [cicd-test](https://github.com/zowe/zowe-install-packagin
### Choose Test Server

- This input is a choice, and it's mandatory.
- You can choose from one of `zzow02`, `zzow03`, `zzow04`, `zzow02,zzow03,zzow04` (if you want to run the test on all zzow servers), or `Any zzow servers` (pick any zzow servers, potentially help reduce wait time)
- You can choose from one of `zzow06`, `zzow07`, `zzow08`, `zzow06,zzow07,zzow08` (if you want to run the test on all zzow servers), or `Any zzow servers` (pick any zzow servers, potentially help reduce wait time)
- Default is `Any zzow servers`

### Choose Install Test
Expand All @@ -29,8 +29,6 @@ Workflow trigger is at [cicd-test](https://github.com/zowe/zowe-install-packagin
- SMPE PTF
- Extensions
- Keyring
- z/OS node v14
- z/OS node v16
- z/OS node v18
- Non-strict Verify External Certificate
- Install PTF twice
Expand Down Expand Up @@ -111,7 +109,7 @@ Background: CICD testing relies on a `zowe.pax` or `zowe-smpe.zip` (for SMPE ins
When running CICD integration tests during RC stage, the following string will be parsed into the Github Actions matrix. As a result, a total of 21 independent jobs will be spawned.

```
basic/install.ts(zzow02,zzow03,zzow04);basic/install-ptf.ts(zzow02,zzow03,zzow04);basic/install-ext.ts(zzow03);extended/keyring.ts(zzow02,zzow03,zzow04);extended/node-versions/node-v8.ts(zzow02,zzow03,zzow04);extended/node-versions/node-v14.ts(zzow02,zzow03,zzow04);extended/certificates/nonstrict-verify-external-certificate.ts(zzow02)
basic/install.ts(zzow06,zzow07,zzow08);basic/install-ptf.ts(zzow06,zzow07,zzow08);basic/install-ext.ts(zzow07);extended/keyring.ts(zzow06,zzow07,zzow08);extended/node-versions/node-v18.ts(zzow06,zzow07,zzow08);extended/certificates/nonstrict-verify-external-certificate.ts(zzow06)
```

Total elapsed time when running in parallel is approximately 3.5 hours on paper idealy if all parallel jobs are executing at the same time. In reality, from numerous tests performed, total elapsed time is around 4 hours.
Expand All @@ -124,8 +122,6 @@ Selected test running elapsed time:
| Convenience Pax | 53m |
| SMPE PTF | 68m |
| z/OS node v18 | 45m |
| z/OS node v16 | 45m |
| z/OS node v14 | 45m |
| Keyring | 53m |
| Non-strict Verify External Certificate | 51m |
| Extensions | 67m
Expand Down
26 changes: 8 additions & 18 deletions .github/workflows/cicd-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ on:
default: 'Any zzow servers'
options:
- Any zzow servers
- zzow02
- zzow03
- zzow04
- zzow02,zzow03,zzow04
- zzow06
- zzow07
- zzow08
- zzow06,zzow07,zzow08
install-test:
description: 'Choose Install Test'
type: choice
Expand All @@ -24,8 +24,6 @@ on:
- SMPE PTF
- Extensions
- Keyring
- z/OS node v14
- z/OS node v16
- z/OS node v18
- Non-strict Verify External Certificate
- Install PTF Twice
Expand Down Expand Up @@ -63,8 +61,8 @@ env:
DEFAULT_ZOWE_EXT_ARTIFACTORY_PATTERN: libs-snapshot-local/org/zowe/{ext-name}/*/{ext-name}-*.pax

# below block can be overwritten, adjusted by DevOps only
# ZOS_NODE_VERSION more to choose from: v14.21.3.1, v16.20.1
ZOS_NODE_VERSION: v14.21.3.1
# ZOS_NODE_VERSION more to choose from: v16.20.1, v18.16.0
ZOS_NODE_VERSION: v16.20.1
CLIENT_NODE_VERSION: v12.18.3
INSTALL_TEST_DEBUG_INFORMATION: zowe-install-test:*
SANITY_TEST_DEBUG_INFORMATION: zowe-sanity-test:*
Expand All @@ -75,8 +73,6 @@ env:
SMPE_PTF_TESTFILE: basic/install-ptf.ts
EXTENSIONS_TESTFILE: basic/install-ext.ts
KEYRING_TESTFILE: extended/keyring.ts
ZOS_NODE_V14_TESTFILE: extended/node-versions/node-v14.ts
ZOS_NODE_V16_TESTFILE: extended/node-versions/node-v16.ts
ZOS_NODE_V18_TESTFILE: extended/node-versions/node-v18.ts
NON_STRICT_VERIFY_EXTERNAL_CERTIFICATE_TESTFILE: extended/certificates/nonstrict-verify-external-certificate.ts
INSTALL_PTF_TWICE_TESTFILE: extended/install-ptf-two-times.ts
Expand All @@ -85,7 +81,7 @@ env:
CONFIG_MANAGER_TESTFILE: extended/config-manager/enable-config-manager.ts
GENERAL_API_DOCUMENTATION_TESTFILE: basic/install-api-gen.ts
ZOWE_NIGHTLY_TESTS_FULL: basic/install.ts(all);basic/install-ptf.ts(all)
ZOWE_RELEASE_TESTS_FULL: basic/install.ts(all);basic/install-ptf.ts(all);basic/install-ext.ts(any);extended/keyring.ts(all);extended/node-versions/node-v14.ts(any);extended/node-versions/node-v16.ts(any);extended/node-versions/node-v18.ts(zzow04):extended/certificates/nonstrict-verify-external-certificate.ts(any);extended/caching-storages/infinispan-storage.ts(any);extended/config-manager/enable-config-manager.ts(any)
ZOWE_RELEASE_TESTS_FULL: basic/install.ts(all);basic/install-ptf.ts(all);basic/install-ext.ts(any);extended/keyring.ts(all);extended/node-versions/node-v18.ts(zzow08):extended/certificates/nonstrict-verify-external-certificate.ts(any);extended/caching-storages/infinispan-storage.ts(any);extended/config-manager/enable-config-manager.ts(any)

jobs:
display-dispatch-event-id:
Expand Down Expand Up @@ -137,7 +133,7 @@ jobs:
- name: '[Prep 2] Setup Node'
uses: actions/setup-node@v3
with:
node-version: '14'
node-version: '18'

- name: '[Prep 3] Cache node modules'
uses: actions/cache@v2
Expand Down Expand Up @@ -293,12 +289,6 @@ jobs:
case 'Keyring':
eta = 53
break;
case 'z/OS node v14':
eta = 45
break;
case 'z/OS node v16':
eta = 45
break;
case 'z/OS node v18':
eta = 45
break;
Expand Down
Loading

0 comments on commit 426e2b0

Please sign in to comment.