Add `categorize` decorator for controlling when tests run by berquist · Pull Request #1573 · sstsimulator/sst-core

berquist · 2026-03-18T18:44:53Z

Closes #1541

github-actions · 2026-03-18T18:46:05Z

CLANG-FORMAT TEST - PASSED

github-actions · 2026-03-31T18:14:48Z

CLANG-FORMAT TEST - PASSED

sst-autotester · 2026-03-31T18:22:59Z

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request.

github-actions · 2026-03-31T18:25:38Z

CLANG-FORMAT TEST - PASSED

sst-autotester · 2026-03-31T18:26:15Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements

Build Num: 2583
Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MR-2

Build Num: 2528
Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MT-2

Build Num: 2526
Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-core_Make-Dist

Build Num: 1089
Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-15-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-elements

Build Num: 1076
Status: STARTED

Using Repos:

Repo: CORE (berquist/sst-core)

Repo: SQE (sstsimulator/sst-sqe)

Repo: ELEMENTS (sstsimulator/sst-elements)

Repo: MACRO (sstsimulator/sst-macro)

Pull Request Author: berquist

berquist · 2026-03-31T18:57:38Z

This will remain draft until updates to SQE are made and confirmed working. This PR isn't strictly dependent on those changes: testing wouldn't be broken, but nightly tests would be unintentionally skipped.

Since the nightly tests are no longer derived from an automatically-set environment variable in Jenkins, the --categorize nightly needs to be passed to sst-test-{core,elements}. In order to not change the bamboo.sh command line, it will probably need to read from BUILD_TAG, which is how the test framework currently implements the nightly check.

sst-autotester · 2026-03-31T19:00:28Z

Status Flag 'Pull Request AutoTester' - ERROR: A user has committed a change to the PR before testing completed. The testing SHA = da57595 Does not match the current commit SHA = 0cc06a7. Testing of this PR must occur again.

berquist · 2026-03-31T19:50:16Z

This will remain draft until updates to SQE are made and confirmed working. This PR isn't strictly dependent on those changes: testing wouldn't be broken, but nightly tests would be unintentionally skipped.

I see bamboo reads from SST_TEST_FRAMEWORKS_PARAMS, so I'll try that, and only changes to Jenkins instead of bamboo might be necessary.

sst-autotester · 2026-03-31T20:27:25Z

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request.

sst-autotester · 2026-03-31T20:30:12Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements

Build Num: 2585
Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MR-2

Build Num: 2530
Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MT-2

Build Num: 2528
Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-core_Make-Dist

Build Num: 1090
Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-15-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-elements

Build Num: 1078
Status: STARTED

Using Repos:

Repo: CORE (berquist/sst-core)

Repo: SQE (sstsimulator/sst-sqe)

Repo: ELEMENTS (sstsimulator/sst-elements)

Repo: MACRO (sstsimulator/sst-macro)

Pull Request Author: berquist

sst-autotester · 2026-03-31T21:02:34Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements

Build Num: 2585
Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MR-2

Build Num: 2530
Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MT-2

Build Num: 2528
Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-core_Make-Dist

Build Num: 1090
Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-15-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-elements

Build Num: 1078
Status: PASSED

sst-autotester · 2026-03-31T21:02:52Z

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
WARNING: NO REVIEWERS HAVE BEEN REQUESTED FOR THIS PULL REQUEST!

sst-autotester · 2026-03-31T21:02:59Z

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

…ng.py

…d line

github-actions · 2026-03-31T22:01:26Z

CLANG-FORMAT TEST - PASSED

berquist · 2026-03-31T22:09:06Z

I've confirmed via AnyBranch that:

With this PR, export SST_TEST_FRAMEWORKS_PARAMS="--categories nightly" does run all the nightly tests, without the decorator, because the logic of testing_check_is_nightly was updated to use the same category logic as the decorator.
Without this PR, adding that shell export causes an unrecognized argument error from argparse. This means that the nightly Jenkins jobs can't be updated for these changes until after the PR is merged.

sst-autotester · 2026-03-31T22:39:27Z

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request.

sst-autotester · 2026-03-31T22:42:41Z

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements

Build Num: 2587
Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MR-2

Build Num: 2532
Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MT-2

Build Num: 2530
Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-core_Make-Dist

Build Num: 1092
Status: STARTED

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-15-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-elements

Build Num: 1080
Status: STARTED

Using Repos:

Repo: CORE (berquist/sst-core)

Repo: SQE (sstsimulator/sst-sqe)

Repo: ELEMENTS (sstsimulator/sst-elements)

Repo: MACRO (sstsimulator/sst-macro)

Pull Request Author: berquist

sst-autotester · 2026-03-31T23:12:55Z

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements

Build Num: 2587
Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MR-2

Build Num: 2532
Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MT-2

Build Num: 2530
Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-core_Make-Dist

Build Num: 1092
Status: PASSED

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-15-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-elements

Build Num: 1080
Status: PASSED

sst-autotester · 2026-03-31T23:13:12Z

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO REVIEWS HAVE BEEN PERFORMED ON THIS PULL REQUEST!

sst-autotester · 2026-03-31T23:13:19Z

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

sst-autotester · 2026-04-01T03:22:53Z

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

sst-autotester · 2026-04-01T08:07:55Z

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

berquist added the SST-Test Frameworks label Mar 18, 2026

github-actions bot added AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) AT: CLANG-FORMAT PASS and removed AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) labels Mar 18, 2026

berquist added the AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) label Mar 18, 2026

github-actions bot removed the AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) label Mar 31, 2026

github-actions bot added AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) and removed AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) labels Mar 31, 2026

berquist added 7 commits March 31, 2026 15:36

Define testing categories and add to command line options

cbde67b

Update calls to get include file values in testsuite_testengine_testi…

9fd42c8

…ng.py

Differentiate between predefined and custom testing categories

7dfb38a

Add and test categorize wrapper

53d7ffb

Fix arguments passed in test wrapper

b39ceff

Add examples of using categorize wrapper in its unit test

2359e3b

Change implementation of testing_check_is_nightly to read from comman…

c17bda9

…d line

berquist force-pushed the 1541-markers-for-test-categorization branch from 0cc06a7 to c17bda9 Compare March 31, 2026 22:00

github-actions bot added the AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) label Mar 31, 2026

github-actions bot removed the AT: WIP Mark PR as a Work in Progress (No Autotesting Performed) label Mar 31, 2026

berquist marked this pull request as ready for review March 31, 2026 22:04

berquist requested review from gvoskuilen and jmlapre March 31, 2026 22:05

berquist added this to the SST V16.0.0 milestone Mar 31, 2026

Conversation

berquist commented Mar 18, 2026

Uh oh!

github-actions bot commented Mar 18, 2026

Uh oh!

github-actions bot commented Mar 31, 2026

Uh oh!

sst-autotester commented Mar 31, 2026

Uh oh!

github-actions bot commented Mar 31, 2026

Uh oh!

sst-autotester commented Mar 31, 2026

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MR-2

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MT-2

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-core_Make-Dist

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-15-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-elements

Using Repos:

Uh oh!

berquist commented Mar 31, 2026

Uh oh!

sst-autotester commented Mar 31, 2026

Uh oh!

berquist commented Mar 31, 2026

Uh oh!

sst-autotester commented Mar 31, 2026

Uh oh!

sst-autotester commented Mar 31, 2026

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MR-2

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MT-2

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-core_Make-Dist

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-15-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-elements

Using Repos:

Uh oh!

sst-autotester commented Mar 31, 2026

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MR-2

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MT-2

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-core_Make-Dist

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-15-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-elements

Uh oh!

sst-autotester commented Mar 31, 2026

Uh oh!

sst-autotester commented Mar 31, 2026

Uh oh!

github-actions bot commented Mar 31, 2026

Uh oh!

berquist commented Mar 31, 2026

Uh oh!

sst-autotester commented Mar 31, 2026

Uh oh!

sst-autotester commented Mar 31, 2026

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MR-2

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-elements_MT-2

Build Information

Test Name: SST__AutotestGen2_NewFW_sst-test_OMPI-4.1.4_PY3.9_sst-core_Make-Dist

Build Information

Test Name: SST__AutotestGen2_NewFW_OSX-15-XC15-ARM2_OMPI-4.1.6_PY3.10_sst-elements

Using Repos:

Uh oh!