Zowe System Tests Plan

[RASakach]

TSC agreed to discuss & investigate whether or not each of the Zowe squads has a satisfactory inventory of SYSTEM TEST. The outcome of the discussion should include the following:

1. An EPIC that all squads can leverage to track work related to this item
2. A roadmap item that describes the goal and strategy for achieving MVP System Tests
3. A document that describes current process and dependencies (i.e. are we depending on Vendor hardware vs. Community Hardware)?
4. A method of tracking tech-debt items that need to be addressed by squad (i.e. upgrade to current release of Java, Node etc.)

[balhar-jakub]

The topic is linked to the topic of pre-requisites and supporting them in this issue - #1978

[balhar-jakub]

As Zowe, we have limited means to verify all the potential configurations and combinations of the hardware and software all the user have installed.

System tests suite should be:

• a way to validate that Zowe works properly for us as a Zowe in our nightly pipelines
• for vendors to validate the same on further combinations of hardware and software
• for users to validate that Zowe is properly installed and configured

With such a suite we can properly validate support for new technologies that we want to support e.g. new z/OS, Java, Node.js versions and others. On top of that the support providers can help assure the customers that the Zowe will run on any combination of hardware and software they may need.

To achieve this we need to:

• define key functionality for Zowe including client and server side
• Each squad define a suite testing key functionalities

The current state:

• We have a suite of a few tests for sanity and installation checking https://github.com/zowe/zowe-install-packaging/tree/v2.x/staging/tests
• We don't have on a per project basis tests that would validate the integrated functionality

Proposed steps:

• Once we release V3 change this issue to Epic for tracking the problem.
• Create issue for the Zowe wide key functionality and build it within the architecture call.
• Create issue for every Zowe squad with project that is in Core and let them provide the test suite covering the key functionality.

Long-term work

• Push on the OMP to provide more hardware and software to have more testing in open.