Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,619 changes: 1,615 additions & 4 deletions process/general_concepts/_assets/score_building_blocks_meta_model.drawio.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to start with big letter, Platform Integration Tests, all other start with big letters to

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
26 changes: 13 additions & 13 deletions process/general_concepts/score_building_blocks_concept.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ The following figure gives an overview about the building blocks of the project
(blue box, top, 2nd column). The objectives of the project platform include
enabling of feature integration for different use cases and domains. This includes safety-critical
applications. Thus the intention is, that the platform can be developed as a Safety
Element out of Context (**SEooC**) (green box, top, 1 column). The objectives of the platform are
Element out of Context (**SEooC**) (green box, top, 1st column). The objectives of the platform are
expressed as concrete **Stakeholder Requirements** (blue box, top, 2nd column), which can be tested
by provided **Platform Tests** (blue box, top, 5nd column) for reference hardware platforms. The
by provided **Platform Integration Tests** (blue box, top, 5th column) for reference hardware platforms. The
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use that form all over the changed documents and images

platform consists of **Features** (yellow box, middle, 2nd column).

Further the project provides **Software Modules** (red box, middle, 1st column), which can also be
Expand All @@ -37,28 +37,28 @@ highest level component in our model. A Software Module represents e.g. executab

Components are the major building blocks of the platform. Components consists of **Units**
(grey box, bottom, 2nd column), the lowest level in our model. The represent the source code,
which implements the Unit. Units has a **Detailed Design** (grey box, middle, 3nd column), which is
also implemented by the **Source Code** (grey box, bottom, 3nd column). The Detailed Design is
verified by **Unit Tests** (grey box, middle, 5nd column).
which implements the Unit. Units has a **Detailed Design** (grey box, middle, 3rd column), which is
also implemented by the **Source Code** (grey box, bottom, 3rd column). The Detailed Design is
verified by **Unit Tests** (grey box, middle, 5th column).

Components are defined by **Component Requirements** (green box, middle, 3nd column) and the
**Component Architecture** (green box, middle, 4nd column). A **Component Safety Analysis**
(green box, middle, 6nd column) is required to verify the Component Architecture, whereby
Components are defined by **Component Requirements** (green box, middle, 3rd column) and the
**Component Architecture** (green box, middle, 4th column). A **Component Safety Analysis**
(green box, middle, 6th column) is required to verify the Component Architecture, whereby
violations of safety requirements must be documented. Potential faults
may mitigated by updating the Component Requirements or by the **Component Assumptions of Use**
(green box, middle, 8nd column). The latter one must be considered by the user of the Component.
**Component Tests** (green box, middle, 5nd column) verify the Component requirements, and
**Component Integration Tests** (green box, middle, 7nd column) verify the Component Architecture
**Component Tests** (green box, middle, 5th column) verify the Component requirements, and
**Component Integration Tests** (green box, middle, 7th column) verify the Component Architecture
as well as the Integration of multiple Units to a Component.

As mentioned above a Software Module is defined as a Component or a set of components realizing a
Feature of the platform. Features consists of Components and are defined by **Feature Requirements**
(yellow box, middle, 3nd column) and the **Feature Architecture** (yellow box, middle, 4nd column).
A **Feature Safety Analysis** (yellow box, middle, 6nd column) is required to verify the Feature
(yellow box, middle, 3rd column) and the **Feature Architecture** (yellow box, middle, 4th column).
A **Feature Safety Analysis** (yellow box, middle, 6th column) is required to verify the Feature
Architecture, whereby violations of safety requirements must be documented. Potential
faults may mitigated by updating the Feature Requirements or by the
**Feature Assumptions of Use** (yellow box, middle, 8nd column). The latter one must be considered
by the user of the Feature. **Feature Integration Tests** (yellow box, middle, 5nd column) verify
by the user of the Feature. **Feature Integration Tests** (yellow box, middle, 5th column) verify
the Feature Requirements and the Feature Architecture as well as the Integration of multiple Units
to a Component.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ deviation of an expected result.
Examples for problem origins:

* Deviations relating to requirements, architecture, implementation or code found by user of the platform.
* Deviations found by contributor based on component, feature or platform integration tests.
* Deviations found by contributor based on component, feature or Platform Integration Tests.
* Deviations found by Quality Management activities as defined in the Quality Management Plan.

Therefore guidelines :need:`gd_temp__problem_template`,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,10 +65,10 @@ Guideline Quality Management Plan
|
| **Planning integration and verification:**
| Integration on the target hardware is not done in the scope of the SW platform project, but SW/SW integration up to the feature level is performed and its test results are part of the :need:`wp__verification_platform_ver_report`.
| The integration on the target hardware, done by the distributor or OEM, is supported by delivering a set of HW/SW feature integration and platform tests which were already run successfully on a reference HW platform.
| The integration on the target hardware, done by the distributor or OEM, is supported by delivering a set of HW/SW feature integration and Platform Integration Tests which were already run successfully on a reference HW platform.
| This is planned by the respective workproducts:
| * :need:`wp__verification_feat_int_test`
| * :need:`wp__verification_platform_test`
| * :need:`wp__verification_platform_int_test`
| Verification planning is documented in :need:`wp__verification_plan`
|
| **Scheduling of audits, conformance checks, work product reviews, release verification and approval:**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ Safety Management Guideline
| The integration on the target hardware done by the distributor or OEM is supported by delivering a set of HW/SW integration tests which were already run successfully on a reference HW platform.
| This is planned by the respective work products:
| * :need:`wp__verification_feat_int_test`
| * :need:`wp__verification_platform_test`
| * :need:`wp__verification_platform_int_test`
| Verification planning is documented in :need:`wp__verification_plan`
|
| **Scheduling of confirmation reviews, audit and assessment:**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ Security Management Guideline
This is planned by the respective work products:

* :need:`wp__verification_feat_int_test`
* :need:`wp__verification_platform_test`
* :need:`wp__verification_platform_int_test`

Verification planning is documented in :need:`wp__verification_plan`

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Test Linking to Requirements
:id: gd_req__verification_link_tests
:status: valid
:tags: prio_1_automation, verification
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_test
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_int_test
:complies: std_req__iso26262__support_6432


Expand Down Expand Up @@ -54,7 +54,7 @@ Test Linking to Requirements
:id: gd_req__verification_link_tests_cpp
:status: valid
:tags: prio_1_automation, verification
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_test
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_int_test
:complies: std_req__iso26262__support_6432


Expand Down Expand Up @@ -87,21 +87,21 @@ Test Linking to Requirements
:id: gd_req__verification_link_tests_python
:status: valid
:tags: prio_1_automation, verification
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_test
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_int_test
:complies: std_req__iso26262__support_6432


For linking python tests to requirements **metadata** shall be used.
For this the 'add_test_properties' decorator has been provided.
For linking python tests to requirements **metadata** shall be used.
For this the 'add_test_properties' decorator has been provided.
You need to add it to the test and fill out:
* partially_verifies OR fully_verifies
* test_type

* partially_verifies OR fully_verifies
* test_type
* derivation_technique

For allowed values for test_type & derivation_technique please check :need:`gd_req__verification_link_tests`
Further more, this decorator will also check if your test has a `docstring` which should act as the description of the test.


.. code-block:: python

Expand All @@ -118,7 +118,7 @@ Test Linking to Requirements
:id: gd_req__verification_link_tests_rust
:status: valid
:tags: prio_1_automation, verification
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_test
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_int_test
:complies: std_req__iso26262__support_6432

For linking Rust tests to requirements **#[record_property]** shall be used:
Expand All @@ -141,7 +141,7 @@ Test Linking to Requirements
:id: gd_req__verification_independence
:status: valid
:tags: done_automation, verification
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_test
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_int_test
:complies:

The approver of a pull request shall differ from the author(s) of the pull request in all pull requests.
Expand Down Expand Up @@ -171,14 +171,14 @@ Test Linking to Requirements
:id: gd_req__verification_checks
:status: valid
:tags: prio_1_automation, verification
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_test
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_int_test
:complies:

The following checks shall be implemented on test metadata:

- TestType and DerivationTechnique shall be set
- Description shall not be empty
- In a Platform Test Partially/FullyVerifies shall be set to a Platform Requirement
- In a Platform Integration Test Partially/FullyVerifies shall be set to a Platform Requirement
- If Partially/FullyVerifies are set in Feature Integration Test these shall link to Feature Requirements
- If Partially/FullyVerifies are set in Component Integration Test these shall link to Component Requirements
- If Partially/FullyVerifies are set in Unit Test these shall link to Component Requirements
Expand All @@ -187,7 +187,7 @@ Test Linking to Requirements
:id: gd_req__verification_checks_extended
:status: draft
:tags: verification
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_test
:satisfies: wf__verification_unit_test, wf__verification_comp_int_test, wf__verification_feat_int_test, wf__verification_platform_int_test
:complies:

The following checks shall be implemented on test metadata:
Expand Down
4 changes: 2 additions & 2 deletions process/process_areas/verification/verification_concept.rst
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ Following aspect should be considered when developing test cases:
For integration testing, the ITF (Integration Test Framework) is used.

For more information, see ``[TODO: Link to ITF documentation once available. Related feature request is #599]``.
* **Platform Testing:** Test the platform with configured features as a whole.
* **Platform Integration Testing:** Test the platform with configured features as a whole.
* **Regression Testing:** Ensure that changes do not introduce new defects.
Automate regression tests where possible as they will get executed as part of the CI.
* **Performance Testing (when applicable):** Evaluate the performance characteristics of the code,
Expand All @@ -173,7 +173,7 @@ integration tests linking to the component requirements and architecture.
Traceability of feature integration tests shall be established through linking those test cases to
feature requirements and architecture as features describe the integrated behavior of all components.

Traceability of platform tests shall be established through linking those test cases to
Traceability of platform integration tests shall be established through linking those test cases to
stakeholder requirements as stakeholder requirements describe the platform behavior.

Note that all the above tests shall only link to requirements of type "Functional" and "Interface".
Expand Down
12 changes: 6 additions & 6 deletions process/process_areas/verification/workflows.rst
Original file line number Diff line number Diff line change
Expand Up @@ -90,20 +90,20 @@ Workflow Verification
The tests are automatically executed as part of the CI after PR merge.
In case of changes at inputs, the workflow need to be executed again as part of maintenance.

.. workflow:: Create/Maintain Platform Test
:id: wf__verification_platform_test
.. workflow:: Create/Maintain Platform Integration Test
:id: wf__verification_platform_int_test
:status: valid
:tags: verification
:responsible: rl__contributor
:approved_by: rl__committer, rl__testing_community
:supported_by: rl__safety_manager
:input: wp__requirements_stkh, wp__verification_plan
:output: wp__verification_platform_test
:output: wp__verification_platform_int_test
:contains: gd_req__verification_link_tests, gd_req__verification_link_tests_cpp, gd_req__verification_link_tests_python, gd_req__verification_link_tests_rust, gd_req__verification_independence, gd_guidl__verification_specification
:has: doc_concept__verification_process, doc_getstrt__verification_process

Platform test cases are based on Stakeholder requirements. This is the highest test level.
Any contributor can create a platform test and create a PR for it.
Platform Integration Test cases are based on Stakeholder requirements. This is the highest test level.
Any contributor can create a platform integration test and create a PR for it.
During the review process the test cases will be approved by a committer.
Committer and contributor need to differ.
The tests are automatically executed as part of the CI after PR merge.
Expand Down Expand Up @@ -183,7 +183,7 @@ Workflow Verification
wp__feature_arch, wp__platform_sw_release_note, wp__platform_mgmt,
wp__feature_fmea, wp__feature_dfa, wp__platform_dfa,
wp__sw_arch_verification, wp__requirements_inspect,
wp__verification_feat_int_test, wp__verification_platform_test
wp__verification_feat_int_test, wp__verification_platform_int_test
:output: wp__verification_platform_ver_report
:contains: gd_temp__platform_ver_report
:has: doc_concept__verification_process, doc_getstrt__verification_process
Expand Down
10 changes: 5 additions & 5 deletions process/process_areas/verification/workproducts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,12 @@ Platform
technology maturity or risks.
This also covers the work product Verification Specification.

.. workproduct:: Platform test
:id: wp__verification_platform_test
.. workproduct:: Platform Integration Test
:id: wp__verification_platform_int_test
:status: valid
:complies: std_wp__iso26262__support_952

Platform Testing verifies Stakeholder Requirements performed on reference HW.
Platform Integration Testing verifies Stakeholder Requirements performed on reference HW.
Depending on the nature of the project, respective tailoring (e.g. for reduced requirements
coverage) has to be reflected in the :need:`wp__verification_plan` and :need:`wp__platform_safety_plan`.

Expand All @@ -54,10 +54,10 @@ Platform
- Formal evidence about the performed DFA
- Formal evidence about the performed Safety Analyses
- Test result per test case from
:need:`wp__verification_platform_test` and :need:`wp__verification_feat_int_test`
:need:`wp__verification_platform_int_test` and :need:`wp__verification_feat_int_test`
with status passed/failed/not_run
- Test log per test case from
:need:`wp__verification_platform_test` and :need:`wp__verification_feat_int_test`
:need:`wp__verification_platform_int_test` and :need:`wp__verification_feat_int_test`
with status passed/failed/not_run

Feature
Expand Down
1,589 changes: 1,585 additions & 4 deletions process/workproducts/_assets/wp_traceability_model.drawio.svg
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, please use same style as others, Platform Integration Test

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading